Python Magic on the Command Line and Web

Austin Godber

DesertPy - 3/26/2014

IPython Installation

  • System Pacakages - sudo apt-get update && sudo apt-get upgrade
  • OS X - Homebrew - brew install ipython
  • PIP - pip install ipython
  • Commercial Python Bundles - anaconda, canopy
  • Open Source Bundles - python(x,y)
  • Use the DesertPy Salt Stack setup - cough not yet

It's not so much about IPython as the other things in your toolkit.

How to Run IPython

Just type:


IPython REPL

An enhanced command line with these features

| Python Help      | Magic                      |
| Sytem Aliases    | Dynamic Object Information |
| Tab Completion   | Searchable History         |
| Logging          | Session save and restore   |
| System Commands  | Colorized output           |
| Input Caching    | Output Caching             |

IPython REPL - Memorize this now!

  • ?
  • %quickref
  • help
  • object?

did you get it?

  • ?
  • %quickref
  • help
  • object?

Knowing these two is sufficient

  • ?
  • %quickref

? - IPython Introduction

Describes IPython features with hints.

%quickref - Prints quick reference guide

(%q TAB - for the lazy)

IPython REPL Examples - line magic

  • line oriented operation
  • prefixed with % (or not)
  • append ? to see docs on a magic - %reset?

execute a file:

%run file.py

%run reloads from the disk every time so its good for iterative development.

open foo.py in your editor and execute it on exit:

%edit foo.py

save IPython lines 9 through 12 into file2.py:

%save file2.py 9-12

print out all variables in interactive namespace:


print more!!!!!


reset interactive namespace, clears variables:


toggle the need for % in front of magics:


show all of the many more magics:


System Commands

  • system commands can be executed by prepending a !
  • output fom these can be stored
!ls *.py
py_files = !ls *.py

Sample Session

a = range(10)
g = !ping google.com -c 5
g. TAB

Magics Sample Session

In [1]: def sq(n):
   ...:     return n*n

In [2]: sq(5)
Out[2]: 25

In [4]: %save sq.py 1

In a new session

In [1]: %run sq.py

In [2]: sq(6)
Out[2]: 36

Other IPython CLI Features

  • Embed into your app
  • Demo mode
  • PDB integration


In [18]:
from IPython.display import Image

Yes, thats what I look like when I relax.

The flipside - IPython Notebook

We are going from this ...

In [19]:

to this ...

In [20]:

IPython Notebook

  • Web based interface to IPython
  • Inline graphics
  • Cell based evaluation
  • Same commands and magics as CLI IPython

Notebook Advantages

  • Web based
  • Remotely (globally) accessible
  • Execution near your data
  • Execution on powerful machine
  • Setup is all on server side

How to run ipython web notebook?

Just type

ipython notebook


ipython notebook --ip= --port=9999 -no-browser


  • Basic unit of execution
  • Type Python code into the cell
  • ENTER for newline
  • SHIFT+ENTER to execute whole cell
In [4]:
# This is a cell
a = 'This '
print a + "is cell output"
This is cell output

Different Cell Types

  • code - in the previous example, executes python code
  • markdown - is a markup language that gets converted to HTML
  • header - HTML Header cells
  • raw - Raw Text too!

IPython Notebook has cell magics

  • similar to the line magics, but works on whole cells
  • prefixed with %%
  • %%capture - capture cell contents
  • %%writefile - write cell contents to file
  • more ...

Execute OTHER interpreters

In [10]:
puts "Hi #{RUBY_VERSION}"
Hi 1.9.3

In [11]:
echo "hi from $BASH"
hi from /bin/bash

Inline Graphics

To get inline graphics (shown as output in the browser) start each notebook with:

%matplotlib inline

or when you start your notebook server start it with the option:


Notebook Examples

Visit the following URL to play along:


(active only during this presentation)

A simple plot

In [11]:
import numpy as np
import matplotlib.pyplot as plt
# don't forget the
# %matplotlib inline
x = np.linspace(0, 10, 20)
y = x ** 2
[<matplotlib.lines.Line2D at 0x3acda10>]

Scikit Image Example

In []:
# from http://scipy-lectures.github.io/packages/scikit-image/
import skimage
from skimage import io
import os
filename = os.path.join(skimage.data_dir, 'camera.png')
camera = skimage.io.imread(filename)
# image is displayed
In []:
(3 * camera).view()

My Pandas Presentation

The demo server has my Pandas presentation checked out into it. You can use cd, ls and %load or %run to run my examples.

cd pandas_examples/
# or

Development Tips

  • Link Working directories into your analysis directory
  • Edit your main code with vim and use %run to load it into notebook
  • The notebook is your scratchpad
  • The notebook can be your documentation (literate programming)

Other Notebook Tips

Notebook Conversion - nbconvert

  • A saved notebook is a JSON file ending with .ipynb.
  • It can be converted to other formats with the nbconvert subcommand of ipython (it had been a separate tool but was merged).

Generate HTML

ipython nbconvert mynotebook.ipynb --to html --template basic

Generate LaTeX

ipython nbconvert mynotebook.ipynb --to latex

Generate LaTeX and make a PDF

ipython nbconvert mynotebook.ipynb --to latex --post PDF

Generate Restructured Text

ipython nbconvert mynotebook.ipynb --to rst

You can even make presentations!

ipython nbconvert IPython_Presentation.ipynb --to slides

Remote Access Trick - ngrok

My notebook runs on my desktop but I'm not home?

Make it accessible using ngrok!

Generate a password hash:

>>> from IPython.lib import passwd
>>> passwd()
Enter password:
Verify password:

Generate the notebook config file

$ ipython profile create nbserver

Place the bash in the config file

# Password to use for web authentication
c = get_config()
c.NotebookApp.password =

Launch the Notebook Server:

ipython notebook --profile==nbserver

Serve it publicly with ngrok:

~/bin/ngrok -proto='https' 8888
# now running on https://e082589425.ngrok.com

Thank You!

Austin Godber