IPython

Python Magic on the Command Line and Web

Austin Godber
@godber

DesertPy - 3/26/2014

IPython Installation

  • System Pacakages - sudo apt-get update && sudo apt-get upgrade
  • OS X - Homebrew - brew install ipython
  • PIP - pip install ipython
  • Commercial Python Bundles - anaconda, canopy
  • Open Source Bundles - python(x,y)
  • Use the DesertPy Salt Stack setup - cough not yet

It's not so much about IPython as the other things in your toolkit.

How to Run IPython

Just type:

ipython

IPython REPL

An enhanced command line with these features

+------------------+----------------------------+
| Python Help      | Magic                      |
| Sytem Aliases    | Dynamic Object Information |
| Tab Completion   | Searchable History         |
| Logging          | Session save and restore   |
| System Commands  | Colorized output           |
| Input Caching    | Output Caching             |
+------------------+----------------------------+

IPython REPL - Memorize this now!

  • ?
  • %quickref
  • help
  • object?

did you get it?

  • ?
  • %quickref
  • help
  • object?

Knowing these two is sufficient

  • ?
  • %quickref

? - IPython Introduction

Describes IPython features with hints.

%quickref - Prints quick reference guide

(%q TAB - for the lazy)

IPython REPL Examples - line magic

  • line oriented operation
  • prefixed with % (or not)
  • append ? to see docs on a magic - %reset?

execute a file:

%run file.py

%run reloads from the disk every time so its good for iterative development.

open foo.py in your editor and execute it on exit:

%edit foo.py

save IPython lines 9 through 12 into file2.py:

%save file2.py 9-12

print out all variables in interactive namespace:

%who

print more!!!!!

%whos

reset interactive namespace, clears variables:

%reset

toggle the need for % in front of magics:

%automagic

show all of the many more magics:

%magic

System Commands

  • system commands can be executed by prepending a !
  • output fom these can be stored
!ls
!ls *.py
py_files = !ls *.py

Sample Session

a = range(10)
g = !ping google.com -c 5
g
g. TAB
g.grep('ttl')
who
whos

Magics Sample Session

In [1]: def sq(n):
   ...:     return n*n
   ...:

In [2]: sq(5)
Out[2]: 25

In [4]: %save sq.py 1

In a new session

In [1]: %run sq.py

In [2]: sq(6)
Out[2]: 36

Other IPython CLI Features

  • Embed into your app
  • Demo mode
  • PDB integration

Relax

In [18]: 
from IPython.display import Image
Image(filename="austin-relax.png")
Out[18]:

Yes, thats what I look like when I relax.

The flipside - IPython Notebook

We are going from this ...

In [19]: 
Image(filename="ipython-cli.png")
Out[19]:

to this ...

In [20]: 
Image(filename="ipython-notebook.png")
Out[20]:

IPython Notebook

  • Web based interface to IPython
  • Inline graphics
  • Cell based evaluation
  • Same commands and magics as CLI IPython

Notebook Advantages

  • Web based
  • Remotely (globally) accessible
  • Execution near your data
  • Execution on powerful machine
  • Setup is all on server side

How to run ipython web notebook?

Just type

ipython notebook

or

ipython notebook --ip=0.0.0.0 --port=9999 -no-browser

Cells

  • Basic unit of execution
  • Type Python code into the cell
  • ENTER for newline
  • SHIFT+ENTER to execute whole cell
In [4]: 
# This is a cell
a = 'This '
print a + "is cell output"

This is cell output

Different Cell Types

  • code - in the previous example, executes python code
  • markdown - is a markup language that gets converted to HTML
  • header - HTML Header cells
  • raw - Raw Text too!

IPython Notebook has cell magics

  • similar to the line magics, but works on whole cells
  • prefixed with %%
  • %%capture - capture cell contents
  • %%writefile - write cell contents to file
  • more ...

Execute OTHER interpreters

In [10]: 
%%ruby
puts "Hi #{RUBY_VERSION}"

Hi 1.9.3
In [11]: 
%%bash
echo "hi from $BASH"

hi from /bin/bash

Inline Graphics

To get inline graphics (shown as output in the browser) start each notebook with:

%matplotlib inline

or when you start your notebook server start it with the option:

--pylab=inline

Notebook Examples

Visit the following URL to play along:

http://ipynb.desertpy.com

(active only during this presentation)

A simple plot

In [11]: 
import numpy as np
import matplotlib.pyplot as plt
# don't forget the
# %matplotlib inline
x = np.linspace(0, 10, 20)
y = x ** 2
plt.plot(x,y)
Out[11]:
[<matplotlib.lines.Line2D at 0x3acda10>]

Scikit Image Example

In []: 
# from http://scipy-lectures.github.io/packages/scikit-image/
import skimage
from skimage import io
import os
filename = os.path.join(skimage.data_dir, 'camera.png')
camera = skimage.io.imread(filename)
camera.view()
# image is displayed
In []: 
(3 * camera).view()

My Pandas Presentation

The demo server has my Pandas presentation checked out into it. You can use cd, ls and %load or %run to run my examples.

cd pandas_examples/
ls
%load
# or
%run

Development Tips

  • Link Working directories into your analysis directory
  • Edit your main code with vim and use %run to load it into notebook
  • The notebook is your scratchpad
  • The notebook can be your documentation (literate programming)

Other Notebook Tips

Notebook Conversion - nbconvert

  • A saved notebook is a JSON file ending with .ipynb.
  • It can be converted to other formats with the nbconvert subcommand of ipython (it had been a separate tool but was merged).

Generate HTML

ipython nbconvert mynotebook.ipynb --to html --template basic

Generate LaTeX

ipython nbconvert mynotebook.ipynb --to latex

Generate LaTeX and make a PDF

ipython nbconvert mynotebook.ipynb --to latex --post PDF

Generate Restructured Text

ipython nbconvert mynotebook.ipynb --to rst

You can even make presentations!

ipython nbconvert IPython_Presentation.ipynb --to slides

Check out the source notebook to this very presentation

http://nbviewer.ipython.org/github/desertpy/presentations/blob/master/ipython-godber/IPython_Presentation.ipynb

Remote Access Trick - ngrok

My notebook runs on my desktop but I'm not home?

Make it accessible using ngrok!

Generate a password hash:

>>> from IPython.lib import passwd
>>> passwd()
Enter password:
Verify password:
'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'

Generate the notebook config file

$ ipython profile create nbserver

Place the bash in the config file

# Password to use for web authentication
c = get_config()
c.NotebookApp.password =
u'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'

Launch the Notebook Server:

ipython notebook --profile==nbserver

Serve it publicly with ngrok:

~/bin/ngrok -proto='https' 8888
# now running on https://e082589425.ngrok.com

Thank You!

Austin Godber

@godber

http://desertpy.com/presentations