Jun 13, 2017

Interactive Notebooks - Part 2: Getting Advanced with nteract

As mentioned in my introductory post on the topic, while doing research for another article, I dug into nteract pretty deeply. It’s how I like to learn about new things. nteract ships with the ability to run JavaScript code in notebooks, and I was wondering if it had the ability to do more than that. What I discovered revealed an entirely new level of awesome for using interactive notebooks.

Under the Covers

nteract is built on top of the Jupyter family of tools. Jupyter started out its life as IPython, which is an interactive notebook engine that’s become pretty popular with data scientists, especially those who are working with Python.

The magic of Jupyter is in its use of “kernels.” Each cell gets executed by a kernel. The IPython kernel is probably the one that’s the most used. Much of the Jupyter documentation is written with the assumption that the IPython kernel is being used.

nteract only ships with a Node.js-based JavaScript kernel. That selection of kernel makes sense when you consider nteract’s architecture. nteract is built on top of Electron, which is a cross-platform framework for building desktop applications using Node.js and friends. The entire user interface of nteract is executing inside of Node.js already, so it’s only natural to assume that the authors would try to take advantage of that environment to execute code inside the notebooks.

Aside: Why did it take me so long to start using interactive notebooks?

I’d heard about it, but hadn’t given it an honest try yet.

I haven’t been tinkering with Python a ton recently. Ruby is still the language that’s quickest under my fingers, and it tends to be the tool that I reach for when I want to solve a simple problem. I’ve yet to take the plunge to force myself to solve those same problems with Python or another language (there are so many to choose from).

I’d initially dismissed Jupyter as a Python-only tool. That was until I spoke at the DC-Baltimore Perl Workshop in 2015 (DCBPW2015). I’d met a few people at a Perl conference who were considering building out the ability to execute Perl code from inside a Jupyter notebook. That got me thinking about the ability to do the same for Ruby. After I dug in and learned more, I wasn’t all that impressed with the tooling available for Jupyter. It ran as a server in the web browser, and that setup didn’t feel like it belonged as part of my normal workflow. When I think about creating a new document, I don’t often reach for my web browser.

So I decided to shelve Jupyter and tools for a while. About a month after DCBPW2015, I noticed in a blog post that GitHub added support for rendering Jupyter notebooks. I thought that was awesome, but still not enough for me to take the plunge and learn how to take advantage of the tool.

It wasn’t until I discovered nteract that I became an instant convert. Being able to open Jupyter notebooks using a text editor that understands them is awesome, and it fits in much better with my writing workflow.

Installing Additional Kernels

I’ve taken the time to get a few additional kernels installed on my computer, and I’ve been able to use all of them successfully with nteract. Getting that configured correctly wasn’t obvious, and as a Jupyter newcomer, I thought others might find it helpful if I documented the steps that I took to get here.

Step zero: Get a python environment setup

This could be an entire article on its own, and maybe it will be one day. For now, I’m going to assume that you’re using pyenv and that you have it set to use a recent version of Python 3. Here’s how pyenv is set up on my computer.

%%bash
pyenv versions

  system
  2.7.11
  3.1.5
  3.2.6
  3.3.6
  3.4.3
* 3.5.1 (set by /Users/mscottford/.python-version)

Step one: Install Jupyter

Once Python’s set up correctly, the next thing to do is install Jupyter via pip. And you’ll need Jupyter installed before you can correctly hook up any other kernels with nteract. I found that out the hard way, so don’t skip this step.

%%bash
pip install jupyter

Requirement already satisfied: jupyter in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages
Requirement already satisfied: ipywidgets in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: notebook in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: jupyter-console in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: nbconvert in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: ipykernel in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: qtconsole in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter)
Requirement already satisfied: ipython>=4.0.0 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipywidgets->jupyter)
Requirement already satisfied: widgetsnbextension>=1.2.6 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipywidgets->jupyter)
Requirement already satisfied: traitlets>=4.2.1 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipywidgets->jupyter)
Requirement already satisfied: jupyter-client in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: tornado>=4 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: terminado>=0.3.3; sys_platform != "win32" in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: nbformat in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: jinja2 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: ipython-genutils in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: jupyter-core in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from notebook->jupyter)
Requirement already satisfied: pygments in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter-console->jupyter)
Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.0 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter-console->jupyter)
Requirement already satisfied: bleach in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied: testpath in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied: pandocfilters>=1.4.1 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied: mistune!=0.6 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied: entrypoints>=0.2.2 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbconvert->jupyter)
Requirement already satisfied: decorator in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: simplegeneric>0.8 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: pickleshare in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: appnope; sys_platform == "darwin" in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: setuptools>=18.5 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: pexpect; sys_platform != "win32" in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: six in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from traitlets>=4.2.1->ipywidgets->jupyter)
Requirement already satisfied: pyzmq>=13 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter-client->notebook->jupyter)
Requirement already satisfied: python-dateutil>=2.1 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jupyter-client->notebook->jupyter)
Requirement already satisfied: ptyprocess in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from terminado>=0.3.3; sys_platform != "win32"->notebook->jupyter)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from nbformat->notebook->jupyter)
Requirement already satisfied: MarkupSafe>=0.23 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from jinja2->notebook->jupyter)
Requirement already satisfied: wcwidth in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter)
Requirement already satisfied: html5lib!=0.9999,!=0.99999,<0.99999999,>=0.999 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from bleach->nbconvert->jupyter)
Requirement already satisfied: packaging>=16.8 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from setuptools>=18.5->ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: appdirs>=1.4.0 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from setuptools>=18.5->ipython>=4.0.0->ipywidgets->jupyter)
Requirement already satisfied: pyparsing in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from packaging>=16.8->setuptools>=18.5->ipython>=4.0.0->ipywidgets->jupyter)

Your output is going to look different than mine, because I already have that package installed from earlier.

Step two: Install the IPython kernel

Next up is to install the ipykernel. The pip package for ipykernel should already be installed from when you ran the pip command to install Jupyter, so you should just have to run:

%%bash
python -m ipykernel install --user

Installed kernelspec python3 in /Users/mscottford/Library/Jupyter/kernels/python3

Now, you should be able to open nteract and you’ll see “Python 3” has a choice under the “Languages” dropdown. Note: If you’ve already got nteract open, you’re going to have to close it before “Python 3” shows up under the languages menu.

Step two-point-one: Play around with the IPython kernel

The IPython kernel isn’t the first one that I installed. It wasn’t even the second. But now, after playing around, I think it should be the first one that you install even if you’re not planning on using Python from inside your notebook all that much. We’ll get to installing additional kernels next, but with the IPython kernel installed, I almost don’t need the other kernels installed.

Magic `%%` commands

I discovered magic commands while trying to figure out if there was a kernel for bash, there is, and there are instructions for how to install it below. That’s when I found an article that talked about using %%bash to execute bash commands. That’s how all of the bash commands have been executed in this document. I’m writing this using nteract with the IPython kernel selected.

When to use the other kernels?

I’m thinking that my workflow is going to start with the IPython kernel, because a lot of the writing that I do is in a polyglot setting. I really like the ability to mix and match bash commands, javascript code and ruby code all in the same document.

%%ruby
puts "This is ruby version #{RUBY_VERSION}."

This is ruby version 2.3.1

%%perl
printf "This is perl version $^V.";

This is perl version v5.20.2.

import sys
print('This is python version', sys.version)

This is python version 3.5.1 (default, Jan 18 2016, 09:49:54)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]

%%bash
echo This is bash version $BASH_VERSION.

This is bash version 4.4.12(1)-release.

If I happen to get to the bottom of a document and I discover that all of the code blocks are executed in the same language, then I’ll take the time to switch to that kernel, so that the %% magic command syntax doesn’t pollute things.

Step three: Install IRuby

To get IRuby working, here are the steps that I followed:

%%bash
brew install libtool autoconf automake autogen

==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core, homebrew/science).
==> Updated Formulae
aws-sdk-cpp
awscli
youtube-dl



Updating Homebrew...
Warning: libtool-2.4.6_1 already installed
Warning: autoconf-2.69 already installed
Warning: automake-1.15 already installed
Warning: autogen-5.18.7 already installed

%%bash
gem install cztop
gem install rbczmq
gem install iruby

Successfully installed cztop-0.11.4
1 gem installed
Building native extensions.  This could take a while...
Successfully installed rbczmq-1.7.9
1 gem installed
Consider installing the optional dependencies to get additional functionality:
  * pry
  * pry-doc
  * awesome_print
  * gnuplot
  * rubyvis
  * nyaplot

Successfully installed iruby-0.2.9
1 gem installed

%%bash
iruby register --force

%%bash
jupyter kernelspec list

Available kernels:
  ruby       /Users/mscottford/.ipython/kernels/ruby
  bash       /Users/mscottford/Library/Jupyter/kernels/bash
  python3    /Users/mscottford/Library/Jupyter/kernels/python3

Step four: Install the bash kernel

I can see myself using interactive notebooks to show how to set up applications via bash, so I wanted to have a kernel installed that did just that. Here’s how I set that up.

%%bash
pip install bash_kernel

Requirement already satisfied: bash_kernel in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages
Requirement already satisfied: pexpect>=4.0 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from bash_kernel)
Requirement already satisfied: ptyprocess>=0.5 in ./.pyenv/versions/3.5.1/lib/python3.5/site-packages (from pexpect>=4.0->bash_kernel)

%%bash
python -m bash_kernel.install

Installing IPython kernel spec

%%bash
jupyter kernelspec list

Available kernels:
  ruby       /Users/mscottford/.ipython/kernels/ruby
  bash       /Users/mscottford/Library/Jupyter/kernels/bash
  python3    /Users/mscottford/Library/Jupyter/kernels/python3

Have fun!

Now you’ve got four Jupyter kernels installed that you can interact with inside your interactive notebooks.

Aside: Converting to Markdown

This notebook was converted to markdown by running:

%%bash
jupyter nbconvert --to markdown getting-advanced-with-nteract.ipynb

[NbConvertApp] Converting notebook getting-advanced-with-nteract.ipynb to markdown
[NbConvertApp] Writing 15132 bytes to getting-advanced-with-nteract.md

Want to be alerted when we publish future blogs? Sign up for our newsletter!