Jupyter Notebook

Difficulty: beginner

Estimated Time: 10 minutes (to get going)

In this scenario, you will run a Jupyter Notebook in a Docker Container.

The underlying Docker Container Image is jupyter/scipy-notebook. It contains these contents:

Minimally-functional Jupyter Notebook server (e.g., no pandoc for saving notebooks as PDFs)
Miniconda Python 3.x in /opt/conda
Pandoc and TeX Live for notebook document conversion
git, emacs, jed, nano, and unzip
pandas, numexpr, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh, sqlalchemy, hdf5, vincent, beautifulsoup, protobuf, and xlrd packages
ipywidgets for interactive visualizations in Python notebooks
Facets for visualizing machine learning datasets

For details on the Jupyter Docker Stacks libary of container images

You will then add a number of supporting libraries into the running container. This scenario adds these libraries after the container has been started:

Plotly
MatplotLib
Cufflinks
Gender Guesser
NLTK

The scenario also deploys a predefined Jupyter Notebook by cloning a GitHub repository. You are free to add additional notebooks in a similar way - or upload them from your own collection.

Finally, you will access the Jupyter Notebook and start playing with it.

You've completed your Jupyter Notebook scenario!

Your environment is currently being packaged as a Docker container and the download will begin shortly. To run the image locally, once Docker has been installed, use the commands

cat scrapbook_lucasjellema_jupyter-notebook_container.tar | docker load

docker run -it /lucasjellema_jupyter-notebook:

Restart Scenario

Next Scenario

Step 1 of 2

Step 1 - Run the Jupyter Notebook environment

You will now run a Docker container for Jupyter Notebook. Note: this may take up to 3 minutes, because of the size of the container image.

Run the Jupyter Notebook

Run the Jupyter Notebook container image:

docker run -p 8888:8888 -d --name jupyter jupyter/scipy-notebook:83ed2c63671f

Further prepare the container

To prepare the container we will run a script inside the container to install several Python packages

Run this script to execute these steps:

copy the script prepareContainer.sh into the container
copy the script, make the copy executable and then run the script inside the container - this will install several Python packages using pip
restart the container

sh runPrep.sh

Notes on what is happening under the covers

These are the individual steps inside this script. You do not have to execute them - because they are in the runPrep.sh script.

First, copy the script into the container docker cp prepareContainer.sh jupyter:/home/jovyan/prepareContainerRoot.sh

This will copy the local file prepareContainer.sh into the container's directory /home/jovyan as prepareContainerRoot.sh; it will be a root owned file that cannot be run straightaway.

Next, copy the script, make the copy executable and then run the script inside the container: docker exec -d jupyter bash -c 'cp ~/prepareContainerRoot.sh ~/prepareContainer.sh && chmod +x ~/prepareContainer.sh'

Next, run the script inside the container: docker exec -d jupyter sh /home/jovyan/prepareContainer.sh

Finally restart the docker container docker restart jupyter

Step 2 - Access the Jupyter Notebook

Access the Jupyter Notebook

When the container is running, execute this statement: docker logs jupyter

This will show something like:

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=f89b02dd78479d52470b3c3a797408b20cc5a11e067e94b8

The token is the value behind /?token=. You need that for logging in.

Next, you can open the Jupyter Notebook at https://[[HOST_SUBDOMAIN]]-8888-[[KATACODA_HOST]].environments.katacoda.com/

Note: you need the value of the Jupyter Token to login to the environment.

The Notebook opens in a new browser window. You can create a new notebook or open a local one. Check out the local folder work for several sample notebooks. For example: open and run pythonForDataAnalysis.ipynb in the work folder. Or open and run Example_word_clouds.ipynb. Or open folder learn-pandas/lessons and start with Python_101.ipynb or Cookbook - Select.ipynb.

The folder work/Data-Analysis contains many notebooks created by Will Koehrsen, who writes many great articles about Data Science and uses Jupyter Notebooks frequently (see his GitHub Repo: https://github.com/WillKoehrsen/Data-Analysis ).

Interactive Widgets

A nice advanced feature in Jupyter Notebooks are the interactive widgets. To have a quick tour of what these widgets can add to a notebook, open work\widgets\Widgets-Overview.ipynb. The code cell under the Data heading contains an erroneous file reference - or it did when I last checked. Change the contents of the cell to:

df = pd.read_parquet('https://github.com/WillKoehrsen/Data-Analysis/blob/master/medium/data/medium_data_2019_01_26?raw=true')
df.head()

Now the cell will correctly retrieve and process the data.

This article on Medium introduced the interactive widgets demonstrated in this notebook: Interactive Controls for Jupyter Notebooks.

Adding interesting Jupyter Notebooks to the container

You can easily add more notebooks to the container, by cloning them from GitHub straight into the container and subsequently opening them in Jupyter Notebook.

For example - to grab the world's most trivial notebook:

docker exec -it jupyter bash -c 'cd ~/work && git clone https://github.com/Noura/hello-jupyter'

After executing this command, this notebook can be opened in the Jupyter Notebook browser window from the folder work/hello-jupyter.

Terminal

Jupyter Notebook

cd <directory>	Change directory
ls	List directory
echo 'contents' > <file>	Write contents to a file
cat <file>	Output contents of file

i	In Command Mode	Change into Insert Mode, you can now insert and edit text in the file
esc	In Insert Mode	Change into Command Mode, you can now execute commands
:wq	In Command Mode	Save and Exit

Welcome!

Jupyter Notebook

Congratulations!

You've completed the scenario!

Scenario Rating

Steps