Jupyter

INSTALL #

Anaconda Way #

This is suggested for very new users to use a stable environment.
It is NOT suitable for normal users / experienced programmers / engineers.

for windows #

Download Python 3 from Anaconda and install.
From Windows “Start” menu, run “Jupyter” directly. (at least since v5.3.1)
Old versions needs to run from navigator as below:

anaconda for linux #

Similar to win, for details, see here.

Original Way Ubuntu 20 #

install pip #

sudo apt-get update && \
sudo apt-get -q install -y python3-pip && \
sudo pip3 install --upgrade pip ; sudo pip install --upgrade pip

install jupyterLab #

// Jupyter is too slow in Chrome nowadays.

Then, JupyterLab with Voila Dashboard is recommended pip install voila

# install jupyter & common libs
sudo -H pip3 install jupyterlab && \
sudo -H pip3 install scipy numpy scikit-learn plotnine matplotlib pandas sympy nose seaborn feather-format notebook ipykernel --pypi-mirror https://pypi.tuna.tsinghua.edu.cn/simple/

To use tsinghua pip source/index:

pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple/

or:

mkdir -p ~/.config/pip/ && \
vim ~/.config/pip/pip.conf

[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple

Or, use -i https://pypi.tuna.tsinghua.edu.cn/simple for every pip cmd, where -i means indexes (of source).

Ubuntu 16 #

(for ubuntu 18, optioanlly remove apt ... ipython-notebook which is deprecated)

Add export PATH=$(echo $PATH | sed 's/\/opt\/conda\/bin//g') to the end of .bashrc, if you avoid conda which is already installed.

# install pip
sudo apt-get update && \
sudo apt-get -q install -y python3-pip && \
sudo -H pip3 install --upgrade pip ; sudo pip install --upgrade pip

# install jupyter & sci-kit
sudo apt-get install -y python-dev python3-dev ipython ipython-notebook && \
sudo -H pip install jupyter; sudo -H pip3 install jupyter && \
sudo -H pip3 install scipy numpy scikit-learn matplotlib pandas sympy nose ggplot

Now, jupyter-notebook uses py3 by default.

To add py2 into jupyter menu:(do NOT use py2 anymore!)

# sudo pip2 install --upgrade pip && \
sudo apt-get -q install -y python-pip && \
sudo python2 -m pip install ipykernel && \
sudo python2 -m ipykernel install --user

(if py3 is lost later for some reason, do the following):

sudo python3 -m pip install jupyter ipykernel
sudo python3 -m ipykernel install --user

ref: stackOverflow

Win #

When having py2, want py3 with jupyter:

pip3 install jupyter
jupyter-notebook.exe

REMOTE JUPYTER IN WEB #

1. setup ssh tunnel (usually 8888): ssh-in-case-of > SSH Tunneling > port forwarding (jupyter exmaple).
2. run jupyter-notebook in server ssh, will see some errors regarding X11 / javascript, ignore them.
3. local browser localhost:80.

WIN BUGS #

Mac: not displaying ggplot erros ref
Win: not displaying ggplot deep erros (ggplot » aex() contains non-existing column name, rstudio can display errors.) (gpu workday » 860_paper_plot.ipynb)

SHORTCUT / KEYMAP #

web (modified to: weidadeyue_jupyter-notebook-cheatsheet-shortcut.xlsx)
customize
totally vim binding (with nix cmd)
for win, run the following: (though output ok, seems not working)

jupyter --data-dir
mkdir -p C:\Users\<user_name>\AppData\Roaming\jupyter\nbextensions
cd C:\Users\<user_name>\AppData\Roaming\jupyter\nbextensions
git clone https://github.com/lambdalisue/jupyter-vim-binding vim_binding
jupyter nbextension enable vim_binding\vim_binding

+ VSCode #

In VSCode: ctrl+shift+p -> “jupyter specify local or remote ….” -> paste remote uri such as “http://localhost:8888/?token=…”.

Later, just drag-drop/open any .ipynb, then VSCode works like a browser with the only difference that the files are local.

[official MS ref.]

+ Pipenv kernels #

To add pipenv based kernels:

# anywhere out of an pipenv
pip3 install ipykernel

# then in a pipenv virtual env
pipenv install ipykernel && \
python3 -m ipykernel install --user --name=<my_virtualenv_name>

# lunch jupyter-lab anywhere outside of pipenv will also include pipenv kernels
jupyter-lab

ref.

For pipenv itself, see learn-python > pipenv & pyenv.

+ R (Original Way) #

See also: install R.
WARN: MUST close other R sessions before installation to update necessary dependencies.
Step 1. install py 3. see: learn-python
Step 2. install jupyter pip3 install jupyter
Step 3. install r packages as below:
Firstly, some system dependencies:

sudo apt-get install -y libssl-dev libssh2-1-dev libcurl4-openssl-dev zlib1g-dev # Ubuntu only: is.gd/JLfcOQ

Then in R (by: sudo -i R)

# run cmd one by one
install.packages(c('devtools', 'digest', 'repr','devtools', 'digest', 'IRdisplay', 'crayon', 'pbdZMQ', 'ggplot', 'IRkernel', 'ggpubr'))
# devtools::install_github('IRkernel/IRkernel')
IRkernel::installspec(name = 'ir402', displayname = 'R_4.0.2')  # choose your own !!!

OBS: the last command should be done by each jupyter user in nix.
To install for more R installations, see IRkernel doc.
Step 4. run jupyter jupyter-notebook.exe

+ R (Anaconda Way) #

Step 1. install miniconda with py 3 (anaconda3 is ok, but too large)

• install for only one user
• do NOT add to system PATH (just keep default)

Step 2. install conda packages
conda install -c r ipython-notebook r-irkernel ipython notebook
Step 3. run
jupyter notebook
ref: Revolution Analytics .com

+ Julia #

Step 1. install Julia
Step 2. install IJulia

.\julia.exe
Pkg.add("IJulia")
Pkg.build("IJulia")

Step 3. run

.\julia.exe
using IJulia
notebook()

or

jupyter notebook

official github ref

Julia + Conda (NOT Suggested) #

conda has no IJulia support yet,
to install julia only:
conda install -c wakari1 julia=0.3.0.git_prerelease
ref

W/ spark #

see spark

+K8s #

JupyterHub on K8S

TIPS #

Listen On All IPs #

jupyter-lab --ip=0.0.0.0

Frequently Used Tools #

supercharging jupyter notebooks

Module Auto-reload #

In ipython notebook (ipynb), put the following code before importing the packages the should be auto-reloaded:

%load_ext autoreload
%autoreload 2

Cell Display Width #

from IPython.core.display import display, HTML
display(HTML("<style>.container { width:80% !important; }</style>")) # Set cell width equal to 'browser width'

Pandas Display #

pd.options.display.max_rows = 6

Table of Contents (TOC) #

nix #

pip show jupyterlab
apt install nodejs npm
jupyter labextension install @jupyterlab/toc --no-build
jupyter lab build

ref: https://github.com/jupyterlab/jupyterlab-toc

dated:

jupyter nbextension install --user https://rawgithub.com/minrk/ipython_extensions/master/nbextensions/toc.js
curl -L https://rawgithub.com/minrk/ipython_extensions/master/nbextensions/toc.css > $(jupyter --data-dir)/nbextensions/toc.css
jupyter nbextension enable toc

win #

jupyter nbextension install --user https://rawgithub.com/minrk/ipython_extensions/master/nbextensions/toc.js

save this link as file: $(jupyter –data-dir)/nbextensions/toc.css

jupyter nbextension enable toc

Git #

pip install --upgrade jupyterlab-git

Collection of Useful Extentions #

ref towardsdatascience (cn)

Extentions can be installed using UI.

Note: maybe better to have virtual env before jupyterhub & jupyterlab lib and those extentions?

• debugger
• TOC
• jupyterlab-execute-time, instead of %timeit !!!
• jupyterlab-spreadsheet
• Matplotlib widget. // %matplotlib widget/inline/notebook
• jupyterlab-system-monitor // to not use TOP for the CPU & RAM
• jupyterlab-kite
• Variable Inspector
• jupyterlab-plotly // offline only since v4. online version is “chart-studio”.

OBS: dated code below, no need to install nodejs anymore !!!
Please use UI to install or update the code below.

pip show jupyterlab
apt install nodejs npm

jupyter labextension install @jupyterlab/toc --no-build
jupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build
jupyter labextension install @lckr/jupyterlab_variableinspector --no-build
pip install jupyterlab_autoversion
jupyter labextension install jupyterlab_autoversion
jupyter serverextension enable --py jupyterlab_autoversion
jupyter lab build # will take some time
jupyter labextension list # to check

HUB #

JupyterLab on JupyterHub

OBS : CONDA #

Conda is a cross-language package management system, its binary packages are in a different path, not the system’s python / R. It installs frequently used packages by default.
See more “Conda vs Anaconda vs Miniconda” & “Conda vs pip” here: continuum.io

FAQ #

Problem: version conflict Installation Succeeded ... Locking Failed! ... pkg_resources.VersionConflict: (importlib-metadata ... when install notebook.
(wrong) Solution: pip install -U pipenv && pipenv install -d --skip-lock [ref]
(correct) Solution: check and use the correct py version.

Problem: no module bz2.
Solution: find the python specific verion file such as _bz2.cpython-36m-x86_64-linux-gnu.so and copy to the virtual env folder .local\share\virtualenvs\<the_virtual_env_name>\lib\python3.6\site-packages\
.

Problem: no module bz2.
Solution: find the python specific verion file such as _bz2.cpython-36m-x86_64-linux-gnu.so and copy to the virtual env folder .local\share\virtualenvs\<the_virtual_env_name>\lib\python3.6\site-packages\