Notebook

CLI¶

In [ ]:

import os, pathlib, tempfile, shutil, atexit, hashlib, pprint
from IPython.display import *
from IPython import get_ipython # needed for `jupyter_execute` because magics?
RTD = os.environ.get("READTHEDOCS")

The jupyter lite (or jupyter-lite) CLI provides tools for lifecycle of combining...

the core JupyterLite static assets
extra application features like Lab Extensions and settings
kernel-specific resources like Python wheels
user-authored content like Notebooks

... into a ready-to-deploy (and optionally reproducible) Jupyter sites which require an HTTP server, but no application server.

Installation¶

In [ ]:

!pip install jupyterlite
!jupyter lite --version

Addon Dependencies¶

Some extra features of different addons have additional dependencies.

pip install jupyterlite[contents] # jupyter_server for contents API indexing
pip install jupyterlite[serve]    # tornado for better local previewing with `serve`
pip install jupyterlite[piplite]  # pkginfo for reading wheel metadata for the pyolite kernel
pip install jupyterlite[mathjax]  # reuse the MathJax assets from jupyter-server-mathjax
pip install jupyterlite[check]    # validate more data with jsonschema
pip install jupyterlite[lab]      # a known-compatible jupyterlab (entails `contents`, `serve`, `check`)

...or, for everything:

pip install jupyterlite[all]      # all of the above!

In [ ]:

!pip install jupyterlite[all]

Quick Start¶

Once installed, one can get to a locally-hosted, empty JupyterLite site with:

jupyter lite serve

This will:

init the baseline assets
build all extra assets
- if a files/ folder is found, those will be copied into _output/files
  - ... and indexed if jupyter_server is installed
serve the site locally and print out a URL

The Lite Dir¶

When you run jupyter lite commands, it assumes your current working directory is the partial contents of a JupyterLite site. You can override this with --lite-dir. By default, the built site will be created in _output, but can be overridden with --output-dir.

In [ ]:

if "TMP_DIR" not in globals():
    TMP_DIR = pathlib.Path(tempfile.mkdtemp(prefix="_my_lite_dir_"))
    def clean():
        shutil.rmtree(TMP_DIR)
    atexit.register(clean)
os.chdir(TMP_DIR)
print(pathlib.Path.cwd())

Well-known Files¶

Some files in your --lite-dir that have special meaning:

look in path...	... for file named	... and if found
`.` `./lab` `./retro`	`jupyter-lite.{json,ipynb}`	merge with static in `_output/{path}/jupyter-lite.{json,ipynb}`
`.` `./lab` `./retro`	`overrides.json`	merge with static in `_output/{*}/jupyter-lite.json`
`./files/`	`*`	copy verbatim to `_output/files/*` and index in `/api/contents`
`./pypi/`	`*.whl`	copy verbatim to `_output/pypi/*` and index
`./static/pyodide`	`pyodide.js`, `.data`, `.js`	copy verbatim to `_output/pyodide/*`

See more about merging contents.

Usage¶

Common Parameters¶

parameter	description	default	environment variable
`--lite-dir`	configuration and content for the site	current working directory	`JUPYTERLITE_DIR`
`--output-dir`	where the hostable site will be created	`_output`	`JUPYTERLITE_OUTPUT_DIR`
`--cache-dir`	a cache directory for downloads	`<lite_dir>/.cache`	`JUPYTERLITE_CACHE_DIR`
`--disable-addons`	disable a specific addon by entrypoint name
`--app-archive`	an alternate site to base off of	bundled
`--contents`	directory to copy to `_output/files/` and available as Contents	`./files`
`--ignore-contents`	patterns that should never be included in `/files/` (even if found in `lite-dir`)	various
`--output-archive`	the path to the archive	`<directory>-jupyterlite.tgz`	`JUPYTERLAB_OUTPUT_ARCHIVE`
`--port`	port on `127.0.0.1` to serve the test server	`8000`	`JUPYTERLITE_PORT`
`--base-url`	the URL prefix to include before the site	`/`	`JUPYTERLITE_BASE_URL`
`--source-date-epoch`	optionally enable additional reproducible build measures (best-effort!)		`SOURCE_DATE_EPOCH`
`--federated-extensions`	paths to folders, `pip`/`conda` packages with extensions see note
`--ignore-sys-prefix`	don't copy any contents, such as install labextensions, from `sys.prefix`	`False`
`--settings-overrides`	additional settings overrides to enable
`--piplite-wheels`	additional wheels to make available
`--pyodide`	the path or URL of a pyodide distribution (or .bz2 archive)		`JUPYTERLITE_PYODIDE_URL`
`--mathjax-dir`	the path to a MathJax 2 distribution

All parameters may be configured via a jupyter_lite_config.json in the directory where jupyter lite is launched, or given via --config.

{hint}
For an advanced example, see the [configuration](https://github.com/jupyterlite/jupyterlite/tree/main/examples) used for this documentation.

Help¶

The CLI provides its own documentation, under --help (or -h).

In [ ]:

!jupyter lite --help

Status¶

Always safe to run, this command provides an overview of what JupyterLite has been doing.

In [ ]:

!jupyter lite status

List¶

Always safe to run, this command provides an overview of what JupyterLite might do.

TODO: improve on default output

In [ ]:

!jupyter lite list

Init¶

Copy all the static data to the --output-dir.

In [ ]:

!jupyter lite init

Build¶

Copy all the user-authored content to the --output-dir, and applies appropriate changes to e.g. generated Contents API responses.

Special well-known files will be merged appropriately, but generally, files that exist in the user directory will overwrite any existing content.

In [ ]:

!jupyter lite build

Serve¶

Serve the --output-dir on http://127.0.0.1:{--port=8000}{--base-url=/}.

{warning}
This is _not_ a production server. Please consider _any_ of the [deployment](./deploying.md) options
before trying to make this something it isn't.

In [ ]:

!jupyter lite serve --help

Check¶

Use all available mechanisms to verify that the build folder conforms to schema, etc.

In [ ]:

!jupyter lite check

Archive¶

Turn the output directory into a .tgz file. This is usually easier to move around than (sometimes) hundreds of files, and can be used as the baseline for future sites.

This command is relatively expensive, and is skipped for documentation purposes

In [ ]:

!jupyter lite archive --help
if not RTD:
    !jupyter lite archive

But let's talk about a more reproducible asset.

In [ ]:

# we clean out the TMP_DIR for the reproducibility examples
shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)

Reproducible Archives¶

🛠️ This feature is a work-in-progress, and should not be relied upon by any production workflows Just Yet.

If --source-date-epoch is given, a number of measures will be taken to try to ensure that the output of jupyter lite archive, an npm-compatible tgz package, always returns a bit-for-bit reproducible build.

The most obvious change is that the modified time of each file "clamped" to that time. Some other changes:

file ownership will be reset
predictable sorting will be used
additional checks will be applied

{note}
This is a shortcut for setting the environment variable `SOURCE_DATE_EPOCH`:

| platform         | command                                               |
|------------------|-------------------------------------------------------|
| Linux<br/>MacOS  | `export SOURCE_DATE_EPOCH=<a timestamp>`              |
| Windows          | `set SOURCE_DATE_EPOCH=<a timestamp>`                 |
| Python           | `os.environ.update(SOURCE_DATE_EPOCH, <a timestamp>)` |

In [ ]:

if not "source_date_epoch" in globals():
    from datetime import datetime
    source_date_epoch = int(datetime.utcnow().timestamp())

print("SOURCE_DATE_EPOCH is", source_date_epoch)

In [ ]:

if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./a.tgz

If we clear out our _output...

In [ ]:

if not RTD:
    shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)
    pprint.pprint([*TMP_DIR.rglob("*")])

...and rebuild, we should always get the same file.

In [ ]:

if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./b.tgz

In [ ]:

if not RTD:
    a, b = [
        hashlib.sha256((TMP_DIR / f"{x}.tgz").read_bytes()).hexdigest() 
        for x in "ab"
    ]
    print("We built app archives with the SHA256SUMS of:\n", a, "\n", b)
    try:
        assert a == b, "We did not reproducibly build today.\n- {}\n- {}\n\n".format(a, b)
    except AssertionError as err:
        if shutil.which("diffoscope"):
            print("We did NOT reproducibly build today, checking in with `diffoscope`...")
            !diffoscope a.tgz b.tgz
        print("...but at least we tried REALLY hard!\n")

Miscellaneous¶

Merging Contents¶

In addition to the default files/ path, provide additional files to show in the File Manager or Tree when JupyterLite launches with --contents, which can be provided multiple times.

{note}
- If `--contents` is provided, the default value of `--contents files` will be ignored
- If given multiple times, and child folder/file names would collide, the **_last_ path wins**

{hint}
For complex file layouts, a `jupyter_lite_config.json` is **highly recommended**.

Contents Example¶

Given a directory like:

my-lite-dir/
  files/
    README.md
    a.md
  more-files/
    README.md
    b.md

Running jupyter lite build without any arguments will yield:

my-lite-dir/
  _output/
    README.md
    b.md

Whereas jupyter lite build --contents more-files --contents files will yield:

my-lite-dir/
  _output/
    files/
      README.md # this will be from my-lite-dir/files
      a.md
      b.md

And jupyter lite build --contents files --contents more-files will yield:

my-lite-dir/
  _output/
    files/
      README.md # this will be from my-lite-dir/more-files
      a.md
      b.md

conda packages¶

While --federated-extensions support the .tar.bz2 created by most conda packages, there are some issues:

anaconda.org uses non-standard HTTP headers to S3 buckets to provide packages
the conda-forge channel provides all of its builds as GitHub releases, and can be predictably transformed, e.g.

https://anaconda.org/conda-forge/jupyterlab_widgets/1.0.0/download/noarch/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2
                                                                   |      |
                                                                   |      +---------------------------------------------+
                                                                   v      v                                             v
         https://github.com/conda-forge/releases/releases/download/noarch/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2/jupyterlab_widgets-1.0.0-pyhd8ed1ab_1.tar.bz2

pyolite wheels¶

Prebuilt Lab Extension developers who wish to include wheels for use with pyolite may use this command to pre-index wheels they wish to distribute. This can be useful if a package needs upstream patches, etc.

In [ ]:

!jupyter lite pip index --help

pyodide¶

Pyodide is a WebAssembly distribution of CPython and powers the pyolite kernel. At nearly 200mb, compressed, a full pyodide distribution includes both the Python interpreter and key anchor packages from the scientific computing stack. Parts of it are fetched as needed by the user's browser, by default, from the official pyodide CDN.

The --pyodide CLI option (or alternately pyodide_url config option or JUPYTERLITE_PYODIDE_URL environment variable) allows for fetching either a compressed pyodide .bz2, or folder containing such an archive's contents. This might be from the official pyodide release page, a nightly or PR asset, or otherwise customized build. Once fetched, it is copied to the output folder, and configured.

$\LaTeX$¶

If jupyter-server-mathjax is installed, the MathJax 2 assets it provides will be copied and configured during the build. This behavior can be:

disabled with --ignore-sys-prefix or --disable-addons mathjax, defaulting to fetching the assets from CDN.
overridden with --mathjax-dir <path>, which must be a full folder of a MathJax 2 distribution