#|hide
#default_exp cli
from nbdev.showdoc import show_doc
Console commands added by the nbdev library
#|export
from nbdev.imports import *
from nbdev.export import *
from nbdev.sync import *
from nbdev.merge import *
from nbdev.export2html import *
from nbdev.clean import *
from nbdev.test import *
from fastcore.script import *
from ghapi.all import GhApi
from urllib.error import HTTPError
nbdev
comes with the following commands. To use any of them, you must be in one of the subfolders of your project: they will search for the settings.ini
recursively in the parent directory but need to access it to be able to work. Their names all begin with nbdev so you can easily get a list with tab completion.
nbdev_build_docs
builds the documentation from the notebooksnbdev_build_lib
builds the library from the notebooksnbdev_bump_version
increments version in settings.py
by onenbdev_clean_nbs
removes all superfluous metadata form the notebooks, to avoid merge conflictsnbdev_detach
exports cell attachments to dest
and updates referencesnbdev_diff_nbs
gives you the diff between the notebooks and the exported librarynbdev_fix_merge
will fix merge conflicts in a notebook filenbdev_install_git_hooks
installs the git hooks that use the last two command automatically on each commit/mergenbdev_nb2md
converts a notebook to a markdown filenbdev_new
creates a new nbdev projectnbdev_read_nbs
reads all notebooks to make sure none are brokennbdev_test_nbs
runs tests in notebooksnbdev_trust_nbs
trusts all notebooks (so that the HTML content is shown)nbdev_update_lib
propagates any change in the library back to the notebooksshow_doc(nbdev_build_lib)
nbdev_build_lib
[source]
nbdev_build_lib
(fname
:"A notebook name or glob to convert"=None
,bare
:"Omit nbdev annotation comments (may break some functionality)."=False
)
Export notebooks matching fname
to python modules
By default (fname
left to None
), the whole library is built from the notebooks in the lib_folder
set in your settings.ini
.
show_doc(nbdev_update_lib)
nbdev_update_lib
[source]
nbdev_update_lib
(fname
:"A python filename or glob to convert"=None
,silent
:"Don't print results"=False
)
Propagates any change in the modules matching fname
to the notebooks that created them
By default (fname
left to None
), the whole library is treated. Note that this tool is only designed for small changes such as typo or small bug fixes. You can't add new cells in notebook from the library.
show_doc(nbdev_diff_nbs)
nbdev_diff_nbs
[source]
nbdev_diff_nbs
()
Prints the diff between an export of the library in notebooks and the actual modules
show_doc(nbdev_test_nbs)
nbdev_test_nbs
[source]
nbdev_test_nbs
(fname
:"A notebook name or glob to convert"=None
,flags
:"Space separated list of flags"=None
,n_workers
:"Number of workers to use"=None
,verbose
:"Print errors along the way"=True
,timing
:"Timing each notebook to see the ones are slow"=False
,pause
:"Pause time (in secs) between notebooks to avoid race conditions"=0.5
)
Test in parallel the notebooks matching fname
, passing along flags
By default (fname
left to None
), the whole library is tested from the notebooks in the lib_folder
set in your settings.ini
.
show_doc(nbdev_build_docs)
nbdev_build_docs
[source]
nbdev_build_docs
(fname
:"A notebook name or glob to convert"=None
,force_all
:"Rebuild even notebooks that haven't changed"=False
,mk_readme
:"Also convert the index notebook to README"=True
,n_workers
:"Number of workers to use"=None
,pause
:"Pause time (in secs) between notebooks to avoid race conditions"=0.5
)
Build the documentation by converting notebooks matching fname
to html
By default (fname
left to None
), the whole documentation is build from the notebooks in the lib_folder
set in your settings.ini
, only converting the ones that have been modified since the their corresponding html was last touched unless you pass force_all=True
. The index is also converted to make the README file, unless you pass along mk_readme=False
.
show_doc(nbdev_nb2md)
nbdev_nb2md
[source]
nbdev_nb2md
(fname
:"A notebook file name to convert",dest
:"The destination folder"='.'
,img_path
:"Folder to export images to"=''
,jekyll
:"To use jekyll metadata for your markdown file or not"=False
)
Convert the notebook in fname
to a markdown file
show_doc(nbdev_detach)
nbdev_detach
[source]
nbdev_detach
(path_nb
:"Path to notebook",dest
:"Destination folder"=''
,use_img
:"Convert markdown images to img tags"=False
,replace
:"Write replacement notebook back topath_bn
"=True
)
Export cell attachments to dest
and update references
show_doc(nbdev_read_nbs)
nbdev_read_nbs
[source]
nbdev_read_nbs
(fname
:"A notebook name or glob to convert"=None
)
Check all notebooks matching fname
can be opened
By default (fname
left to None
), the all the notebooks in lib_folder
are checked.
show_doc(nbdev_trust_nbs)
nbdev_trust_nbs
[source]
nbdev_trust_nbs
(fname
:"A notebook name or glob to convert"=None
,force_all
:"Trust even notebooks that haven't changed"=False
)
Trust notebooks matching fname
By default (fname
left to None
), the all the notebooks in lib_folder
are trusted. To speed things up, only the ones touched since the last time this command was run are trusted unless you pass along force_all=True
.
show_doc(nbdev_fix_merge)
nbdev_fix_merge
[source]
nbdev_fix_merge
(fname
:"A notebook filename to fix",fast
:"Fast fix: automatically fix the merge conflicts in outputs or metadata"=True
,trust_us
:"Use local outputs/metadata when fast merging"=True
)
Fix merge conflicts in notebook fname
When you have merge conflicts after a git pull
, the notebook file will be broken and won't open in jupyter notebook anymore. This command fixes this by changing the notebook to a proper json file again and add markdown cells to signal the conflict, you just have to open that notebook again and look for >>>>>>>
to see those conflicts and manually fix them. The old broken file is copied with a .ipynb.bak
extension, so is still accessible in case the merge wasn't successful.
Moreover, if fast=True
, conflicts in outputs and metadata will automatically be fixed by using the local version if trust_us=True
, the remote one if trust_us=False
. With this option, it's very likely you won't have anything to do, unless there is a real conflict.
#|export
def bump_version(version, part=2):
version = version.split('.')
version[part] = str(int(version[part]) + 1)
for i in range(part+1, 3): version[i] = '0'
return '.'.join(version)
test_eq(bump_version('0.1.1' ), '0.1.2')
test_eq(bump_version('0.1.1', 1), '0.2.0')
#|export
@call_parse
def nbdev_bump_version(
part:int=2 # Part of version to bump
):
"Increment version in `settings.py` by one"
cfg = get_config()
print(f'Old version: {cfg.version}')
cfg.d['version'] = bump_version(get_config().version, part)
cfg.save()
update_version()
print(f'New version: {cfg.version}')
#|export
@call_parse
def nbdev_install_git_hooks():
"Install git hooks to clean/trust notebooks automatically"
try: path = get_config().config_file.parent
except: path = Path.cwd()
hook_path = path/'.git'/'hooks'
fn = hook_path/'post-merge'
hook_path.mkdir(parents=True, exist_ok=True)
#Trust notebooks after merge
fn.write_text("#!/bin/bash\necho 'Trusting notebooks'\nnbdev_trust_nbs")
os.chmod(fn, os.stat(fn).st_mode | stat.S_IEXEC)
#Clean notebooks on commit/diff
(path/'.gitconfig').write_text("""# Generated by nbdev_install_git_hooks
#
# If you need to disable this instrumentation do:
# git config --local --unset include.path
#
# To restore the filter
# git config --local include.path .gitconfig
#
# If you see notebooks not stripped, checked the filters are applied in .gitattributes
#
[filter "clean-nbs"]
clean = nbdev_clean_nbs --read_input_stream True
smudge = cat
required = true
[diff "ipynb"]
textconv = nbdev_clean_nbs --disp True --fname
""")
cmd = "git config --local include.path ../.gitconfig"
print(f"Executing: {cmd}")
run(cmd)
print("Success: hooks are installed and repo's .gitconfig is now trusted")
try: nb_path = get_config().path("nbs_path")
except: nb_path = Path.cwd()
(nb_path/'.gitattributes').write_text("**/*.ipynb filter=clean-nbs\n**/*.ipynb diff=ipynb\n")
This command installs git hooks to make sure notebooks are cleaned before you commit them to GitHub and automatically trusted at each merge. To be more specific, this creates:
nbdev_trust_nbs
.gitconfig
file that uses nbev_clean_nbs
has a filter/diff on all notebook files inside nbs_folder
and a .gitattributes
file generated in this folder (copy this file in other folders where you might have notebooks you want cleaned as well)#|export
_template_git_repo = "https://github.com/fastai/nbdev_template.git"
#|export
import tarfile
#|export
def extract_tgz(url, dest='.'):
with urlopen(url) as u: tarfile.open(mode='r:gz', fileobj=u).extractall(dest)
#|export
def _get_branch(owner, repo, default='main'):
api = GhApi(owner=owner, repo=repo, token=os.getenv('GITHUB_TOKEN'))
try: return api.repos.get().default_branch
except HTTPError:
msg= [f"Could not access repo: {owner}/{repo} to find your default branch - `{default} assumed.\n",
"Edit `settings.ini` if this is incorrect.\n"
"In the future, you can allow nbdev to see private repos by setting the environment variable GITHUB_TOKEN as described here: https://nbdev.fast.ai/cli.html#Using-nbdev_new-with-private-repos \n", ]
print(''.join(msg))
return default
#|hide
# should see warning message.
assert _get_branch('fastai', 'definitely-does-not-exist') == 'main';
Could not access repo: fastai/definitely-does-not-exist to find your default branch - `main assumed. Edit `docs/_config.yml` if this is incorrect. In the future, you can allow nbdev to see private repos by setting the environment variable GITHUB_TOKEN as described here: https://nbdev.fast.ai/cli.html#Using-nbdev_new-with-private-repos
#|export
@call_parse
def nbdev_new():
"Create a new nbdev project from the current git repo"
url = run('git config --get remote.origin.url')
if not url: raise Exception('This does not appear to be a cloned git directory with a remote')
author = run('git config --get user.name').strip()
email = run('git config --get user.email').strip()
if not (author and email): raise Exception('User name and email not configured in git')
# download and untar template, and optionally notebooks
tgnm = urljson('https://api.github.com/repos/fastai/nbdev_template/releases/latest')['tag_name']
FILES_URL = f"https://github.com/fastai/nbdev_template/archive/{tgnm}.tar.gz"
extract_tgz(FILES_URL)
path = Path()
nbexists = True if first(path.glob('*.ipynb')) else False
for o in (path/f'nbdev_template-{tgnm}').ls():
if o.name == '00_core.ipynb':
if not nbexists: shutil.move(str(o), './')
elif not Path(f'./{o.name}').exists(): shutil.move(str(o), './')
shutil.rmtree(f'nbdev_template-{tgnm}')
# auto-config settings.ini from git
settings_path = Path('settings.ini')
settings = settings_path.read_text()
owner,repo = repo_details(url)
branch = _get_branch(owner, repo)
settings = settings.format(lib_name=repo, user=owner, author=author, author_email=email, branch=branch)
settings_path.write_text(settings)
nbdev_install_git_hooks()
nbdev_new
is a command line tool that creates a new nbdev project from the current directory, which must be a cloned git repo.
After you run nbdev_new
, please check the contents of settings.ini
look good, and then run nbdev_build_lib
.
nbdev_new
with private repos¶nbdev_new
attempts to find your repo on GitHub to determine your default branch. If your repo is private, you will need to set the environment variable GITHUB_TOKEN
with a personal access token prior to running nbdev_new
, or nbdev_new
will use a default value instead.
You will receive a warning message if nbdev is not able to find your repo on GitHub that will tell you to replace the default branch name in settings.ini
(which is the root of your repo).
#|hide
from nbdev.export import notebook2script
notebook2script()
Converted 00_export.ipynb. Converted 01_sync.ipynb. Converted 02_showdoc.ipynb. Converted 03_export2html.ipynb. Converted 04_test.ipynb. Converted 05_merge.ipynb. Converted 06_cli.ipynb. Converted 07_clean.ipynb. Converted 99_search.ipynb. Converted example.ipynb. Converted index.ipynb. Converted tutorial.ipynb.