!pip install papermill
import platform
print(f"Python version: {platform.python_version()}")
assert platform.python_version_tuple() >= ("3", "6")
import os
import papermill as pm
from IPython.display import YouTubeVideo
Python version: 3.7.5
import this
The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
a = 3
b = 2
# Non-pythonic
tmp = a
a = b
b = tmp
# Pythonic
a, b = b, a
my_list = ["a", "b", "c"]
def do_something(item):
# print(item)
pass
# Non-pythonic
i = 0
while i < len(my_list):
do_something(my_list[i])
i += 1
# Still non-pythonic
for i in range(len(my_list)):
do_something(my_list[i])
# Pythonic
for item in my_list:
do_something(item)
my_list = ["a", "b", "c"]
# Non-pythonic
for i in range(len(my_list)):
print(i, "->", my_list[i])
# Pythonic
for i, item in enumerate(my_list):
print(i, "->", item)
0 -> a 1 -> b 2 -> c 0 -> a 1 -> b 2 -> c
fruits = ["apples", "oranges", "bananas", "grapes"]
fruit = "cherries"
# Non-pythonic
found = False
size = len(fruits)
for i in range(0, size):
if fruits[i] == fruit:
found = True
# Pythonic
found = fruit in fruits
This feature is called list comprehension.
numbers = [1, 2, 3, 4, 5, 6]
# Non-pythonic
doubles = []
for i in range(len(numbers)):
if numbers[i] % 2 == 0:
doubles.append(numbers[i] * 2)
else:
doubles.append(numbers[i])
# Pythonic
doubles = [x * 2 if x % 2 == 0 else x for x in numbers]
Focusing on style and PEP8-compliance might make you miss more fundamental code imperfections.
YouTubeVideo("wf-BqAjZb8M")
def complex(real=0.0, imag=0.0):
"""Form a complex number.
Keyword arguments:
real -- the real part (default 0.0)
imag -- the imaginary part (default 0.0)
"""
if imag == 0.0 and real == 0.0:
return complex_zero
def greeting(name: str) -> str:
return "Hello " + name
# greeting('Alice') # OK
# greeting(3) # mypy error: incompatible type "int"; expected "str"
def inc(x):
return x + 1
def test_answer():
assert inc(3) == 5 # AssertionError: assert 4 == 5
X.Y.Z
.A requirements.txt
file is the most basic way of declaring dependencies in Python.
certifi>=2020.11.0
chardet==4.0.0
click>=6.5.0, <7.1
download==0.3.5
Flask>=1.1.0
The pip package installer can read this file and act accordingly, downloading dependencies from PyPI.
pip install -r requirements.txt
Installed as part of the Anaconda distribution, the conda package manager reads an environment.yml
file to install the dependencies associated to a specific virtual environment.
name: example-env
channels:
- conda-forge
- defaults
dependencies:
- python=3.7
- matplotlib
- numpy
Poetry is a recent packaging and dependency management tool for Python. It downloads packages from PyPI by default.
# Create a new poetry-compliant project
poetry new <project name>
# Initialize an already existing project for Poetry
poetry init
# Install defined dependencies
poetry install
# Add a package to project dependencies and install it
poetry add <package name>
# Update dependencies to sync them with configuration file
poetry update
By default, Poetry creates a virtual environment for the configured project in a user-specific folder. A standard practice is to store it in the project's folder.
# Tell Poetry to store the environment in the local project folder
poetry config virtualenvs.in-project true
# Activate the environment
poetry shell
Poetry configuration file, soon-to-be standard for Python projects.
[tool.poetry]
name = "poetry example"
version = "0.1.0"
description = ""
[tool.poetry.dependencies]
python = ">=3.7.1,<3.10"
jupyter = "^1.0.0"
matplotlib = "^3.3.2"
sklearn = "^0.0"
pandas = "^1.1.3"
ipython = "^7.0.0"
[tool.poetry.dev-dependencies]
pytest = "^6.1.1"
Offers a way to precisely define dependency versions.
Requirement | Versions allowed |
---|---|
^1.2.3 | >=1.2.3 <2.0.0 |
^1.2 | >=1.2.0 <2.0.0 |
~1.2.3 | >=1.2.3 <1.3.0 |
~1.2 | >=1.2.0 <1.3.0 |
1.2.3 | 1.2.3 only |
poetry.lock
file that contains the exact versions of all installed packages.Monolithic notebooks can grow over time and become hard to understand and maintain.
Just like in a traditional software project, it is possible to split them into separate parts, thus following the separation of concerns design principle.
Code can be splitted into several sub-notebooks and/or external Python files. The latter facilitates unit testing and version control.
# Doesn't work on Google Colaboratory. Workaround here:
# https://colab.research.google.com/github/rjdoubleu/Colab-Papermill-Patch/blob/master/Colab-Papermill-Driver.ipynb
notebook_dir = "./papermill"
result = pm.execute_notebook(
os.path.join(notebook_dir, "simple_input.ipynb"),
os.path.join(notebook_dir, "simple_output.ipynb"),
parameters={"msg": "Hello"},
)
HBox(children=(IntProgress(value=0, description='Executing', max=4, style=ProgressStyle(description_width='iniā¦