Caching variables in a file

The ipycache package defines a %%cache cell magic allowing you to cache results of long-lasting computations in a pickle file.

import numpy as np

%load_ext ipycache

def long_function(a, b):
    import time
    time.sleep(1)
    return a + b

The following cell will only be executed if the pickle file does not exist (also, there is a --force option to always execute the cell and overwrite the results in the pickle file).

%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)

Saved variables a, b, c to file 'myvars.pkl'.

print(a, b, c)

(3, array([-0.95123539, -0.17912806, -0.50877324]), array([ 2.04876461,  2.82087194,  2.49122676]))

del a, b, c

Now, we execute the same cell again (in practice, the cell only appears once in the notebook, so this simulates multiple executions of the same cell).

%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)

Skipped the cell's code and loaded variables a, b, c from file 'myvars.pkl'.

The cell has been skipped, but the variables have been recovered from the pickle file.

print(a, b, c)

(3, array([-0.95123539, -0.17912806, -0.50877324]), array([ 2.04876461,  2.82087194,  2.49122676]))

Now, let's delete the file.

import os
os.remove('myvars.pkl')

Let's execute the cell once more.

%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)

Saved variables a, b, c to file 'myvars.pkl'.

print(a, b, c)

(3, array([ 0.19778047,  1.81953213, -0.1259566 ]), array([ 3.19778047,  4.81953213,  2.8740434 ]))

os.remove('myvars.pkl')