# %load /Users/facai/Study/book_notes/preconfig.py
%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
sns.set(font='SimHei', font_scale=2.5)
plt.rcParams['axes.grid'] = False

def show_image(filename, figsize=None, res_dir=True):
    if figsize:
        plt.figure(figsize=figsize)

    if res_dir:
        filename = './res/{}'.format(filename)

    plt.imshow(plt.imread(filename))

11 Practical Methodology

practical design process:

1. Determine your goals.
   • which error metric to use?
     • coverage:
   • what level of performance you desire?
2. Establish a working end-to-end pipeline as soon as possible.
3. Instrument the system well to determine bottlenecks in performance.
4. Repeatedly make incremental changes.
   • Determinig whether to gather more data <= when train error is accpted, while test error not.
     • plot curves showing the relationship between training set size and generalization error. usually logarithmic scale.
   • Selecting Hyperparameters
     • manually: learning rate
     • automatic
       1. grid search: typically use logarithmic scale
       2. random search: more convenient to use, and converges much faster
   • Debugging strategies
     1. Visualize the model in action.
     2. Visualize the worst mistakes.
     3. Reasoning about software using train and test error.
     4. Fit a tiny dataset.
     5. Compare back-propagated derivatives to numerical derivatives.
     6. Monitor histograms of activations and gradient:
        • visualize statistics of neural network activations and gradients, collected over a large amount of training iterations (maybe one epoch).
        • compare the magnitude of parameter gradients to themselves.