Overtraining and Regularization:¶
Open the Tensorflow Playground
(https://playground.tensorflow.org) and select on the left the checkerboard
pattern as the data basis (see Exercise 3.3). As input features, select the two
independent variables $x_1$ and $x_2$ and set the noise to $50\%$.
- Choose a deep (many layers) and wide (many nodes) network and train it for more than 1000 epochs. Comment on your observations.
- Apply L2 regularization to reduce overfitting. Try low and high regularization rates. What do you observe?
- Compare the effects of L1 and L2 regularization.