#!/usr/bin/env python
# coding: utf-8

# # Exercise 5.1
# ## Overtraining and Regularization:
# Open the Tensorflow Playground
# (https://playground.tensorflow.org) and select on the left the checkerboard
# pattern as the data basis (see [Exercise 3.3](Exercise_3_3.ipynb)). As input features, select the two
# independent variables $x_1$ and $x_2$ and set the noise to $50\%$.
# 
# 
# [![Checkerboard](./images/checkerboard_regularization.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=50&networkShape=6,6,6,6,6&seed=0.82577&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=true)
# 
# 
# ## Tasks
# 1. Choose a deep (many layers) and wide (many nodes) network and train it for more than 1000 epochs. Comment on your observations.
# 2. Apply L2 regularization to reduce overfitting. Try low and high regularization rates. What do you observe?
# 3. Compare the effects of L1 and L2 regularization.