#!/usr/bin/env python
# coding: utf-8

# # Deep Learning Tutorial with Keras and Tensorflow

# <div>
#     <img style="text-align: left" src="imgs/keras-tensorflow-logo.jpg" width="40%" />
# <div>

# ## Get the Materials

# <img src="imgs/github.jpg" />
# ```shell
# 
# git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git
# ```

# ---

# # Outline at a glance
# 
# - **Part I**: **Introduction**
# 
#     - Intro to Artificial Neural Networks
#         - Perceptron and MLP    
#         - naive pure-Python implementation
#         - fast forward, sgd, backprop
#         
#     - Introduction to Deep Learning Frameworks
#         - Intro to Theano
#         - Intro to Tensorflow
#         - Intro to Keras
#             - Overview and main features
#             - Overview of the `core` layers
#             - Multi-Layer Perceptron and Fully Connected
#                 - Examples with `keras.models.Sequential` and `Dense`
#             - Keras Backend
#     
# - **Part II**: **Supervised Learning **
#     
#     - Fully Connected Networks and Embeddings
#         - Intro to MNIST Dataset
#         - Hidden Leayer Representation and Embeddings
#         
#     - Convolutional Neural Networks
#         - meaning of convolutional filters
#             - examples from ImageNet    
#         - Visualising ConvNets 
# 
#         - Advanced CNN
#             - Dropout
#             - MaxPooling
#             - Batch Normalisation
# 
#         - HandsOn: MNIST Dataset
#             - FC and MNIST
#             - CNN and MNIST
#         
#         - Deep Convolutiona Neural Networks with Keras (ref: `keras.applications`)
#             - VGG16
#             - VGG19
#             - ResNet50
#     - Transfer Learning and FineTuning
#     - Hyperparameters Optimisation 
#         
# - **Part III**: **Unsupervised Learning**
# 
#     - AutoEncoders and Embeddings
# 	- AutoEncoders and MNIST
#     	- word2vec and doc2vec (gensim) with `keras.datasets`
#         - word2vec and CNN
#     
# - **Part IV**: **Recurrent Neural Networks**
#     - Recurrent Neural Network in Keras 
#         -  `SimpleRNN`, `LSTM`, `GRU`
#     - LSTM for Sentence Generation
# 		
# - **PartV**: **Additional Materials**:  
#    - Custom Layers in Keras 
#    - Multi modal Network Topologies with Keras

# ---

# # Requirements

# This tutorial requires the following packages:
# 
# - Python version 3.5
#     - Python 3.4 should be fine as well
#     - likely Python 2.7 would be also fine, but *who knows*? :P
#     
# - `numpy` version 1.10 or later: http://www.numpy.org/
# - `scipy` version 0.16 or later: http://www.scipy.org/
# - `matplotlib` version 1.4 or later: http://matplotlib.org/
# - `pandas` version 0.16 or later: http://pandas.pydata.org
# - `scikit-learn` version 0.15 or later: http://scikit-learn.org
# - `keras` version 2.0 or later: http://keras.io
# - `tensorflow` version 1.0 or later: https://www.tensorflow.org
# - `ipython`/`jupyter` version 4.0 or later, with notebook support
# 
# (Optional but recommended):
# 
# - `pyyaml`
# - `hdf5` and `h5py` (required if you use model saving/loading functions in keras)
# - **NVIDIA cuDNN** if you have NVIDIA GPUs on your machines.
#     [https://developer.nvidia.com/rdp/cudnn-download]()
# 
# The easiest way to get (most) these is to use an all-in-one installer such as [Anaconda](http://www.continuum.io/downloads) from Continuum. These are available for multiple architectures.

# ---

# ### Python Version

# I'm currently running this tutorial with **Python 3** on **Anaconda**

# In[1]:


get_ipython().system('python --version')


# ### Configure Keras with tensorflow
# 
# 1) Create the `keras.json` (if it does not exist):
# 
# ```shell
# touch $HOME/.keras/keras.json
# ```
# 
# 2) Copy the following content into the file:
# 
# ```
# {
#     "epsilon": 1e-07,
#     "backend": "tensorflow",
#     "floatx": "float32",
#     "image_data_format": "channels_last"
# }
# ```

# In[2]:


get_ipython().system('cat ~/.keras/keras.json')


# ---

# # Test if everything is up&running

# ## 1. Check import

# In[3]:


import numpy as np
import scipy as sp
import pandas as pd
import matplotlib.pyplot as plt
import sklearn


# In[4]:


import keras


# ## 2. Check installeded Versions

# In[5]:


import numpy
print('numpy:', numpy.__version__)

import scipy
print('scipy:', scipy.__version__)

import matplotlib
print('matplotlib:', matplotlib.__version__)

import IPython
print('iPython:', IPython.__version__)

import sklearn
print('scikit-learn:', sklearn.__version__)


# In[6]:


import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)


# <br>
# <h1 style="text-align: center;">If everything worked till down here, you're ready to start!</h1>

# ---
#