In this section, we train a Convolutional Neural Network to classify images from the MNIST database.
MNIST is one of the most famous datasets in the field of machine learning.
from keras.datasets import mnist
# use Keras to import pre-shuffled MNIST database
(X_train, y_train), (X_test, y_test) = mnist.load_data()
print("The MNIST database has a training set of %d examples." % len(X_train))
print("The MNIST database has a test set of %d examples." % len(X_test))
Using TensorFlow backend.
The MNIST database has a training set of 60000 examples. The MNIST database has a test set of 10000 examples.
import matplotlib.pyplot as plt
%matplotlib inline
import matplotlib.cm as cm
import numpy as np
# plot first six training images
fig = plt.figure(figsize=(20,20))
for i in range(6):
ax = fig.add_subplot(1, 6, i+1, xticks=[], yticks=[])
ax.imshow(X_train[i], cmap='gray')
ax.set_title(str(y_train[i]))
def visualize_input(img, ax):
ax.imshow(img, cmap='gray')
width, height = img.shape
thresh = img.max()/2.5
for x in range(width):
for y in range(height):
ax.annotate(str(round(img[x][y],2)), xy=(y,x),
horizontalalignment='center',
verticalalignment='center',
color='white' if img[x][y]<thresh else 'black')
fig = plt.figure(figsize = (12,12))
ax = fig.add_subplot(111)
visualize_input(X_train[0], ax)
# normalize the data to accelerate learning
mean = np.mean(X_train)
std = np.std(X_train)
X_train = (X_train-mean)/(std+1e-7)
X_test = (X_test-mean)/(std+1e-7)
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
X_train shape: (60000, 28, 28) 60000 train samples 10000 test samples
from keras.utils import np_utils
num_classes = 10
# print first ten (integer-valued) training labels
print('Integer-valued labels:')
print(y_train[:10])
# one-hot encode the labels
# convert class vectors to binary class matrices
y_train = np_utils.to_categorical(y_train, num_classes)
y_test = np_utils.to_categorical(y_test, num_classes)
# print first ten (one-hot) training labels
print('One-hot labels:')
print(y_train[:10])
Integer-valued labels: [5 0 4 1 9 2 1 3 1 4] One-hot labels: [[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.] [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.] [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 0. 0. 0. 0. 0. 1.] [0. 0. 1. 0. 0. 0. 0. 0. 0. 0.] [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.] [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]]
# input image dimensions 28x28 pixel images.
img_rows, img_cols = 28, 28
X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
input_shape = (img_rows, img_cols, 1)
print('image input shape: ', input_shape)
print('x_train shape:', X_train.shape)
image input shape: (28, 28, 1) x_train shape: (60000, 28, 28, 1)
To implement LeNet-5 in Keras, read the original paper and extract the architecture information from pages 6, 7 and 8. Here are the main takeaways to build the LeNet-5 network:
Things to remember
from keras.models import Sequential
from keras.layers import Conv2D, AveragePooling2D, Flatten, Dense
#Instantiate an empty model
model = Sequential()
# C1 Convolutional Layer
model.add(Conv2D(6, kernel_size=(5, 5), strides=(1, 1), activation='tanh', input_shape=input_shape, padding='same'))
# S2 Pooling Layer
model.add(AveragePooling2D(pool_size=(2, 2), strides=2, padding='valid'))
# C3 Convolutional Layer
model.add(Conv2D(16, kernel_size=(5, 5), strides=(1, 1), activation='tanh', padding='valid'))
# S4 Pooling Layer
model.add(AveragePooling2D(pool_size=(2, 2), strides=2, padding='valid'))
# C5 Fully Connected Convolutional Layer
model.add(Conv2D(120, kernel_size=(5, 5), strides=(1, 1), activation='tanh', padding='valid'))
#Flatten the CNN output so that we can connect it with fully connected layers
model.add(Flatten())
# FC6 Fully Connected Layer
model.add(Dense(84, activation='tanh'))
# Output Layer with softmax activation
model.add(Dense(10, activation='softmax'))
# print the model summary
model.summary()
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_1 (Conv2D) (None, 28, 28, 6) 156 _________________________________________________________________ average_pooling2d_1 (Average (None, 14, 14, 6) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 10, 10, 16) 2416 _________________________________________________________________ average_pooling2d_2 (Average (None, 5, 5, 16) 0 _________________________________________________________________ conv2d_3 (Conv2D) (None, 1, 1, 120) 48120 _________________________________________________________________ flatten_1 (Flatten) (None, 120) 0 _________________________________________________________________ dense_1 (Dense) (None, 84) 10164 _________________________________________________________________ dense_2 (Dense) (None, 10) 850 ================================================================= Total params: 61,706 Trainable params: 61,706 Non-trainable params: 0 _________________________________________________________________
In this exercise, we will use Adam optimizer
# the loss function is categorical cross entropy since we have multiple classes (10)
# compile the model by defining the loss function, optimizer, and performance metric
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
LeCun and his team used scheduled decay learning where the value of the learning rate was decreased using the following schedule: 0.0005 for the first two epochs, 0.0002 for the next three epochs, 0.00005 for the next four, and then 0.00001 thereafter. In the paper, the authors trained their network for 20 epochs.
from keras.callbacks import ModelCheckpoint, LearningRateScheduler
# set the learning rate schedule as created in the original paper
def lr_schedule(epoch):
if epoch <= 2:
lr = 5e-4
elif epoch > 2 and epoch <= 5:
lr = 2e-4
elif epoch > 5 and epoch <= 9:
lr = 5e-5
else:
lr = 1e-5
return lr
lr_scheduler = LearningRateScheduler(lr_schedule)
# set the checkpointer
checkpointer = ModelCheckpoint(filepath='model.weights.best.hdf5', verbose=1,
save_best_only=True)
# train the model
hist = model.fit(X_train, y_train, batch_size=32, epochs=20,
validation_data=(X_test, y_test), callbacks=[checkpointer, lr_scheduler],
verbose=2, shuffle=True)
Train on 60000 samples, validate on 10000 samples Epoch 1/20 - 14s - loss: 0.2266 - accuracy: 0.9351 - val_loss: 0.0884 - val_accuracy: 0.9735 Epoch 00001: val_loss improved from inf to 0.08839, saving model to model.weights.best.hdf5 Epoch 2/20 - 12s - loss: 0.0786 - accuracy: 0.9762 - val_loss: 0.0617 - val_accuracy: 0.9798 Epoch 00002: val_loss improved from 0.08839 to 0.06172, saving model to model.weights.best.hdf5 Epoch 3/20 - 12s - loss: 0.0560 - accuracy: 0.9825 - val_loss: 0.0492 - val_accuracy: 0.9829 Epoch 00003: val_loss improved from 0.06172 to 0.04924, saving model to model.weights.best.hdf5 Epoch 4/20 - 12s - loss: 0.0347 - accuracy: 0.9900 - val_loss: 0.0371 - val_accuracy: 0.9878 Epoch 00004: val_loss improved from 0.04924 to 0.03714, saving model to model.weights.best.hdf5 Epoch 5/20 - 13s - loss: 0.0298 - accuracy: 0.9914 - val_loss: 0.0362 - val_accuracy: 0.9882 Epoch 00005: val_loss improved from 0.03714 to 0.03624, saving model to model.weights.best.hdf5 Epoch 6/20 - 13s - loss: 0.0262 - accuracy: 0.9923 - val_loss: 0.0335 - val_accuracy: 0.9883 Epoch 00006: val_loss improved from 0.03624 to 0.03353, saving model to model.weights.best.hdf5 Epoch 7/20 - 12s - loss: 0.0188 - accuracy: 0.9952 - val_loss: 0.0307 - val_accuracy: 0.9893 Epoch 00007: val_loss improved from 0.03353 to 0.03069, saving model to model.weights.best.hdf5 Epoch 8/20 - 12s - loss: 0.0176 - accuracy: 0.9955 - val_loss: 0.0308 - val_accuracy: 0.9890 Epoch 00008: val_loss did not improve from 0.03069 Epoch 9/20 - 13s - loss: 0.0167 - accuracy: 0.9957 - val_loss: 0.0295 - val_accuracy: 0.9894 Epoch 00009: val_loss improved from 0.03069 to 0.02953, saving model to model.weights.best.hdf5 Epoch 10/20 - 13s - loss: 0.0157 - accuracy: 0.9962 - val_loss: 0.0316 - val_accuracy: 0.9889 Epoch 00010: val_loss did not improve from 0.02953 Epoch 11/20 - 13s - loss: 0.0141 - accuracy: 0.9969 - val_loss: 0.0297 - val_accuracy: 0.9893 Epoch 00011: val_loss did not improve from 0.02953 Epoch 12/20 - 15s - loss: 0.0138 - accuracy: 0.9968 - val_loss: 0.0293 - val_accuracy: 0.9894 Epoch 00012: val_loss improved from 0.02953 to 0.02935, saving model to model.weights.best.hdf5 Epoch 13/20 - 13s - loss: 0.0136 - accuracy: 0.9969 - val_loss: 0.0294 - val_accuracy: 0.9894 Epoch 00013: val_loss did not improve from 0.02935 Epoch 14/20 - 12s - loss: 0.0135 - accuracy: 0.9970 - val_loss: 0.0293 - val_accuracy: 0.9893 Epoch 00014: val_loss improved from 0.02935 to 0.02929, saving model to model.weights.best.hdf5 Epoch 15/20 - 12s - loss: 0.0133 - accuracy: 0.9972 - val_loss: 0.0294 - val_accuracy: 0.9890 Epoch 00015: val_loss did not improve from 0.02929 Epoch 16/20 - 12s - loss: 0.0132 - accuracy: 0.9971 - val_loss: 0.0294 - val_accuracy: 0.9895 Epoch 00016: val_loss did not improve from 0.02929 Epoch 17/20 - 12s - loss: 0.0130 - accuracy: 0.9972 - val_loss: 0.0295 - val_accuracy: 0.9891 Epoch 00017: val_loss did not improve from 0.02929 Epoch 18/20 - 12s - loss: 0.0128 - accuracy: 0.9974 - val_loss: 0.0295 - val_accuracy: 0.9895 Epoch 00018: val_loss did not improve from 0.02929 Epoch 19/20 - 12s - loss: 0.0127 - accuracy: 0.9973 - val_loss: 0.0292 - val_accuracy: 0.9895 Epoch 00019: val_loss improved from 0.02929 to 0.02924, saving model to model.weights.best.hdf5 Epoch 20/20 - 13s - loss: 0.0125 - accuracy: 0.9974 - val_loss: 0.0293 - val_accuracy: 0.9894 Epoch 00020: val_loss did not improve from 0.02924
# load the weights that yielded the best validation accuracy
model.load_weights('model.weights.best.hdf5')
# evaluate test accuracy
score = model.evaluate(X_test, y_test, verbose=0)
accuracy = 100*score[1]
# print test accuracy
print('Test accuracy: %.4f%%' % accuracy)
Test accuracy: 98.9500%
import matplotlib.pyplot as plt
f, ax = plt.subplots()
ax.plot([None] + hist.history['accuracy'], 'o-')
ax.plot([None] + hist.history['val_accuracy'], 'x-')
# Plot legend and use the best location automatically: loc = 0.
ax.legend(['Train acc', 'Validation acc'], loc = 0)
ax.set_title('Training/Validation acc per Epoch')
ax.set_xlabel('Epoch')
ax.set_ylabel('acc')
plt.show()
import matplotlib.pyplot as plt
f, ax = plt.subplots()
ax.plot([None] + hist.history['loss'], 'o-')
ax.plot([None] + hist.history['val_loss'], 'x-')
# Plot legend and use the best location automatically: loc = 0.
ax.legend(['Train loss', "Val loss"], loc = 0)
ax.set_title('Training/Validation Loss per Epoch')
ax.set_xlabel('Epoch')
ax.set_ylabel('Loss')
plt.show()
from keras.models import Sequential
from keras.layers import Conv2D, AveragePooling2D, Flatten, Dense,Activation,MaxPool2D, BatchNormalization, Dropout
from keras.regularizers import l2
Using TensorFlow backend.
# Instantiate an empty sequential model
model = Sequential(name="Alexnet")
# 1st layer (conv + pool + batchnorm)
model.add(Conv2D(filters= 96, kernel_size= (11,11), strides=(4,4), padding='valid', kernel_regularizer=l2(0.0005),
input_shape = (227,227,3)))
model.add(Activation('relu')) #<---- activation function can be added on its own layer or within the Conv2D function
model.add(MaxPool2D(pool_size=(3,3), strides= (2,2), padding='valid'))
model.add(BatchNormalization())
# 2nd layer (conv + pool + batchnorm)
model.add(Conv2D(filters=256, kernel_size=(5,5), strides=(1,1), padding='same', kernel_regularizer=l2(0.0005)))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(3,3), strides=(2,2), padding='valid'))
model.add(BatchNormalization())
# layer 3 (conv + batchnorm) <--- note that the authors did not add a POOL layer here
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=l2(0.0005)))
model.add(Activation('relu'))
model.add(BatchNormalization())
# layer 4 (conv + batchnorm) <--- similar to layer 3
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=l2(0.0005)))
model.add(Activation('relu'))
model.add(BatchNormalization())
# layer 5 (conv + batchnorm)
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=l2(0.0005)))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(MaxPool2D(pool_size=(3,3), strides=(2,2), padding='valid'))
# Flatten the CNN output to feed it with fully connected layers
model.add(Flatten())
# layer 6 (Dense layer + dropout)
model.add(Dense(units = 4096, activation = 'relu'))
model.add(Dropout(0.5))
# layer 7 (Dense layers)
model.add(Dense(units = 4096, activation = 'relu'))
model.add(Dropout(0.5))
# layer 8 (softmax output layer)
model.add(Dense(units = 1000, activation = 'softmax'))
# print the model summary
model.summary()
Model: "Alexnet" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_1 (Conv2D) (None, 55, 55, 96) 34944 _________________________________________________________________ activation_1 (Activation) (None, 55, 55, 96) 0 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 27, 27, 96) 0 _________________________________________________________________ batch_normalization_1 (Batch (None, 27, 27, 96) 384 _________________________________________________________________ conv2d_2 (Conv2D) (None, 27, 27, 256) 614656 _________________________________________________________________ activation_2 (Activation) (None, 27, 27, 256) 0 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 13, 13, 256) 0 _________________________________________________________________ batch_normalization_2 (Batch (None, 13, 13, 256) 1024 _________________________________________________________________ conv2d_3 (Conv2D) (None, 13, 13, 384) 885120 _________________________________________________________________ activation_3 (Activation) (None, 13, 13, 384) 0 _________________________________________________________________ batch_normalization_3 (Batch (None, 13, 13, 384) 1536 _________________________________________________________________ conv2d_4 (Conv2D) (None, 13, 13, 384) 1327488 _________________________________________________________________ activation_4 (Activation) (None, 13, 13, 384) 0 _________________________________________________________________ batch_normalization_4 (Batch (None, 13, 13, 384) 1536 _________________________________________________________________ conv2d_5 (Conv2D) (None, 13, 13, 256) 884992 _________________________________________________________________ activation_5 (Activation) (None, 13, 13, 256) 0 _________________________________________________________________ batch_normalization_5 (Batch (None, 13, 13, 256) 1024 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 6, 6, 256) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 9216) 0 _________________________________________________________________ dense_1 (Dense) (None, 4096) 37752832 _________________________________________________________________ dropout_1 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_2 (Dense) (None, 4096) 16781312 _________________________________________________________________ dropout_2 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_3 (Dense) (None, 1000) 4097000 ================================================================= Total params: 62,383,848 Trainable params: 62,381,096 Non-trainable params: 2,752 _________________________________________________________________
from keras.models import Sequential
from keras.layers import Conv2D, AveragePooling2D, Flatten, Dense, Activation, MaxPool2D, BatchNormalization, Dropout, ZeroPadding2D
Using TensorFlow backend.
model = Sequential()
# first block
model.add(Conv2D(filters=64, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same',input_shape=(224,224, 3)))
model.add(Conv2D(filters=64, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# second block
model.add(Conv2D(filters=128, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=128, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# third block
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# forth block
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# fifth block
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# sixth block (classifier)
model.add(Flatten())
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1000, activation='softmax'))
model.summary()
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_1 (Conv2D) (None, 224, 224, 64) 1792 _________________________________________________________________ conv2d_2 (Conv2D) (None, 224, 224, 64) 36928 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 112, 112, 64) 0 _________________________________________________________________ conv2d_3 (Conv2D) (None, 112, 112, 128) 73856 _________________________________________________________________ conv2d_4 (Conv2D) (None, 112, 112, 128) 147584 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 56, 56, 128) 0 _________________________________________________________________ conv2d_5 (Conv2D) (None, 56, 56, 256) 295168 _________________________________________________________________ conv2d_6 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ conv2d_7 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 28, 28, 256) 0 _________________________________________________________________ conv2d_8 (Conv2D) (None, 28, 28, 512) 1180160 _________________________________________________________________ conv2d_9 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ conv2d_10 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ max_pooling2d_4 (MaxPooling2 (None, 14, 14, 512) 0 _________________________________________________________________ conv2d_11 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ conv2d_12 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ conv2d_13 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ max_pooling2d_5 (MaxPooling2 (None, 7, 7, 512) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 25088) 0 _________________________________________________________________ dense_1 (Dense) (None, 4096) 102764544 _________________________________________________________________ dropout_1 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_2 (Dense) (None, 4096) 16781312 _________________________________________________________________ dropout_2 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_3 (Dense) (None, 1000) 4097000 ================================================================= Total params: 138,357,544 Trainable params: 138,357,544 Non-trainable params: 0 _________________________________________________________________
vgg_19 = Sequential()
# first block
vgg_19.add(Conv2D(filters=64, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same',input_shape=(224,224, 3)))
vgg_19.add(Conv2D(filters=64, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(MaxPool2D((2,2), strides=(2,2)))
# second block
vgg_19.add(Conv2D(filters=128, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=128, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(MaxPool2D((2,2), strides=(2,2)))
# third block
vgg_19.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(MaxPool2D((2,2), strides=(2,2)))
# forth block
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(MaxPool2D((2,2), strides=(2,2)))
# fifth block
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(Conv2D(filters=512, kernel_size=(3,3), strides=(1,1), activation='relu', padding='same'))
vgg_19.add(MaxPool2D((2,2), strides=(2,2)))
# seventh block (classifier)
vgg_19.add(Flatten())
vgg_19.add(Dense(4096, activation='relu'))
vgg_19.add(Dropout(0.5))
vgg_19.add(Dense(4096, activation='relu'))
vgg_19.add(Dropout(0.5))
vgg_19.add(Dense(1000, activation='softmax'))
vgg_19.summary()
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_14 (Conv2D) (None, 224, 224, 64) 1792 _________________________________________________________________ conv2d_15 (Conv2D) (None, 224, 224, 64) 36928 _________________________________________________________________ max_pooling2d_6 (MaxPooling2 (None, 112, 112, 64) 0 _________________________________________________________________ conv2d_16 (Conv2D) (None, 112, 112, 128) 73856 _________________________________________________________________ conv2d_17 (Conv2D) (None, 112, 112, 128) 147584 _________________________________________________________________ max_pooling2d_7 (MaxPooling2 (None, 56, 56, 128) 0 _________________________________________________________________ conv2d_18 (Conv2D) (None, 56, 56, 256) 295168 _________________________________________________________________ conv2d_19 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ conv2d_20 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ conv2d_21 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ max_pooling2d_8 (MaxPooling2 (None, 28, 28, 256) 0 _________________________________________________________________ conv2d_22 (Conv2D) (None, 28, 28, 512) 1180160 _________________________________________________________________ conv2d_23 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ conv2d_24 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ conv2d_25 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ max_pooling2d_9 (MaxPooling2 (None, 14, 14, 512) 0 _________________________________________________________________ conv2d_26 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ conv2d_27 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ conv2d_28 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ conv2d_29 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ max_pooling2d_10 (MaxPooling (None, 7, 7, 512) 0 _________________________________________________________________ flatten_2 (Flatten) (None, 25088) 0 _________________________________________________________________ dense_4 (Dense) (None, 4096) 102764544 _________________________________________________________________ dropout_3 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_5 (Dense) (None, 4096) 16781312 _________________________________________________________________ dropout_4 (Dropout) (None, 4096) 0 _________________________________________________________________ dense_6 (Dense) (None, 1000) 4097000 ================================================================= Total params: 143,667,240 Trainable params: 143,667,240 Non-trainable params: 0 _________________________________________________________________