Manual Hyperparameter Tuning a Neural Network¶
Introduction¶
In Challenge 4, we built a simple neural network using Keras. In this challenge, we will manually experiment with hyperarameters.
This notebook contains the same workflow. Step 2 is the section that shows the simple 3-layer neural network from chapter 4. Section three is for you to change the same neural network by adding additional layers, neurons, and changing the number of epochs - then running the plot graph cell to see the results.
NOTE: Be sure to re-run the begining cells before working on Section 3.
Dataset¶
The advertising dataset captures the sales revenue generated with respect to advertisement costs across numerous platforms like radio, TV, and newspapers.
Features:¶
Digital: advertising dollars spent on Internet.¶
TV: advertising dollars spent on TV.¶
Radio: advertising dollars spent on Radio.¶
Newspaper: advertising dollars spent on Newspaper.¶
Target (Label):¶
Sales budget¶
Step 1: Data Preparation¶
Import Libraries¶
In [21]:
# Import the necessary libraries
# For Data loading, Exploraotry Data Analysis, Graphing
import pandas as pd # Pandas for data processing libraries
import numpy as np # Numpy for mathematical functions
import matplotlib.pyplot as plt # Matplotlib for visualization tasks
import seaborn as sns # Seaborn for data visualization library based on matplotlib.
%matplotlib inline
import sklearn # ML tasks
from sklearn.model_selection import train_test_split # Split the dataset
from sklearn.metrics import mean_squared_error # Calculate Mean Squared Error
# Build the Network
from tensorflow import keras
from keras.models import Sequential
#from tensorflow.keras.models import Sequential
from keras.layers import Dense
In [22]:
# Next, you read the dataset into a Pandas dataframe.
url = 'https://github.com/LinkedInLearning/artificial-intelligence-foundations-neural-networks-4381282/blob/main/Advertising_2023.csv?raw=true'
advertising_df= pd.read_csv(url,index_col=0)
In [23]:
# Pandas info() function is used to get a concise summary of the dataframe.
advertising_df.info()
<class 'pandas.core.frame.DataFrame'> Int64Index: 1199 entries, 1 to 1197 Data columns (total 5 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 digital 1199 non-null float64 1 TV 1199 non-null float64 2 radio 1199 non-null float64 3 newspaper 1199 non-null float64 4 sales 1199 non-null float64 dtypes: float64(5) memory usage: 56.2 KB
In [24]:
### Get summary of statistics of the data
advertising_df.describe()
Out[24]:
| digital | TV | radio | newspaper | sales | |
|---|---|---|---|---|---|
| count | 1199.000000 | 1199.00000 | 1199.000000 | 1199.000000 | 1199.000000 |
| mean | 135.472394 | 146.61985 | 23.240617 | 30.529942 | 14.005505 |
| std | 135.730821 | 85.61047 | 14.820827 | 21.712507 | 5.202804 |
| min | 0.300000 | 0.70000 | 0.000000 | 0.300000 | 1.600000 |
| 25% | 24.250000 | 73.40000 | 9.950000 | 12.800000 | 10.300000 |
| 50% | 64.650000 | 149.70000 | 22.500000 | 25.600000 | 12.900000 |
| 75% | 256.950000 | 218.50000 | 36.500000 | 45.100000 | 17.400000 |
| max | 444.600000 | 296.40000 | 49.600000 | 114.000000 | 27.000000 |
In [25]:
#shape of dataframe - 1199 rows, five columns
advertising_df.shape
Out[25]:
(1199, 5)
Let's check for any null values.
In [26]:
# The isnull() method is used to check and manage NULL values in a data frame.
advertising_df.isnull().sum()
Out[26]:
digital 0 TV 0 radio 0 newspaper 0 sales 0 dtype: int64
Exploratory Data Analysis (EDA)¶
Let's create some simple plots to check out the data!
In [27]:
## Plot the heatmap so that the values are shown.
plt.figure(figsize=(10,5))
sns.heatmap(advertising_df.corr(),annot=True,vmin=0,vmax=1,cmap='ocean')
Out[27]:
<Axes: >
In [28]:
#create a correlation matrix
corr = advertising_df.corr()
plt.figure(figsize=(10, 5))
sns.heatmap(corr[(corr >= 0.5) | (corr <= -0.7)],
cmap='viridis', vmax=1.0, vmin=-1.0, linewidths=0.1,
annot=True, annot_kws={"size": 8}, square=True)
plt.tight_layout()
display(plt.show())
None
In [29]:
advertising_df.corr()
Out[29]:
| digital | TV | radio | newspaper | sales | |
|---|---|---|---|---|---|
| digital | 1.000000 | 0.474256 | 0.041316 | 0.048023 | 0.380101 |
| TV | 0.474256 | 1.000000 | 0.055697 | 0.055579 | 0.781824 |
| radio | 0.041316 | 0.055697 | 1.000000 | 0.353096 | 0.576528 |
| newspaper | 0.048023 | 0.055579 | 0.353096 | 1.000000 | 0.227039 |
| sales | 0.380101 | 0.781824 | 0.576528 | 0.227039 | 1.000000 |
In [30]:
### Visualize Correlation
# Generate a mask for the upper triangle
mask = np.zeros_like(advertising_df.corr(), dtype=np.bool)
mask[np.triu_indices_from(mask)] = True
# Set up the matplotlib figure
f, ax = plt.subplots(figsize=(11, 9))
# Generate a custom diverging colormap
cmap = sns.diverging_palette(220, 10, as_cmap=True)
# Draw the heatmap with the mask and correct aspect ratio
sns.heatmap(advertising_df.corr(), mask=mask, cmap=cmap, vmax=.9, square=True, linewidths=.5, ax=ax)
<ipython-input-30-6c77a4103e7b>:4: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations mask = np.zeros_like(advertising_df.corr(), dtype=np.bool)
Out[30]:
<Axes: >
Since Sales is our target variable, we should identify which variable correlates the most with Sales.
As we can see, TV has the highest correlation with Sales. Let's visualize the relationship of variables using scatterplots.
Rather than plot them separately, an efficient way to view the linear relationsips between variables is to use a "for loop" that plots all of the features at once.
It seems there's no clear linear relationships between the predictors.
At this point, we know that the variable TV will more likely give better prediction of Sales because of the high correlation and linearity of the two.
In [31]:
'''=== Show the linear relationship between features and sales Thus, it provides that how the scattered
they are and which features has more impact in prediction of house price. ==='''
# visiualize all variables with sales
from scipy import stats
#creates figure
plt.figure(figsize=(18, 18))
for i, col in enumerate(advertising_df.columns[0:13]): #iterates over all columns except for price column (last one)
plt.subplot(5, 3, i+1) # each row three figure
x = advertising_df[col] #x-axis
y = advertising_df['sales'] #y-axis
plt.plot(x, y, 'o')
# Create regression line
plt.plot(np.unique(x), np.poly1d(np.polyfit(x, y, 1)) (np.unique(x)), color='red')
plt.xlabel(col) # x-label
plt.ylabel('sales') # y-label
Concluding results after observing the Graph The relation bw TV and Sales is stong and increases in linear fashion The relation bw Radio and Sales is less stong The relation bw TV and Sales is weak
Training a Linear Regression Model¶
Regression is a supervised machine learning process. It is similar to classification, but rather than predicting a label, you try to predict a continuous value. Linear regression defines the relationship between a target variable (y) and a set of predictive features (x). Simply stated, If you need to predict a number, then use regression.
Let's now begin to train your regression model! You will need to first split up your data into an X array that contains the features to train on, and a y array with the target variable, in this case the Price column. You will toss out the Address column because it only has text info that the linear regression model can't use.
Data Preprocessing¶
Split: X (features) and y (target)¶
Next, let's define the features and label. Briefly, feature is input; label is output. This applies to both classification and regression problems.
In [32]:
X = advertising_df[['digital', 'TV', 'radio', 'newspaper']]
y = advertising_df['sales']
Scaling (Normalization)¶
In [33]:
'''=== Noramlization the features. Since it is seen that features have different ranges, it is best practice to
normalize/standarize the feature before using them in the model ==='''
#feature normalization
normalized_feature = keras.utils.normalize(X.values)
Train - Test - Split¶
Now let's split the data into a training and test set. Note: Best pracices is to split into three - training, validation, and test set.
By default - It splits the given data into 75-25 ratio
In [34]:
# Import train_test_split function from sklearn.model_selection
from sklearn.model_selection import train_test_split
# Split up the data into a training set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=101)
In [35]:
print(X_train.shape,X_test.shape, y_train.shape, y_test.shape )
(719, 4) (480, 4) (719,) (480,)
Step 2: Build Network¶
Build and Train the Network¶
In [36]:
## Build Model (Building a three layer network - with one hidden layer)
model = Sequential()
model.add(Dense(4,input_dim=4, activation='relu')) # You don't have to specify input size.Just define the hidden layers
model.add(Dense(3,activation='relu'))
model.add(Dense(1))
# Compile Model
model.compile(optimizer='adam', loss='mse',metrics=['mse'])
# Fit the Model
history = model.fit(X_train, y_train, validation_data = (X_test, y_test),
epochs = 32)
Epoch 1/32 23/23 [==============================] - 1s 15ms/step - loss: 50.4749 - mse: 50.4749 - val_loss: 36.3439 - val_mse: 36.3439 Epoch 2/32 23/23 [==============================] - 0s 4ms/step - loss: 27.4183 - mse: 27.4183 - val_loss: 22.9221 - val_mse: 22.9221 Epoch 3/32 23/23 [==============================] - 0s 5ms/step - loss: 18.0418 - mse: 18.0418 - val_loss: 15.3263 - val_mse: 15.3263 Epoch 4/32 23/23 [==============================] - 0s 5ms/step - loss: 12.4106 - mse: 12.4106 - val_loss: 10.9890 - val_mse: 10.9890 Epoch 5/32 23/23 [==============================] - 0s 4ms/step - loss: 9.2734 - mse: 9.2734 - val_loss: 8.2866 - val_mse: 8.2866 Epoch 6/32 23/23 [==============================] - 0s 7ms/step - loss: 7.3088 - mse: 7.3088 - val_loss: 6.7661 - val_mse: 6.7661 Epoch 7/32 23/23 [==============================] - 0s 5ms/step - loss: 6.1253 - mse: 6.1253 - val_loss: 5.7897 - val_mse: 5.7897 Epoch 8/32 23/23 [==============================] - 0s 5ms/step - loss: 5.4477 - mse: 5.4477 - val_loss: 5.1563 - val_mse: 5.1563 Epoch 9/32 23/23 [==============================] - 0s 6ms/step - loss: 4.9096 - mse: 4.9096 - val_loss: 4.7248 - val_mse: 4.7248 Epoch 10/32 23/23 [==============================] - 0s 6ms/step - loss: 4.5706 - mse: 4.5706 - val_loss: 4.4026 - val_mse: 4.4026 Epoch 11/32 23/23 [==============================] - 0s 6ms/step - loss: 4.2998 - mse: 4.2998 - val_loss: 4.1570 - val_mse: 4.1570 Epoch 12/32 23/23 [==============================] - 0s 4ms/step - loss: 4.0745 - mse: 4.0745 - val_loss: 3.9596 - val_mse: 3.9596 Epoch 13/32 23/23 [==============================] - 0s 4ms/step - loss: 3.8879 - mse: 3.8879 - val_loss: 3.8291 - val_mse: 3.8291 Epoch 14/32 23/23 [==============================] - 0s 6ms/step - loss: 3.7615 - mse: 3.7615 - val_loss: 3.7142 - val_mse: 3.7142 Epoch 15/32 23/23 [==============================] - 0s 6ms/step - loss: 3.6386 - mse: 3.6386 - val_loss: 3.6410 - val_mse: 3.6410 Epoch 16/32 23/23 [==============================] - 0s 5ms/step - loss: 3.5446 - mse: 3.5446 - val_loss: 3.5481 - val_mse: 3.5481 Epoch 17/32 23/23 [==============================] - 0s 4ms/step - loss: 3.4425 - mse: 3.4425 - val_loss: 3.4898 - val_mse: 3.4898 Epoch 18/32 23/23 [==============================] - 0s 5ms/step - loss: 3.3588 - mse: 3.3588 - val_loss: 3.4327 - val_mse: 3.4327 Epoch 19/32 23/23 [==============================] - 0s 5ms/step - loss: 3.3429 - mse: 3.3429 - val_loss: 3.3852 - val_mse: 3.3852 Epoch 20/32 23/23 [==============================] - 0s 4ms/step - loss: 3.2292 - mse: 3.2292 - val_loss: 3.3472 - val_mse: 3.3472 Epoch 21/32 23/23 [==============================] - 0s 4ms/step - loss: 3.1885 - mse: 3.1885 - val_loss: 3.3065 - val_mse: 3.3065 Epoch 22/32 23/23 [==============================] - 0s 5ms/step - loss: 3.1413 - mse: 3.1413 - val_loss: 3.2622 - val_mse: 3.2622 Epoch 23/32 23/23 [==============================] - 0s 5ms/step - loss: 3.0911 - mse: 3.0911 - val_loss: 3.2725 - val_mse: 3.2725 Epoch 24/32 23/23 [==============================] - 0s 6ms/step - loss: 3.0540 - mse: 3.0540 - val_loss: 3.2128 - val_mse: 3.2128 Epoch 25/32 23/23 [==============================] - 0s 4ms/step - loss: 3.0387 - mse: 3.0387 - val_loss: 3.2104 - val_mse: 3.2104 Epoch 26/32 23/23 [==============================] - 0s 5ms/step - loss: 2.9848 - mse: 2.9848 - val_loss: 3.1682 - val_mse: 3.1682 Epoch 27/32 23/23 [==============================] - 0s 7ms/step - loss: 2.9165 - mse: 2.9165 - val_loss: 3.1106 - val_mse: 3.1106 Epoch 28/32 23/23 [==============================] - 0s 5ms/step - loss: 2.8810 - mse: 2.8810 - val_loss: 3.0621 - val_mse: 3.0621 Epoch 29/32 23/23 [==============================] - 0s 6ms/step - loss: 2.8477 - mse: 2.8477 - val_loss: 3.0435 - val_mse: 3.0435 Epoch 30/32 23/23 [==============================] - 0s 7ms/step - loss: 2.8276 - mse: 2.8276 - val_loss: 3.0129 - val_mse: 3.0129 Epoch 31/32 23/23 [==============================] - 0s 6ms/step - loss: 2.7864 - mse: 2.7864 - val_loss: 3.0320 - val_mse: 3.0320 Epoch 32/32 23/23 [==============================] - 0s 7ms/step - loss: 2.7550 - mse: 2.7550 - val_loss: 2.9606 - val_mse: 2.9606
Visualization¶
You can add more 'flavor' to the graph by making it bigger and adding labels and names, as shown below.
In [37]:
## Plot a graph of model loss # show the graph of model loss in trainig and validation
plt.figure(figsize=(15,8))
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss (MSE) on Training and Validation Data')
plt.ylabel('Loss-Mean Squred Error')
plt.xlabel('Epoch')
plt.legend(['Val Loss', 'Train Loss'], loc='upper right')
plt.show()
Step 3: Tune the Neural Network Hyperparameters¶
CHALLENGE: Manual Hyperparameter Tuning¶
Play with:
- Adding additional layers
- Adding additional neurons
- Change the number of epochs.
NOTE: After each change, run the cell to plot the graph and review the loss curves.
In [40]:
## Build Model
model = Sequential()
model.add(Dense(4,input_dim=4, activation='relu'))
model.add(Dense(4,activation='relu'))
model.add(Dense(3,activation='relu'))
model.add(Dense(1))
# Compile Model
model.compile(optimizer='adam', loss='mse',metrics=['mse'])
# Fit the Model
history = model.fit(X_train, y_train, validation_data = (X_test, y_test),
epochs = 100)
Epoch 1/100 23/23 [==============================] - 2s 35ms/step - loss: 3506.0830 - mse: 3506.0830 - val_loss: 2508.3508 - val_mse: 2508.3508 Epoch 2/100 23/23 [==============================] - 0s 7ms/step - loss: 2138.6150 - mse: 2138.6150 - val_loss: 1538.7827 - val_mse: 1538.7827 Epoch 3/100 23/23 [==============================] - 0s 5ms/step - loss: 1385.6256 - mse: 1385.6256 - val_loss: 1034.8229 - val_mse: 1034.8229 Epoch 4/100 23/23 [==============================] - 0s 5ms/step - loss: 960.9625 - mse: 960.9625 - val_loss: 749.4362 - val_mse: 749.4362 Epoch 5/100 23/23 [==============================] - 0s 7ms/step - loss: 711.8473 - mse: 711.8473 - val_loss: 557.6386 - val_mse: 557.6386 Epoch 6/100 23/23 [==============================] - 0s 9ms/step - loss: 542.9380 - mse: 542.9380 - val_loss: 426.8971 - val_mse: 426.8971 Epoch 7/100 23/23 [==============================] - 0s 9ms/step - loss: 415.0869 - mse: 415.0869 - val_loss: 331.9332 - val_mse: 331.9332 Epoch 8/100 23/23 [==============================] - 0s 8ms/step - loss: 321.8935 - mse: 321.8935 - val_loss: 257.6204 - val_mse: 257.6204 Epoch 9/100 23/23 [==============================] - 0s 6ms/step - loss: 249.4613 - mse: 249.4613 - val_loss: 196.5948 - val_mse: 196.5948 Epoch 10/100 23/23 [==============================] - 0s 6ms/step - loss: 190.8183 - mse: 190.8183 - val_loss: 151.0053 - val_mse: 151.0053 Epoch 11/100 23/23 [==============================] - 0s 7ms/step - loss: 147.6170 - mse: 147.6170 - val_loss: 115.5755 - val_mse: 115.5755 Epoch 12/100 23/23 [==============================] - 0s 6ms/step - loss: 113.8717 - mse: 113.8717 - val_loss: 90.6626 - val_mse: 90.6626 Epoch 13/100 23/23 [==============================] - 0s 10ms/step - loss: 90.4448 - mse: 90.4448 - val_loss: 73.7393 - val_mse: 73.7393 Epoch 14/100 23/23 [==============================] - 0s 6ms/step - loss: 74.5241 - mse: 74.5241 - val_loss: 62.6111 - val_mse: 62.6111 Epoch 15/100 23/23 [==============================] - 0s 7ms/step - loss: 63.9885 - mse: 63.9885 - val_loss: 55.8501 - val_mse: 55.8501 Epoch 16/100 23/23 [==============================] - 0s 8ms/step - loss: 57.6333 - mse: 57.6333 - val_loss: 52.0885 - val_mse: 52.0885 Epoch 17/100 23/23 [==============================] - 0s 8ms/step - loss: 53.7017 - mse: 53.7017 - val_loss: 50.1687 - val_mse: 50.1687 Epoch 18/100 23/23 [==============================] - 0s 8ms/step - loss: 51.2566 - mse: 51.2566 - val_loss: 48.5141 - val_mse: 48.5141 Epoch 19/100 23/23 [==============================] - 0s 9ms/step - loss: 49.0285 - mse: 49.0285 - val_loss: 46.2384 - val_mse: 46.2384 Epoch 20/100 23/23 [==============================] - 0s 8ms/step - loss: 46.0897 - mse: 46.0897 - val_loss: 40.9961 - val_mse: 40.9961 Epoch 21/100 23/23 [==============================] - 0s 4ms/step - loss: 37.8058 - mse: 37.8058 - val_loss: 29.0406 - val_mse: 29.0406 Epoch 22/100 23/23 [==============================] - 0s 5ms/step - loss: 27.3646 - mse: 27.3646 - val_loss: 22.6871 - val_mse: 22.6871 Epoch 23/100 23/23 [==============================] - 0s 4ms/step - loss: 21.1398 - mse: 21.1398 - val_loss: 18.1918 - val_mse: 18.1918 Epoch 24/100 23/23 [==============================] - 0s 4ms/step - loss: 17.0045 - mse: 17.0045 - val_loss: 15.2210 - val_mse: 15.2210 Epoch 25/100 23/23 [==============================] - 0s 6ms/step - loss: 14.2455 - mse: 14.2455 - val_loss: 13.1136 - val_mse: 13.1136 Epoch 26/100 23/23 [==============================] - 0s 4ms/step - loss: 12.2923 - mse: 12.2923 - val_loss: 11.8689 - val_mse: 11.8689 Epoch 27/100 23/23 [==============================] - 0s 5ms/step - loss: 11.0822 - mse: 11.0822 - val_loss: 11.0416 - val_mse: 11.0416 Epoch 28/100 23/23 [==============================] - 0s 6ms/step - loss: 10.0864 - mse: 10.0864 - val_loss: 10.2116 - val_mse: 10.2116 Epoch 29/100 23/23 [==============================] - 0s 6ms/step - loss: 9.3629 - mse: 9.3629 - val_loss: 9.6737 - val_mse: 9.6737 Epoch 30/100 23/23 [==============================] - 0s 5ms/step - loss: 8.7603 - mse: 8.7603 - val_loss: 9.3001 - val_mse: 9.3001 Epoch 31/100 23/23 [==============================] - 0s 4ms/step - loss: 8.3676 - mse: 8.3676 - val_loss: 8.9180 - val_mse: 8.9180 Epoch 32/100 23/23 [==============================] - 0s 4ms/step - loss: 7.9829 - mse: 7.9829 - val_loss: 8.6022 - val_mse: 8.6022 Epoch 33/100 23/23 [==============================] - 0s 7ms/step - loss: 7.6926 - mse: 7.6926 - val_loss: 8.3250 - val_mse: 8.3250 Epoch 34/100 23/23 [==============================] - 0s 6ms/step - loss: 7.4608 - mse: 7.4608 - val_loss: 8.1087 - val_mse: 8.1087 Epoch 35/100 23/23 [==============================] - 0s 4ms/step - loss: 7.2107 - mse: 7.2107 - val_loss: 7.8911 - val_mse: 7.8911 Epoch 36/100 23/23 [==============================] - 0s 6ms/step - loss: 6.9898 - mse: 6.9898 - val_loss: 7.5872 - val_mse: 7.5872 Epoch 37/100 23/23 [==============================] - 0s 7ms/step - loss: 6.8245 - mse: 6.8245 - val_loss: 7.3937 - val_mse: 7.3937 Epoch 38/100 23/23 [==============================] - 0s 4ms/step - loss: 6.6139 - mse: 6.6139 - val_loss: 7.2311 - val_mse: 7.2311 Epoch 39/100 23/23 [==============================] - 0s 5ms/step - loss: 6.4406 - mse: 6.4406 - val_loss: 6.9990 - val_mse: 6.9990 Epoch 40/100 23/23 [==============================] - 0s 4ms/step - loss: 6.2684 - mse: 6.2684 - val_loss: 6.8597 - val_mse: 6.8597 Epoch 41/100 23/23 [==============================] - 0s 5ms/step - loss: 6.1341 - mse: 6.1341 - val_loss: 6.7209 - val_mse: 6.7209 Epoch 42/100 23/23 [==============================] - 0s 7ms/step - loss: 6.0117 - mse: 6.0117 - val_loss: 6.5869 - val_mse: 6.5869 Epoch 43/100 23/23 [==============================] - 0s 5ms/step - loss: 5.8700 - mse: 5.8700 - val_loss: 6.3684 - val_mse: 6.3684 Epoch 44/100 23/23 [==============================] - 0s 7ms/step - loss: 5.7430 - mse: 5.7430 - val_loss: 6.2055 - val_mse: 6.2055 Epoch 45/100 23/23 [==============================] - 0s 7ms/step - loss: 5.6399 - mse: 5.6399 - val_loss: 6.0564 - val_mse: 6.0564 Epoch 46/100 23/23 [==============================] - 0s 4ms/step - loss: 5.5278 - mse: 5.5278 - val_loss: 5.9825 - val_mse: 5.9825 Epoch 47/100 23/23 [==============================] - 0s 6ms/step - loss: 5.4394 - mse: 5.4394 - val_loss: 5.9016 - val_mse: 5.9016 Epoch 48/100 23/23 [==============================] - 0s 5ms/step - loss: 5.3608 - mse: 5.3608 - val_loss: 5.8026 - val_mse: 5.8026 Epoch 49/100 23/23 [==============================] - 0s 4ms/step - loss: 5.2566 - mse: 5.2566 - val_loss: 5.7005 - val_mse: 5.7005 Epoch 50/100 23/23 [==============================] - 0s 6ms/step - loss: 5.1707 - mse: 5.1707 - val_loss: 5.6008 - val_mse: 5.6008 Epoch 51/100 23/23 [==============================] - 0s 6ms/step - loss: 5.1398 - mse: 5.1398 - val_loss: 5.4763 - val_mse: 5.4763 Epoch 52/100 23/23 [==============================] - 0s 6ms/step - loss: 5.0451 - mse: 5.0451 - val_loss: 5.3944 - val_mse: 5.3944 Epoch 53/100 23/23 [==============================] - 0s 5ms/step - loss: 4.9759 - mse: 4.9759 - val_loss: 5.3346 - val_mse: 5.3346 Epoch 54/100 23/23 [==============================] - 0s 4ms/step - loss: 4.8913 - mse: 4.8913 - val_loss: 5.2506 - val_mse: 5.2506 Epoch 55/100 23/23 [==============================] - 0s 4ms/step - loss: 4.8265 - mse: 4.8265 - val_loss: 5.1479 - val_mse: 5.1479 Epoch 56/100 23/23 [==============================] - 0s 7ms/step - loss: 4.7823 - mse: 4.7823 - val_loss: 5.1077 - val_mse: 5.1077 Epoch 57/100 23/23 [==============================] - 0s 6ms/step - loss: 4.7346 - mse: 4.7346 - val_loss: 5.0210 - val_mse: 5.0210 Epoch 58/100 23/23 [==============================] - 0s 4ms/step - loss: 4.6689 - mse: 4.6689 - val_loss: 4.9335 - val_mse: 4.9335 Epoch 59/100 23/23 [==============================] - 0s 4ms/step - loss: 4.6378 - mse: 4.6378 - val_loss: 4.8900 - val_mse: 4.8900 Epoch 60/100 23/23 [==============================] - 0s 5ms/step - loss: 4.5987 - mse: 4.5987 - val_loss: 4.8383 - val_mse: 4.8383 Epoch 61/100 23/23 [==============================] - 0s 5ms/step - loss: 4.5841 - mse: 4.5841 - val_loss: 4.7810 - val_mse: 4.7810 Epoch 62/100 23/23 [==============================] - 0s 6ms/step - loss: 4.5086 - mse: 4.5086 - val_loss: 4.7339 - val_mse: 4.7339 Epoch 63/100 23/23 [==============================] - 0s 6ms/step - loss: 4.4555 - mse: 4.4555 - val_loss: 4.6665 - val_mse: 4.6665 Epoch 64/100 23/23 [==============================] - 0s 6ms/step - loss: 4.4198 - mse: 4.4198 - val_loss: 4.6079 - val_mse: 4.6079 Epoch 65/100 23/23 [==============================] - 0s 10ms/step - loss: 4.3743 - mse: 4.3743 - val_loss: 4.5643 - val_mse: 4.5643 Epoch 66/100 23/23 [==============================] - 0s 11ms/step - loss: 4.3301 - mse: 4.3301 - val_loss: 4.4769 - val_mse: 4.4769 Epoch 67/100 23/23 [==============================] - 0s 10ms/step - loss: 4.3014 - mse: 4.3014 - val_loss: 4.4467 - val_mse: 4.4467 Epoch 68/100 23/23 [==============================] - 0s 10ms/step - loss: 4.2559 - mse: 4.2559 - val_loss: 4.4027 - val_mse: 4.4027 Epoch 69/100 23/23 [==============================] - 0s 14ms/step - loss: 4.2372 - mse: 4.2372 - val_loss: 4.3380 - val_mse: 4.3380 Epoch 70/100 23/23 [==============================] - 0s 10ms/step - loss: 4.1748 - mse: 4.1748 - val_loss: 4.2853 - val_mse: 4.2853 Epoch 71/100 23/23 [==============================] - 0s 8ms/step - loss: 4.1665 - mse: 4.1665 - val_loss: 4.2983 - val_mse: 4.2983 Epoch 72/100 23/23 [==============================] - 0s 10ms/step - loss: 4.1231 - mse: 4.1231 - val_loss: 4.1919 - val_mse: 4.1919 Epoch 73/100 23/23 [==============================] - 0s 14ms/step - loss: 4.1004 - mse: 4.1004 - val_loss: 4.2340 - val_mse: 4.2340 Epoch 74/100 23/23 [==============================] - 0s 11ms/step - loss: 4.0443 - mse: 4.0443 - val_loss: 4.1107 - val_mse: 4.1107 Epoch 75/100 23/23 [==============================] - 0s 7ms/step - loss: 4.0370 - mse: 4.0370 - val_loss: 4.1593 - val_mse: 4.1593 Epoch 76/100 23/23 [==============================] - 0s 5ms/step - loss: 3.9813 - mse: 3.9813 - val_loss: 3.9958 - val_mse: 3.9958 Epoch 77/100 23/23 [==============================] - 0s 4ms/step - loss: 3.9562 - mse: 3.9562 - val_loss: 3.9837 - val_mse: 3.9837 Epoch 78/100 23/23 [==============================] - 0s 5ms/step - loss: 3.9416 - mse: 3.9416 - val_loss: 3.9131 - val_mse: 3.9131 Epoch 79/100 23/23 [==============================] - 0s 5ms/step - loss: 3.8706 - mse: 3.8706 - val_loss: 3.9065 - val_mse: 3.9065 Epoch 80/100 23/23 [==============================] - 0s 7ms/step - loss: 3.8327 - mse: 3.8327 - val_loss: 3.8323 - val_mse: 3.8323 Epoch 81/100 23/23 [==============================] - 0s 4ms/step - loss: 3.8026 - mse: 3.8026 - val_loss: 3.7951 - val_mse: 3.7951 Epoch 82/100 23/23 [==============================] - 0s 5ms/step - loss: 3.7533 - mse: 3.7533 - val_loss: 3.7478 - val_mse: 3.7478 Epoch 83/100 23/23 [==============================] - 0s 5ms/step - loss: 3.7175 - mse: 3.7175 - val_loss: 3.8240 - val_mse: 3.8240 Epoch 84/100 23/23 [==============================] - 0s 4ms/step - loss: 3.7260 - mse: 3.7260 - val_loss: 3.7256 - val_mse: 3.7256 Epoch 85/100 23/23 [==============================] - 0s 7ms/step - loss: 3.6750 - mse: 3.6750 - val_loss: 3.6248 - val_mse: 3.6248 Epoch 86/100 23/23 [==============================] - 0s 7ms/step - loss: 3.6505 - mse: 3.6505 - val_loss: 3.6168 - val_mse: 3.6168 Epoch 87/100 23/23 [==============================] - 0s 9ms/step - loss: 3.6059 - mse: 3.6059 - val_loss: 3.5766 - val_mse: 3.5766 Epoch 88/100 23/23 [==============================] - 0s 8ms/step - loss: 3.5425 - mse: 3.5425 - val_loss: 3.4958 - val_mse: 3.4958 Epoch 89/100 23/23 [==============================] - 0s 9ms/step - loss: 3.5188 - mse: 3.5188 - val_loss: 3.5057 - val_mse: 3.5057 Epoch 90/100 23/23 [==============================] - 0s 9ms/step - loss: 3.4880 - mse: 3.4880 - val_loss: 3.4756 - val_mse: 3.4756 Epoch 91/100 23/23 [==============================] - 0s 7ms/step - loss: 3.4640 - mse: 3.4640 - val_loss: 3.4473 - val_mse: 3.4473 Epoch 92/100 23/23 [==============================] - 0s 8ms/step - loss: 3.4121 - mse: 3.4121 - val_loss: 3.4104 - val_mse: 3.4104 Epoch 93/100 23/23 [==============================] - 0s 9ms/step - loss: 3.4050 - mse: 3.4050 - val_loss: 3.3817 - val_mse: 3.3817 Epoch 94/100 23/23 [==============================] - 0s 9ms/step - loss: 3.3259 - mse: 3.3259 - val_loss: 3.3454 - val_mse: 3.3454 Epoch 95/100 23/23 [==============================] - 0s 7ms/step - loss: 3.2848 - mse: 3.2848 - val_loss: 3.3096 - val_mse: 3.3096 Epoch 96/100 23/23 [==============================] - 0s 7ms/step - loss: 3.2267 - mse: 3.2267 - val_loss: 3.3464 - val_mse: 3.3464 Epoch 97/100 23/23 [==============================] - 0s 9ms/step - loss: 3.1978 - mse: 3.1978 - val_loss: 3.1914 - val_mse: 3.1914 Epoch 98/100 23/23 [==============================] - 0s 9ms/step - loss: 3.1454 - mse: 3.1454 - val_loss: 3.1706 - val_mse: 3.1706 Epoch 99/100 23/23 [==============================] - 0s 8ms/step - loss: 3.1417 - mse: 3.1417 - val_loss: 3.1463 - val_mse: 3.1463 Epoch 100/100 23/23 [==============================] - 0s 10ms/step - loss: 3.1047 - mse: 3.1047 - val_loss: 3.1292 - val_mse: 3.1292
In [41]:
plt.figure(figsize=(15,8))
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss (MSE) on Training and Validation Data')
plt.ylabel('Loss-Mean Squred Error')
plt.xlabel('Epoch')
plt.legend(['Val Loss', 'Train Loss'], loc='upper right')
plt.show()
