To run any of Eden's notebooks, please check the guides on our Wiki page.
There you will find instructions on how to deploy the notebooks on your local system, on Google Colab, or on MyBinder, as well as other useful links, troubleshooting tips, and more.
For this notebook you will need to download the Cotton-100619-Healthy-zz-V1-20210225102300, Black nightsade-220519-Weed-zz-V1-20210225102034, Tomato-240519-Healthy-zz-V1-20210225103740 and Velvet leaf-220519-Weed-zz-V1-20210225104123 datasets from Eden Library, and you may want to use the eden_tensorflow_transfer_learning.yml file to recreate a suitable conda environment.
Note: If you find any issues while executing the notebook, don't hesitate to open an issue on Github. We will try to reply as soon as possible.
In this notebook, we are going to cover a technique called Transfer Learning, which generally refers to a process where a machine learning model is trained on one problem, and afterwards, it is reused in some way on a second (possibly) related problem (Bengio, 2012). Specifically, in deep learning, this technique is used by training only some layers of the pre-trained network. Its promise is that the training will be more efficient and in the best of the cases the performance will be better compared to a model trained from scratch. In this example we are using MobileNet architectures.
MobileNet V1, with its appearance in 2017, started a new path of deep learning research in computer vision: models that can run in embedded devices. This lead to several important works including ShuffleNet(V1 and V2), MNasNet, CondenseNet, and EffNet (among others). Due to its success, new versions of MobileNet have also appeared more recently. These new versions included new advances, such as Inverted Residual Block or the use of Neural Architecture Search for optimizing the creation of new neural components. MobileNet Versions 2 (Sandler et al., 2018) and 3 (Howard et al., 2019) are used as an example in this notebook.
In agriculture, since weeds compete with crops in the domain of space, light and nutrients, they are an important problem that can lead to a poorer harvest by farmers. To avoid this, weeds should be removed at every step of the growth, but especially at the initial stages. For that reason, identifying weeds accurately by deep learning has arisen as an important objective. Related to this, we can find the disease detection problem, where transfer learning has also been used. Among the most relevant recent works, we can find:
Wang et al., (2017) used transfer learning in order to obtain the best neural-based method for disease detection in plants. They extended the apple black rot images in the PlantVillage dataset, which were further annotated by botanists with four severity stages as ground truth. Then, they evaluated the performances of shallow networks trained from scratch and deep models fine-tuned by transfer learning. Their best model was the VGG16 architecture trained with transfer learning, which yielded an overall accuracy of 90.4% on the hold-out test set. In Mehdipour-Ghazi et al., (2017), the authors used the plant datasets of LifeCLEF 2015. Three popular deep learning architectures were evaluated: GoogLeNet, AlexNet, and VGGNet. Their best combined system (a combination of GoogleNet and VGGNet) achieved an overall accuracy of 80% on the validation set and an overall inverse rank score of 0.752 on the official test set. In Suh et al., (2018), the authors compared different transfer learning approaches in order to find a suitable approach for weed detection (volunteer potato). Their highest classification accuracy for AlexNet was 98.0%. Comparing different networks, their highest classification accuracy was 98.7%, which was obtained with VGG-19. Additionally, all scenarios and pre-trained networks were feasible for real-time applications (classification time < 0.1 s). Another relevant study has been performed by Kounalakis et al., (2019) where they evaluated transfer learning by a combination of CNN-based feature extraction and linear classifiers to recognize rumex under real-world conditions. Their best system (Inception_v1+L2regLogReg) achieved an accuracy of 96.13 with a false positive rate of 3.62. In Too et al., (2019), the authors used transfer learning achieving a performance of 99.75% with the DenseNet architecture. Finally, in Espejo-Garcia et al., (2020), authors used transfer learning using agricultural datasets for pre-training neural networks, and afterwards, they fine-tuned the networks for classifying 4 species extracted from the Eden Platform. Their maximum performance was 99.54% by using the Xception architecture.
It is important to note that in this notebook a technique integrating neural-network feature-extraction and "traditional" machine learning algorithms is used. This technique was used in Kounalakis et al., (2019) and Espejo-Garcia et al., (2020), and represents an extension over the previous Eden notebooks:
import warnings
warnings.filterwarnings("ignore")
import numpy as np
import cv2
import os
import csv
import random
import matplotlib.pyplot as plt
from tqdm import tqdm
from glob import glob
from pathlib import Path
import tensorflow as tf
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.applications import *
from tensorflow.keras import layers
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.callbacks import ReduceLROnPlateau
import tensorflow.keras.backend as K
from sklearn.model_selection import train_test_split
from sklearn.metrics import f1_score
from tensorflow.keras.layers.experimental import preprocessing
from tensorflow.keras.models import Sequential
from tensorflow.keras.applications.mobilenet_v3 import preprocess_input
Check the docstrings for more information.
def plot_sample(X):
"""
Given the array of images <X>, it plots a random subsample of 9 of them.
Parameters:
X (ndarray): The array with all the images.
"""
nb_rows = 3
nb_cols = 3
fig, axs = plt.subplots(nb_rows, nb_cols, figsize=(6, 6))
for i in range(0, nb_rows):
for j in range(0, nb_cols):
axs[i, j].xaxis.set_ticklabels([])
axs[i, j].yaxis.set_ticklabels([])
axs[i, j].imshow(X[random.randint(0, X.shape[0] - 1)])
def read_data(path_list, im_size=(224, 224)):
"""
Given the list of paths where the images are stored <path_list>,
and the size for image decimation <im_size>, it returns 2 Numpy Arrays
with the images and labels.
Parameters:
path_list (List[String]): The list of paths to the images.
im_size (Tuple): The height and width values.
Returns:
X (ndarray): Images
y (ndarray): Labels
"""
X = []
y = []
# Extract filenames of the datasets we ingest and create a label dictionary
tag2idx = {tag.split(os.path.sep)[-1]: i for i, tag in enumerate(path_list)}
for path in path_list:
for im_file in tqdm(glob(path + "*/*")): # Read all files in path
try:
# os.path.separator is OS agnostic (either '/' or '\'),[-2] to grab folder name
label = im_file.split(os.path.sep)[-2]
im = cv2.imread(im_file, cv2.IMREAD_COLOR)
# By default OpenCV reads with BGR format, convert to RGB
im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
# Resize to appropriate dimensions
im = cv2.resize(im, im_size, interpolation=cv2.INTER_AREA)
X.append(im)
y.append(tag2idx[label]) # Append the label name to y
except Exception as e:
# In case annotations or metadata are found
print("Not a picture")
X = np.array(X) # Convert list to numpy array.
y = np.eye(len(np.unique(y)))[y].astype(np.uint8)
return X, y
def get_callbacks(weights_file, patience, lr_factor):
"""
Callbacks are used for saving the best weights and early stopping.
Given some configuration parameters, it creates the callbacks that
will be used by Keras after each epoch.
Parameters:
weights_file (String): File name for saving the best model weights.
patience (Integer): Number of epochs without improvement to wait.
lr_factor: Factor for reducing the learning rate when performance
is not improving.
Returns:
callbacks (List[Callbacks]): Configured callback ready for using.
"""
return [
# Only save weights that correspond to the maximum validation accuracy
ModelCheckpoint(
filepath=weights_file,
monitor="val_accuracy",
mode="max",
save_best_only=True,
save_weights_only=True,
),
# If val_loss does not improve after a number of epochs (patience),
# training will stop to avoid overfitting.
EarlyStopping(monitor="val_loss", mode="min", patience=patience, verbose=1),
# Learning rate is reduced by 'lr_factor' if val_loss stagnates
# for a number of epochs (patience).
# ReduceLROnPlateau(monitor="val_loss", mode="min",
# factor=lr_factor, min_lr=1e-6, patience=patience//2, verbose=1 )
]
def plot_performances(performances):
"""
Given the list of performances (validation accuracies) and method-name <performances>,
it plots how the validation accuracy progressed during the training/validation process.
Parameters:
performances (List[Tuple]): The list of method-performance tuples.
"""
plt.title("Validation Accuracy vs. Number of Training Epochs")
plt.xlabel("Training Epochs")
plt.ylabel("Validation Accuracy")
for performance in performances:
plt.plot(
range(1, len(performance[1]) + 1), performance[1], label=performance[0]
)
plt.ylim((0.5, 1.05))
plt.xticks(np.arange(1, NUM_EPOCHS + 1, 1.0))
plt.legend()
plt.show()
INPUT_SHAPE = (224, 224, 3)
IM_SIZE = (224, 224)
NUM_EPOCHS = 20
BATCH_SIZE = 4
TEST_SPLIT = 0.2
VAL_SPLIT = 0.2
RANDOM_STATE = 2021
WEIGHTS_FILE = "weights.h5" # File to store updated weights
# Eden datasets we will work on
PATH_LIST = [
"Cotton-100619-Healthy-zz-V1-20210225102300",
"Black nightsade-220519-Weed-zz-V1-20210225102034",
"Tomato-240519-Healthy-zz-V1-20210225103740",
"Velvet leaf-220519-Weed-zz-V1-20210225104123",
]
i = 0
for path in PATH_LIST:
# Define paths in an OS agnostic way.
PATH_LIST[i] = str(
Path(Path.cwd()).parents[0].joinpath("eden_library_datasets").joinpath(path)
)
i += 1
X, y = read_data(PATH_LIST, IM_SIZE)
plot_sample(X)
The TF augmentation layers are included as part of the model architecture.
img_augmentation = Sequential(
[
preprocessing.RandomRotation(factor=0.15),
preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),
preprocessing.RandomFlip(),
preprocessing.RandomContrast(factor=0.1),
],
name="img_augmentation",
)
IMAGE_IX = 10
image = tf.expand_dims(X[IMAGE_IX], axis=0)
plt.figure(figsize=(8, 8))
for i in range(9):
ax = plt.subplot(3, 3, i + 1)
aug_img = img_augmentation(image)
plt.imshow(aug_img[0].numpy().astype("uint8"))
plt.axis("off")
plt.show()
def get_architecture(y, mobilenet_size, as_feature_extractor):
"""
This function returns a pre-trained Tensorflow MobileNet architecture.
Versions 2 and 3-Large are available.
Parameters:
y (ndarray): Array with labels. It is used for computing the number of classes.
mobilenet_version (string): which MobileNet version to import (2 or 3 Large).
as_feature_extractor (Boolean): Whether to fine-tune the "convolutional" layers.
Returns:
model (Model): MobileNet architecture ready for training.
"""
inputs = layers.Input(shape=INPUT_SHAPE)
input_aug = img_augmentation(inputs)
input_norm = layers.Lambda(preprocess_input)(input_aug) # placeholder
if mobilenet_size == "small":
feature_extractor = MobileNetV3Small(
weights="imagenet", include_top=False, input_tensor=input_norm
)
elif mobilenet_size == "large":
feature_extractor = MobileNetV3Large(
weights="imagenet", include_top=False, input_tensor=input_norm
)
# Freeze the base_model as we do not want to update the initial weights
if as_feature_extractor:
feature_extractor.trainable = False
else:
feature_extractor.trainable = True # Not necessary
# Create new model on top
features = layers.GlobalAveragePooling2D(name="pool")(
feature_extractor.output
) # Flattening layer
fully = layers.Dense(units=64, activation="relu")(features) # Fully connected layer
# Create a Classifier with shape=number_of_training_classes
fully = layers.Dropout(0.3)(fully) # Regularize with dropout
out = layers.Dense(units=y.shape[1], activation="softmax")(fully)
model = Model(inputs, out) # Final model
# Defining base learning rate for Adam optimizer
base_learning_rate = 1e-4
model.compile(
loss="categorical_crossentropy",
optimizer=tf.keras.optimizers.Adam(lr=base_learning_rate),
metrics=["accuracy"],
)
return model
When using transfer learning, there are several strategies for reusing the weights or the knowledge learned on another task. Here 2 strategies are shown and compared:
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=TEST_SPLIT,
shuffle=True,
stratify=y,
random_state=RANDOM_STATE
)
X_train, X_val, y_train, y_val = train_test_split(
X_train,
y_train,
test_size=VAL_SPLIT,
shuffle=True,
stratify=y_train,
random_state=RANDOM_STATE,
)
model = get_architecture(y, mobilenet_size="small", as_feature_extractor=True)
history_v3Small_fe = model.fit(
X_train,
y_train,
batch_size=BATCH_SIZE,
epochs=NUM_EPOCHS,
validation_data=(X_val, y_val),
callbacks=get_callbacks(WEIGHTS_FILE, NUM_EPOCHS // 2, 0.25),
)
model.load_weights(WEIGHTS_FILE)
final_accuracy = model.evaluate(X_test, y_test, batch_size=1, verbose=0)[1]
print("*" * 72)
print(f"Final MobileNetV3-Small (Feature Extraction) Accuracy: {final_accuracy}")
print("*" * 72)
Epoch 1/20 80/80 [==============================] - 5s 39ms/step - loss: 1.2944 - accuracy: 0.4406 - val_loss: 1.0144 - val_accuracy: 0.6125 Epoch 2/20 80/80 [==============================] - 3s 32ms/step - loss: 0.9928 - accuracy: 0.6000 - val_loss: 0.9002 - val_accuracy: 0.6000 Epoch 3/20 80/80 [==============================] - 3s 32ms/step - loss: 0.7730 - accuracy: 0.7219 - val_loss: 0.7219 - val_accuracy: 0.8000 Epoch 4/20 80/80 [==============================] - 3s 31ms/step - loss: 0.6658 - accuracy: 0.7812 - val_loss: 0.6950 - val_accuracy: 0.7250 Epoch 5/20 80/80 [==============================] - 3s 31ms/step - loss: 0.6158 - accuracy: 0.7844 - val_loss: 0.6419 - val_accuracy: 0.7750 Epoch 6/20 80/80 [==============================] - 3s 31ms/step - loss: 0.5311 - accuracy: 0.8562 - val_loss: 0.5540 - val_accuracy: 0.8125 Epoch 7/20 80/80 [==============================] - 3s 32ms/step - loss: 0.4473 - accuracy: 0.8625 - val_loss: 0.4973 - val_accuracy: 0.8750 Epoch 8/20 80/80 [==============================] - 3s 32ms/step - loss: 0.4238 - accuracy: 0.8594 - val_loss: 0.6072 - val_accuracy: 0.7000 Epoch 9/20 80/80 [==============================] - 2s 31ms/step - loss: 0.4175 - accuracy: 0.8687 - val_loss: 0.4388 - val_accuracy: 0.8375 Epoch 10/20 80/80 [==============================] - 2s 31ms/step - loss: 0.3917 - accuracy: 0.8813 - val_loss: 0.3750 - val_accuracy: 0.8750 Epoch 11/20 80/80 [==============================] - 2s 31ms/step - loss: 0.3212 - accuracy: 0.9187 - val_loss: 0.5161 - val_accuracy: 0.7375 Epoch 12/20 80/80 [==============================] - 2s 31ms/step - loss: 0.3220 - accuracy: 0.9094 - val_loss: 0.3032 - val_accuracy: 0.9125 Epoch 13/20 80/80 [==============================] - 2s 31ms/step - loss: 0.2983 - accuracy: 0.9219 - val_loss: 0.3504 - val_accuracy: 0.8750 Epoch 14/20 80/80 [==============================] - 3s 31ms/step - loss: 0.3018 - accuracy: 0.9125 - val_loss: 0.3106 - val_accuracy: 0.8875 Epoch 15/20 80/80 [==============================] - 2s 31ms/step - loss: 0.2572 - accuracy: 0.9406 - val_loss: 0.3497 - val_accuracy: 0.8750 Epoch 16/20 80/80 [==============================] - 2s 31ms/step - loss: 0.2100 - accuracy: 0.9563 - val_loss: 0.3085 - val_accuracy: 0.8875 Epoch 17/20 80/80 [==============================] - 3s 31ms/step - loss: 0.2311 - accuracy: 0.9406 - val_loss: 0.3463 - val_accuracy: 0.8750 Epoch 18/20 80/80 [==============================] - 2s 31ms/step - loss: 0.1992 - accuracy: 0.9438 - val_loss: 0.2991 - val_accuracy: 0.9000 Epoch 19/20 80/80 [==============================] - 3s 31ms/step - loss: 0.2171 - accuracy: 0.9500 - val_loss: 0.4130 - val_accuracy: 0.8000 Epoch 20/20 80/80 [==============================] - 2s 31ms/step - loss: 0.2301 - accuracy: 0.9531 - val_loss: 0.1984 - val_accuracy: 0.9375 ************************************************************************ Final MobileNetV3-Small (Feature Extraction) Accuracy: 0.949999988079071 ************************************************************************
model = get_architecture(y, mobilenet_size="small", as_feature_extractor=False)
history_v3Small_ft = model.fit(
X_train,
y_train,
batch_size=BATCH_SIZE,
epochs=NUM_EPOCHS,
validation_data=(X_val, y_val),
callbacks=get_callbacks(WEIGHTS_FILE, NUM_EPOCHS // 2, 0.25),
)
model.load_weights(WEIGHTS_FILE)
final_accuracy = model.evaluate(X_test, y_test, batch_size=1, verbose=0)[1]
print("*" * 72)
print(f"Final MobileNetV3-Small (Fine-Tuning) Accuracy: {final_accuracy}")
print("*" * 72)
Epoch 1/20 80/80 [==============================] - 11s 104ms/step - loss: 1.3443 - accuracy: 0.4062 - val_loss: 1.2339 - val_accuracy: 0.4250 Epoch 2/20 80/80 [==============================] - 8s 97ms/step - loss: 0.9910 - accuracy: 0.6000 - val_loss: 1.2494 - val_accuracy: 0.4125 Epoch 3/20 80/80 [==============================] - 8s 97ms/step - loss: 0.7807 - accuracy: 0.6938 - val_loss: 1.3763 - val_accuracy: 0.4125 Epoch 4/20 80/80 [==============================] - 8s 98ms/step - loss: 0.7820 - accuracy: 0.6812 - val_loss: 0.8089 - val_accuracy: 0.6625 Epoch 5/20 80/80 [==============================] - 8s 98ms/step - loss: 0.5072 - accuracy: 0.8062 - val_loss: 0.9012 - val_accuracy: 0.5750 Epoch 6/20 80/80 [==============================] - 8s 97ms/step - loss: 0.5059 - accuracy: 0.8188 - val_loss: 0.4925 - val_accuracy: 0.8625 Epoch 7/20 80/80 [==============================] - 8s 97ms/step - loss: 0.4081 - accuracy: 0.8625 - val_loss: 0.3601 - val_accuracy: 0.8875 Epoch 8/20 80/80 [==============================] - 8s 97ms/step - loss: 0.3303 - accuracy: 0.8906 - val_loss: 0.2963 - val_accuracy: 0.9250 Epoch 9/20 80/80 [==============================] - 8s 97ms/step - loss: 0.2718 - accuracy: 0.9000 - val_loss: 0.4144 - val_accuracy: 0.7875 Epoch 10/20 80/80 [==============================] - 8s 97ms/step - loss: 0.3433 - accuracy: 0.8844 - val_loss: 0.1898 - val_accuracy: 0.9375 Epoch 11/20 80/80 [==============================] - 8s 97ms/step - loss: 0.2812 - accuracy: 0.8969 - val_loss: 0.2038 - val_accuracy: 0.9375 Epoch 12/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1978 - accuracy: 0.9406 - val_loss: 0.2660 - val_accuracy: 0.8875 Epoch 13/20 80/80 [==============================] - 8s 97ms/step - loss: 0.2543 - accuracy: 0.9219 - val_loss: 0.1098 - val_accuracy: 0.9875 Epoch 14/20 80/80 [==============================] - 8s 98ms/step - loss: 0.2182 - accuracy: 0.9281 - val_loss: 0.1012 - val_accuracy: 0.9750 Epoch 15/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1868 - accuracy: 0.9438 - val_loss: 0.0717 - val_accuracy: 0.9875 Epoch 16/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1485 - accuracy: 0.9594 - val_loss: 0.0761 - val_accuracy: 0.9750 Epoch 17/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1580 - accuracy: 0.9500 - val_loss: 0.0624 - val_accuracy: 0.9875 Epoch 18/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1492 - accuracy: 0.9500 - val_loss: 0.0523 - val_accuracy: 0.9875 Epoch 19/20 80/80 [==============================] - 8s 98ms/step - loss: 0.1098 - accuracy: 0.9719 - val_loss: 0.0602 - val_accuracy: 0.9875 Epoch 20/20 80/80 [==============================] - 8s 97ms/step - loss: 0.1126 - accuracy: 0.9625 - val_loss: 0.0407 - val_accuracy: 0.9875 ************************************************************************ Final MobileNetV3-Small (Fine-Tuning) Accuracy: 0.9599999785423279 ************************************************************************
model = get_architecture(y, mobilenet_size="large", as_feature_extractor=True)
history_v3Large_fe = model.fit(
X_train,
y_train,
batch_size=BATCH_SIZE,
epochs=NUM_EPOCHS,
validation_data=(X_val, y_val),
callbacks=get_callbacks(WEIGHTS_FILE, NUM_EPOCHS // 2, 0.25),
)
model.load_weights(WEIGHTS_FILE)
final_accuracy = model.evaluate(X_test, y_test, batch_size=1, verbose=0)[1]
print("*" * 73)
print(f"Final MobileNetV3-Large (Feature Extraction) Accuracy: {final_accuracy}")
print("*" * 73)
Epoch 1/20 80/80 [==============================] - 8s 67ms/step - loss: 1.2652 - accuracy: 0.4594 - val_loss: 0.9498 - val_accuracy: 0.6375 Epoch 2/20 80/80 [==============================] - 5s 59ms/step - loss: 0.9566 - accuracy: 0.6344 - val_loss: 0.7578 - val_accuracy: 0.7250 Epoch 3/20 80/80 [==============================] - 5s 59ms/step - loss: 0.7646 - accuracy: 0.7156 - val_loss: 0.7186 - val_accuracy: 0.7750 Epoch 4/20 80/80 [==============================] - 5s 59ms/step - loss: 0.6303 - accuracy: 0.7656 - val_loss: 0.7255 - val_accuracy: 0.7625 Epoch 5/20 80/80 [==============================] - 5s 59ms/step - loss: 0.5429 - accuracy: 0.8031 - val_loss: 0.6215 - val_accuracy: 0.8500 Epoch 6/20 80/80 [==============================] - 5s 59ms/step - loss: 0.4601 - accuracy: 0.8781 - val_loss: 0.5467 - val_accuracy: 0.8500 Epoch 7/20 80/80 [==============================] - 5s 59ms/step - loss: 0.4686 - accuracy: 0.8562 - val_loss: 0.4964 - val_accuracy: 0.8500 Epoch 8/20 80/80 [==============================] - 5s 59ms/step - loss: 0.3978 - accuracy: 0.8781 - val_loss: 0.4314 - val_accuracy: 0.8875 Epoch 9/20 80/80 [==============================] - 5s 59ms/step - loss: 0.3800 - accuracy: 0.8750 - val_loss: 0.4390 - val_accuracy: 0.8500 Epoch 10/20 80/80 [==============================] - 5s 59ms/step - loss: 0.3635 - accuracy: 0.8750 - val_loss: 0.4724 - val_accuracy: 0.8750 Epoch 11/20 80/80 [==============================] - 5s 58ms/step - loss: 0.3213 - accuracy: 0.9156 - val_loss: 0.3823 - val_accuracy: 0.9000 Epoch 12/20 80/80 [==============================] - 5s 59ms/step - loss: 0.3167 - accuracy: 0.9187 - val_loss: 0.4065 - val_accuracy: 0.8875 Epoch 13/20 80/80 [==============================] - 5s 60ms/step - loss: 0.2960 - accuracy: 0.9000 - val_loss: 0.3422 - val_accuracy: 0.8750 Epoch 14/20 80/80 [==============================] - 5s 58ms/step - loss: 0.2798 - accuracy: 0.9125 - val_loss: 0.4072 - val_accuracy: 0.8375 Epoch 15/20 80/80 [==============================] - 5s 58ms/step - loss: 0.2836 - accuracy: 0.9187 - val_loss: 0.3523 - val_accuracy: 0.9000 Epoch 16/20 80/80 [==============================] - 5s 58ms/step - loss: 0.2538 - accuracy: 0.9281 - val_loss: 0.2765 - val_accuracy: 0.9250 Epoch 17/20 80/80 [==============================] - 5s 59ms/step - loss: 0.2472 - accuracy: 0.9125 - val_loss: 0.3789 - val_accuracy: 0.8875 Epoch 18/20 80/80 [==============================] - 5s 59ms/step - loss: 0.2423 - accuracy: 0.9312 - val_loss: 0.2922 - val_accuracy: 0.9250 Epoch 19/20 80/80 [==============================] - 5s 58ms/step - loss: 0.2328 - accuracy: 0.9219 - val_loss: 0.2855 - val_accuracy: 0.8875 Epoch 20/20 80/80 [==============================] - 5s 59ms/step - loss: 0.2287 - accuracy: 0.9250 - val_loss: 0.2618 - val_accuracy: 0.9125 ************************************************************************* Final MobileNetV3-Large (Feature Extraction) Accuracy: 0.9100000262260437 *************************************************************************
model = get_architecture(y, mobilenet_size="large", as_feature_extractor=False)
history_v3Large_ft = model.fit(
X_train,
y_train,
batch_size=BATCH_SIZE,
epochs=NUM_EPOCHS,
validation_data=(X_val, y_val),
callbacks=get_callbacks(WEIGHTS_FILE, NUM_EPOCHS // 2, 0.25),
)
model.load_weights(WEIGHTS_FILE)
final_accuracy = model.evaluate(X_test, y_test, batch_size=1, verbose=0)[1]
print("*" * 73)
print(f"Final MobileNetV3-Large (Fine Tuning) Accuracy: {final_accuracy}")
print("*" * 73)
Epoch 1/20 80/80 [==============================] - 23s 248ms/step - loss: 1.1406 - accuracy: 0.5156 - val_loss: 1.1863 - val_accuracy: 0.4125 Epoch 2/20 80/80 [==============================] - 19s 239ms/step - loss: 0.7857 - accuracy: 0.7094 - val_loss: 1.0722 - val_accuracy: 0.4500 Epoch 3/20 80/80 [==============================] - 19s 239ms/step - loss: 0.5427 - accuracy: 0.8062 - val_loss: 0.5968 - val_accuracy: 0.7375 Epoch 4/20 80/80 [==============================] - 19s 239ms/step - loss: 0.3869 - accuracy: 0.8594 - val_loss: 0.3860 - val_accuracy: 0.8875 Epoch 5/20 80/80 [==============================] - 19s 239ms/step - loss: 0.3457 - accuracy: 0.8844 - val_loss: 0.3505 - val_accuracy: 0.8750 Epoch 6/20 80/80 [==============================] - 19s 239ms/step - loss: 0.2111 - accuracy: 0.9219 - val_loss: 0.3448 - val_accuracy: 0.8375 Epoch 7/20 80/80 [==============================] - 19s 239ms/step - loss: 0.1727 - accuracy: 0.9344 - val_loss: 0.2243 - val_accuracy: 0.9375 Epoch 8/20 80/80 [==============================] - 19s 239ms/step - loss: 0.2318 - accuracy: 0.9094 - val_loss: 0.2877 - val_accuracy: 0.9250 Epoch 9/20 80/80 [==============================] - 19s 235ms/step - loss: 0.1370 - accuracy: 0.9531 - val_loss: 0.1760 - val_accuracy: 0.9625 Epoch 10/20 80/80 [==============================] - 19s 235ms/step - loss: 0.1316 - accuracy: 0.9469 - val_loss: 0.1509 - val_accuracy: 0.9750 Epoch 11/20 80/80 [==============================] - 19s 234ms/step - loss: 0.0672 - accuracy: 0.9781 - val_loss: 0.1325 - val_accuracy: 0.9875 Epoch 12/20 80/80 [==============================] - 19s 235ms/step - loss: 0.1526 - accuracy: 0.9563 - val_loss: 0.3163 - val_accuracy: 0.9125 Epoch 13/20 80/80 [==============================] - 19s 234ms/step - loss: 0.0928 - accuracy: 0.9719 - val_loss: 0.1223 - val_accuracy: 0.9875 Epoch 14/20 80/80 [==============================] - 19s 235ms/step - loss: 0.0747 - accuracy: 0.9844 - val_loss: 0.1863 - val_accuracy: 0.9625 Epoch 15/20 80/80 [==============================] - 19s 235ms/step - loss: 0.1184 - accuracy: 0.9656 - val_loss: 0.2501 - val_accuracy: 0.9500 Epoch 16/20 80/80 [==============================] - 19s 234ms/step - loss: 0.1149 - accuracy: 0.9563 - val_loss: 0.1851 - val_accuracy: 0.9625 Epoch 17/20 80/80 [==============================] - 19s 234ms/step - loss: 0.0382 - accuracy: 0.9906 - val_loss: 0.3659 - val_accuracy: 0.9125 Epoch 18/20 80/80 [==============================] - 19s 234ms/step - loss: 0.0314 - accuracy: 0.9906 - val_loss: 0.2301 - val_accuracy: 0.9250 Epoch 19/20 80/80 [==============================] - 19s 235ms/step - loss: 0.0495 - accuracy: 0.9844 - val_loss: 0.3805 - val_accuracy: 0.9250 Epoch 20/20 80/80 [==============================] - 19s 235ms/step - loss: 0.0195 - accuracy: 0.9969 - val_loss: 0.1808 - val_accuracy: 0.9625 ************************************************************************* Final MobileNetV3-Large (Fine Tuning) Accuracy: 0.9700000286102295 *************************************************************************
plot_performances([
("MobileNetV3-Small-Fine Tuning", history_v3Small_ft.history["val_accuracy"]),
("MobileNetV3-Small-Feature Extractor", history_v3Small_fe.history["val_accuracy"]),
("MobileNetV3-Large-Fine Tuning", history_v3Large_ft.history["val_accuracy"]),
("MobileNetV3-Large-Feature Extractor", history_v3Large_fe.history["val_accuracy"])
])
Bengio, Y., 2012. Deep Learning of Representations for Unsupervised and Transfer Learning. In: Journal of Machine Learning Research; 17–37.
Wang, G., Sun, Y., Wang, J., (2017). Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning. Computational Intelligence and Neuroscience; 2017:8.
Mehdipour-Ghazi, M., Yanikoglu, B.A., & Aptoula, E. (2017). Plant identification using deep neural networks via optimization of transfer learning parameters. Neurocomputing, 235, 228-235.
Suh, H.K., IJsselmuiden, J., Hofstee, J.W., van Henten, E.J., (2018). Transfer learning for the classification of sugar beet and volunteer potato under field conditions. Biosystems Engineering; 174:50–65.
Kounalakis T., Triantafyllidis G. A., Nalpantidis L., (2019). Deep learning-based visual recognition of rumex for robotic precision farming. Computers and Electronics in Agriculture.
Too, E.C., Yujian, L., Njuki, S., & Ying-chun, L. (2019). A comparative study of fine-tuning deep learning models for plant disease identification. Comput. Electron. Agric., 161, 272-279.
Espejo-Garcia, B., Mylonas, N., Athanasakos, L., & Fountas, S., (2020). Improving Weeds Identification with a Repository of Agricultural Pre-trained Deep Neural Networks. Computers and Electronics in Agriculture; 175 (August).
Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., & Chen, L. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4510-4520.
Howard, A.G., Sandler, M., Chu, G., Chen, L., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q.V., & Adam, H. (2019). Searching for MobileNetV3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 1314-1324.
https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html
https://www.oreilly.com/library/view/programming-pytorch-for/9781492045342/ch04.html