Welcome to lesson 1! For those of you who are using a Jupyter Notebook for the first time, you can learn about this useful tool in a tutorial we prepared specially for you; click
Open now and click
In this lesson we will build our first image classifier from scratch, and see if we can achieve world-class results. Let's dive in!
Every notebook starts with the following three lines; they ensure that any edits to libraries you make are reloaded here automatically, and also that any charts or images displayed are shown in this notebook.
%reload_ext autoreload %autoreload 2 %matplotlib inline
from fastai import * from fastai.vision import *
We are going to use the Oxford-IIIT Pet Dataset by O. M. Parkhi et al., 2012 which features 12 cat breeds and 25 dogs breeds. Our model will need to learn to differentiate between these 37 distinct categories. According to their paper, the best accuracy they could get in 2012 was 59.21%, using a complex model that was specific to pet detection, with separate "Image", "Head", and "Body" models for the pet photos. Let's see how accurate we can be using deep learning!
We are going to use the
untar_data function to which we must pass a URL as an argument and which will download and extract the data.
# DEBUG URLs.PETS
path = untar_data(URLs.PETS) path
path_anno = path / 'annotations' path_img = path / 'images'
The first thing we do when we approach a problem is to take a look at the data. We always need to understand very well what the problem is and what the data looks like before we can figure out how to solve it. Taking a look at the data means understanding how the data directories are structured, what the labels are and what some sample images look like.
The main difference between the handling of image classification datasets is the way labels are stored. In this particular dataset, labels are stored in the filenames themselves. We will need to extract them to be able to classify the images into the correct categories. Fortunately, the fastai library has a handy function made exactly for this,
ImageDataBunch.from_name_re gets the labels from the filenames using a regular expression.
# DEBUG path_anno, path_img
fnames = get_image_files(path_img) fnames[:5]
np.random.seed(2) pat = r'/([^/]+)_\d+.jpg$'
data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224) data.normalize(imagenet_stats)
print(data.classes) len(data.classes), data.c # the number of classes
Now we will start training our model. We will use a convolutional neural network backbone and a fully connected head with a single hidden layer as a classifier. Don't know what these things mean? Not to worry, we will dive deeper in the coming lessons. For the moment you need to know that we are building a model which will take images as input and will output the predicted probability for each of the categories (in this case, it will have 37 ouptuts).
We will train for 5 epochs (5 cycles through all our data).
learn = ConvLearner(data, models.resnet34, metrics=error_rate)
Let's see what results we have got.
We will first see which were the categories that the model most confused with one another. We will try to see if what the model predicted was reasonable or not. In this case the mistakes look reasonable (none of the mistakes seems obviously naive). This is an indicator that our classifier is working correctly.
Furthermore, when we plot the confusion matrix, we can see that the distribution is heavily skewed: the model makes the same mistakes over and over again but it rarely confuses other categories. This suggests that it just finds it difficult to distinguish some specific categories between each other; this is normal behaviour.
interp = ClassificationInterpretation.from_learner(learn)
Since our model is working as we expect it to, we will unfreeze our model and train some more.
Fix the previous error by updating fastai library to the latest version today
learn.unfreeze() learn.fit_one_cycle(2, max_lr=slice(1e-6, 1e-4))
That's a pretty accurate model!
Now we will train in the same way as before but with one caveat: instead of using resnet34 as our backbone we will use resnet50 (resnet34 is a 34 layer residual network while resnet50 has 50 layers. Later in the course you can learn the details in the resnet paper).
Basically, resnet50 usually performs better because it is a deeper network with more parameters. Let's see if we can achieve a higher performance here.
data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=299, bs=48) data.normalize(imagenet_stats)
learn = ConvLearner(data, models.resnet50, metrics=error_rate)
Total time: 17:07 epoch train loss valid loss error_rate 1 0.641380 0.236811 0.068556 (03:56) 2 0.341766 0.211440 0.069875 (03:18) 3 0.242888 0.207498 0.069875 (03:17) 4 0.159888 0.167932 0.054054 (03:17) 5 0.107927 0.163711 0.050758 (03:17)
It's astonishing that it's possible to recognize pet breeds so accurately! Let's see if full fine-tuning helps:
learn.unfreeze() learn.fit_one_cycle(1, max_lr=slice(1e-6,1e-4))
Total time: 04:24 epoch train loss valid loss error_rate 1 0.103458 0.155929 0.051417 (04:24)
In this case it doesn't, so let's go back to our previous model.
interp = ClassificationInterpretation.from_learner(learn)
[('Egyptian_Mau', 'Bengal', 7), ('american_bulldog', 'staffordshire_bull_terrier', 4), ('staffordshire_bull_terrier', 'american_bulldog', 4), ('boxer', 'american_bulldog', 4), ('Russian_Blue', 'British_Shorthair', 4), ('american_pit_bull_terrier', 'staffordshire_bull_terrier', 4), ('Ragdoll', 'Birman', 3)]
path = untar_data(URLs.MNIST_SAMPLE) path
tfms = get_transforms(do_flip=False) data = ImageDataBunch.from_folder(path, ds_tfms=tfms, size=26)
learn = ConvLearner(data, models.resnet18, metrics=accuracy) learn.fit(2)
Total time: 00:23 epoch train loss valid loss accuracy 1 0.114837 0.023115 0.993621 (00:12) 2 0.067166 0.014504 0.995584 (00:10)
df = pd.read_csv(path / 'labels.csv') df.head()
data = ImageDataBunch.from_csv(path, ds_tfms=tfms, size=28)
data.show_batch(rows=3, figsize=(5,5)) data.classes
data = ImageDataBunch.from_df(path, df, ds_tfms=tfms, size=24) data.classes
fn_paths = [path / name for name in df['name']]; fn_paths[:2]
pat = r"/(\d)/\d+\.png$" data = ImageDataBunch.from_name_re(path, fn_paths, pat=pat, ds_tfms=tfms, size=24) data.classes
data = ImageDataBunch.from_name_func(path, fn_paths, ds_tfms=tfms, size=24, label_func = lambda x: '3' if '/3/' in str(x) else '7') data.classes
labels = [('3' if '/3/' in str(x) else '7') for x in fn_paths] labels[:5]
['3', '3', '3', '3', '3']
data = ImageDataBunch.from_lists(path, fn_paths, labels=labels, ds_tfms=tfms, size=24) data.classes