Classifying Pet Breeds
We'll try to figure out what breed of pet is shown in each image of a dataset.
- Training a simple model
- Picking the Learning Rate
- Unfreezing and Transfer Learning
- Using Deeper Architectures
- Exporting our model
In this blog post, we'll try to figure out what breed of pet is shown in each image of a dataset.
The dataset we'll be using is the Oxford-IIIT Pet dataset. It's a 37 category pet dataset with roughly 200 images for each class. The images have a large variations in scale, pose and lighting. The dataset can be downloaded here.
First, let's install fastai and import all its modules.
!pip install -Uqq fastbook
import fastbook
fastbook.setup_book()
from fastbook import *
from fastai.vision.all import *
We'll use untar_data to download files from a URL. We'll set a path to this dataset.
path = untar_data(URLs.PETS)
The below code helps us by showing only the relevant paths, and not showing the parent path folders.
Path.BASE_PATH = path
Let's see what's in the directory.
path.ls()
The dataset provides us with the images and annotations directories. The website for the dataset tells us that the annotations directory contains information about where the pets are rather than what they are. Hence, we will ignore the annotations directory.
Let's look at the images directory.
(path/'images').ls()
There are 7,393 images in the folder. The breed of the pet is embedded in the image name along with a number and the extension. So, we can use a regular expression to extract the valuable information.
Let's try out an example first.
We'll pick one the filenames.
fname = (path/'images').ls()[0]
re.findall(r'(.+)_\d+.jpg', fname.name)
This regular expression plucks out all the characters leading up to the last underscore character, as long as the subsequence chracters are numerical digits and then the JPEG file extension.
Now that we have confirmed that the regular expression works for the example, let's use it to label the whole dataset. For labeling with regular expressions, we'll use the RegexLabeller class.
pets = DataBlock(blocks = (ImageBlock, CategoryBlock),
get_items=get_image_files,
splitter=RandomSplitter(seed=42),
get_y=using_attr(RegexLabeller(r'(.+)_\d+.jpg'), 'name'),
item_tfms=Resize(460),
batch_tfms=aug_transforms(size=224, min_scale=0.75))
dls = pets.dataloaders(path/'images')
Let's find out what breeds are present in the dataset.
dls.vocab
dls.show_batch(max_n=9, figsize=(6,7))
learn = cnn_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(15)
Let's look at a graph of the training and validation loss.
learn.recorder.plot_loss()
Even our simple model gives us really good accuracy. Let's use a classification matrix to interpret our model; see where it's doing well and where it's doing badly.
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix(figsize=(12,12), dpi=60)
We have 37 different breeds of pet, which means we have 37 X 37 entries in this matrix, which makes it hard to interpret. We'll use the most_confused method, which just shows us the cells of the confusion matrix with the most incorrect predictions.
interp.most_confused(min_val=5)
These are the breeds of cats and dogs that are hardest to predict by our model. Coincidently, these are also pretty common comparisons that even experts get confused about, for e.g. the differences in appearance between the Staffordshire Bull Terrier and the American Pit Bull Terrier (i.e. the comparison that confused our model the most) are so minute that most humans will find it hard to distinguish them.
interp.plot_top_losses(8, nrows=2)
learn.show_results()
Let's save this model.
learn.save('model1')
learn2 = cnn_learner(dls, resnet34, metrics=error_rate)
lr_min, lr_steep = learn2.lr_find()
print(f'Minimum/10: {lr_min: .2e}, steepest_point: {lr_steep: .2e}')
Let's now pick the learning rate at the steepest point in the graph.
learn2 = cnn_learner(dls, resnet34, metrics=error_rate)
learn2.fine_tune(15, 5.25e-3)
Let's plot the losses on a graph.
learn2.recorder.plot_loss()
learn2.save('model2')
Unfreezing and Transfer Learning
When fine-tuning, we aim to replace the random weights in our added linear layers with weights that correctly achieve our desired task of classifying pet breeds without breaking the carefully pretrained weights and the other layers. We'll tell the optimizer to only update the weights in those randomly added final layers, and not change the weights in the rest of the neural network. This is called freezing the pretrained layers.
When we create a model from a pretrained network, fastai automatically freexzes all of the pretrained layers for us.
We can try and improve our model by changing the parameters of the fine_tune method.
First, we'll train the randomly added layers for three epochs, using fit_one_cycle, which is the suggested way to train models without using fine_tune.
learn3 = cnn_learner(dls, resnet34, metrics=error_rate)
learn3.fit_one_cycle(4, 5.25e-3)
Now, we'll unfreeze the model.
learn3.unfreeze()
We'll run lr_find again because having more layers to train, and weights that have already been trained for three epochs, means our previously found learning rate isn't appropriate anymore.
learn3.lr_find()
We can see that we have a somewhat flat area before a sharp increase, and we should take a point well before that sharp increase. But the deepest layers of our pretrained model might not need as high a learning rate as the last ones, so we should use different learning rates for different layers, which is known as discriminative learning rates.
We'll use a lower learning rate for the early layers of the neural network, and a higher learning rate for the later layers (and especially the randomly added layers).
fastai lets you pass a Python slice object anywhere that a learning rate is expected. The first value passed will be the learning rate in the earliest layer of the neural network, and the second value will be the learning rate in the final layer. The layers in between will have learning rates that are multiplicatively equidistant throughout that range.
learn3 = cnn_learner(dls, resnet34, metrics=error_rate)
learn3.fit_one_cycle(4, 5.25e-3)
learn3.unfreeze()
learn3.fit_one_cycle(12, lr_max=slice(1e-6,1e-4))
learn3.recorder.plot_loss()
learn3.save('model3')
Using Deeper Architectures
Let's try to improve our model's performance using deeper architectures.
Deeper architectures like ResNet50 take longer to train. We can speed things up by using mixed-precision training, wherein we can use less-precise numbers like fp16 (half-precision floating point). To enable this feature, we just add to_fp16() to the Learner.
from fastai.callback.fp16 import *
learn4 = cnn_learner(dls, resnet50, metrics=error_rate).to_fp16()
learn4.fine_tune(15)
learn4.recorder.plot_loss()
learn4.save('model4')
learn.export()
path = Path()
path.ls(file_exts='.pkl')
learn_inf = load_learner('export.pkl')
learn_inf.dls.vocab
from fastai.vision.widgets import *
def on_click_classify(change):
img = PILImage.create(btn_upload.data[-1])
out_pl.clear_output()
with out_pl: display(img.to_thumb(128,128))
pred,pred_idx,probs = learn_inf.predict(img)
lbl_pred.value = f'Prediction: {pred}; Probability: {probs[pred_idx]:.04f}'
btn_run.on_click(on_click_classify)
VBox([widgets.Label('Select your pet!'),
btn_upload, btn_run, out_pl, lbl_pred])