How to Use Transfer Learning with Pre-Trained Models in Computer Vision Lessons

The Microsoft AI-For-Beginners Computer Vision curriculum teaches transfer learning by freezing a VGG-16 backbone pre-trained on ImageNet, attaching a lightweight binary classifier, and training only the new head on the Cats vs Dogs dataset to achieve over 95% validation accuracy with minimal data and computation.

The Computer Vision lessons in the microsoft/AI-For-Beginners repository provide a hands-on implementation of transfer learning with pre-trained models using TensorFlow and Keras. This approach allows you to repurpose powerful convolutional networks trained on millions of ImageNet images for custom classification tasks without requiring massive datasets or GPU hours.

Loading the Cats vs Dogs Dataset

The lesson utilizes a helper utility to prepare the binary classification data. In lessons/4-ComputerVision/08-TransferLearning/tfcv.py, the function load_cats_dogs_dataset downloads the Kaggle dataset, automatically removes corrupt JPEG files, and returns TensorFlow Dataset objects for training and validation.

from tfcv import load_cats_dogs_dataset

ds_train, ds_test = load_cats_dogs_dataset(batch_size=64)

This preprocessing step ensures clean data fed directly into the pipeline, handling the tedious data wrangling so you can focus on model architecture.

Loading a Pre-Trained VGG-16 Backbone

The notebook lessons/4-ComputerVision/08-TransferLearning/TransferLearningTF.ipynb demonstrates loading VGG-16 from keras.applications. By setting include_top=False, you remove the final ImageNet classification layers, retaining only the convolutional base that outputs rich feature maps.

from tensorflow import keras

vgg_backbone = keras.applications.VGG16(
    include_top=False,               
    weights='imagenet',              
    input_shape=(224, 224, 3)        
)

Every pre-trained network requires specific input scaling. For VGG-16, you must apply keras.applications.vgg16.preprocess_input to normalize pixel values and subtract the ImageNet mean before feeding images into the network.

Building the Transfer Learning Model

To prevent overwriting the learned ImageNet features, you must freeze the pre-trained weights before adding your classifier. The lesson demonstrates creating a Sequential model that combines the frozen backbone with a new trainable head consisting of a Flatten layer and a single dense unit with sigmoid activation for binary classification.

vgg_backbone.trainable = False

model = keras.models.Sequential([
    vgg_backbone,
    keras.layers.Flatten(),
    keras.layers.Dense(1, activation='sigmoid')
])

model.summary()  # Shows ~14M frozen params, ~25K trainable params

This architecture dramatically reduces the number of trainable parameters from millions to thousands, enabling fast training on limited data while maintaining high representational capacity.

Training the Classifier Head

With the backbone frozen, you compile the model for binary classification and train only the new dense layer. Because the convolutional base already extracts meaningful features, the head converges quickly.

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc'])

history = model.fit(ds_train, 
                    validation_data=ds_test,
                    epochs=5)

According to the source code in TransferLearningTF.ipynb, this configuration typically reaches greater than 95% validation accuracy on the Cats vs Dogs split within just a few epochs, demonstrating the efficiency of transfer learning with pre-trained models.

Fine-Tuning the Convolutional Layers

After the classifier head stabilizes, you can optionally fine-tune the deeper convolutional layers to adapt the generic ImageNet features to your specific domain. The notebook outlines unfreezing the last four layers of the VGG base and continuing training with a substantially lower learning rate.

for layer in vgg_backbone.layers[-4:]:
    layer.trainable = True

model.compile(optimizer=keras.optimizers.Adam(1e-5),  # Lower learning rate

              loss='binary_crossentropy',
              metrics=['acc'])

fine_tune_history = model.fit(ds_train,
                              validation_data=ds_test,
                              epochs=3)

This two-stage training strategy—first training the head, then fine-tuning the backbone—balances training speed with final model accuracy while preventing catastrophic forgetting of the pre-trained features.

Summary

• The load_cats_dogs_dataset helper in tfcv.py handles dataset download and cleaning automatically.
• VGG-16 serves as the pre-trained backbone, loaded via keras.applications.VGG16 with include_top=False to remove ImageNet-specific layers.
• Freezing the backbone with trainable = False preserves learned features and reduces trainable parameters to approximately 25,000.
• A lightweight classifier head (Flatten + Dense) trains rapidly on the extracted features, achieving >95% accuracy.
• Fine-tuning the last four convolutional layers with a low learning rate (1e-5) adapts the model to domain-specific patterns without overfitting.

Frequently Asked Questions

What dataset is used in the transfer learning lesson?

The lesson uses the Cats vs Dogs dataset from Kaggle, accessed through the load_cats_dogs_dataset function in lessons/4-ComputerVision/08-TransferLearning/tfcv.py. This helper automatically downloads the images, filters out corrupt JPEG files, and creates TensorFlow Dataset objects ready for training and validation.

Why is include_top=False used when loading VGG-16?

Setting include_top=False removes the final classification layers that were trained to recognize 1,000 ImageNet categories. This allows you to use the convolutional base as a feature extractor that outputs a 7×7×512 feature map, which you can then pair with a custom classifier suited for your specific number of classes—in this case, a single binary output for cats versus dogs.

How many parameters are trainable versus frozen in the initial setup?

In the initial transfer learning configuration, approximately 14 million parameters from the VGG-16 backbone remain frozen, while only the new classifier head contributes roughly 25,000 trainable parameters. This massive reduction enables training on smaller datasets without overfitting while leveraging powerful pre-learned visual representations.

When should I fine-tune the pre-trained backbone?

You should fine-tune only after the classifier head has converged and stabilized. The notebook recommends unfreezing the last four convolutional layers and training with a reduced learning rate (around 1e-5) for a few additional epochs. This optional step adapts the high-level features to your specific dataset while keeping training data requirements modest.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}