# How to Perform Object Detection with TensorFlow in the AI-For-Beginners Curriculum

> Learn object detection with TensorFlow in the AI-For-Beginners curriculum. Master sliding-window classification, bounding-box regression, and RetinaNet implementations.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: how-to-guide
- Published: 2026-08-06

---

**The Microsoft AI-For-Beginners curriculum teaches TensorFlow-based object detection through a three-stage progression: naïve sliding-window classification with VGG-16, synthetic bounding-box regression, and references to production RetinaNet implementations.**

The `microsoft/AI-For-Beginners` repository provides a structured introduction to computer vision using TensorFlow and Keras. In `lessons/4-ComputerVision/11-ObjectDetection/ObjectDetection.ipynb`, learners progress from basic classification techniques to building regression models that predict bounding-box coordinates. This hands-on approach demystifies how modern object detectors work by deconstructing the problem into manageable components.

## Stage 1: Sliding-Window Detection with VGG-16

The curriculum begins with a **naïve sliding-window approach** that repurposes a pre-trained classifier for localization. Instead of training a new model, learners use `keras.applications.vgg16.VGG16` (loaded with ImageNet weights) to evaluate overlapping tiles of an input image.

In the `predict` function implementation, the model preprocesses a 224×224 tile and returns the summed probability across ImageNet class indices 281–294 (the "cat" category). The `predict_map` function systematically slides this window across the image to build a heat-map of object presence.

```python
from tensorflow import keras
import cv2, numpy as np, matplotlib.pyplot as plt

vgg = keras.applications.vgg16.VGG16(weights="imagenet")

def predict(img):
    im = cv2.resize(img, (224, 224))
    im = keras.applications.vgg16.preprocess_input(im)
    pr = vgg.predict(np.expand_dims(im, axis=0))[0]
    # ImageNet cat classes are 281-294

    return np.sum(pr[281:295])

def predict_map(img, n):
    dx = img.shape[0] // n
    heat = np.zeros((n, n), dtype=np.float32)
    for i in range(n):
        for j in range(n):
            tile = img[dx*i:dx*(i+1), dx*j:dx*(j+1)]
            heat[i, j] = predict(tile)
    return heat

```

This technique demonstrates how classification models can approximate localization through exhaustive search, though it remains computationally expensive for real-world applications.

## Stage 2: Bounding-Box Regression on Synthetic Data

The second stage introduces the core concept of **direct coordinate prediction** using a custom regression network. The notebook generates synthetic 8×8 grayscale images containing black rectangles via the `generate_images` function, creating a controlled dataset for training.

A small fully-connected network built with `keras.Sequential` learns to predict the four bounding-box parameters `[x, y, w, h]`. The architecture flattens the input and passes it through a 200-unit ReLU layer with dropout before outputting normalized coordinates. The curriculum evaluates performance using a custom **Intersection-over-Union (IoU)** metric.

```python
import numpy as np

def generate_images(num_imgs, img_size=8, min_sz=1, max_sz=4):
    bboxes = np.zeros((num_imgs, 4))
    imgs = np.zeros((num_imgs, img_size, img_size))
    for i in range(num_imgs):
        w, h = np.random.randint(min_sz, max_sz, size=2)
        x = np.random.randint(0, img_size - w)
        y = np.random.randint(0, img_size - h)
        imgs[i, x:x+w, y:y+h] = 1.  # draw rectangle

        bboxes[i] = [x, y, w, h]
    return imgs, bboxes

def IOU(b1, b2):
    x1, y1, w1, h1 = b1
    x2, y2, w2, h2 = b2
    w_i = min(x1 + w1, x2 + w2) - max(x1, x2)
    h_i = min(y1 + h1, y2 + h2) - max(y1, y2)
    if w_i <= 0 or h_i <= 0: 
        return 0.0
    inter = w_i * h_i
    union = w1 * h1 + w2 * h2 - inter
    return inter / union

# Generate and normalize data

imgs, bboxes = generate_images(100_000)
bb_norm = bboxes / 8.0  # scale to [0, 1]

# Build regression model

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(8, 8)),
    keras.layers.Dense(200, activation="relu"),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(4)  # predicts [x, y, w, h]

])
model.compile(optimizer="sgd", loss="mse")
model.fit((imgs - np.mean(imgs)) / np.std(imgs), bb_norm, epochs=30)

```

This stage establishes the fundamental mechanics of modern single-shot detectors without the complexity of working with large-scale image datasets.

## Stage 3: Production-Grade Detection with RetinaNet

After mastering the basics, the curriculum directs learners to **production-ready architectures** by referencing the official Keras RetinaNet implementation. Unlike the simplified regression model, RetinaNet incorporates a Feature Pyramid Network (FPN), focal loss, and multi-scale anchor boxes to handle objects at various scales and aspect ratios.

The notebook explicitly points to the Keras RetinaNet tutorial (`keras.io/examples/vision/retinanet/`) and the `fizyr/keras-retinanet` library. This transition illustrates how the simple regression concepts from Stage 2 scale into state-of-the-art detectors that process high-resolution images with multiple objects per frame.

## Summary

- The AI-For-Beginners curriculum in `microsoft/AI-For-Beginners` teaches object detection through three progressive TensorFlow implementations.
- **Stage 1** uses `keras.applications.vgg16.VGG16` with a sliding-window strategy to build class activation heat-maps without training new weights.
- **Stage 2** implements a custom `keras.Sequential` regressor on synthetic 8×8 images, predicting `[x, y, w, h]` coordinates with mean-squared error loss and evaluating with the `IOU` metric.
- **Stage 3** bridges to production systems by referencing RetinaNet, demonstrating how feature pyramids and focal loss solve real-world detection challenges.
- All implementations are contained in `lessons/4-ComputerVision/11-ObjectDetection/ObjectDetection.ipynb`.

## Frequently Asked Questions

### What TensorFlow version does the AI-For-Beginners curriculum require?

The code examples in `ObjectDetection.ipynb` use the modern Keras API integrated into TensorFlow 2.x. The `keras.applications` module and `Sequential` API patterns shown are compatible with TensorFlow 2.4 and later versions.

### Do I need a GPU to run the object detection examples?

The sliding-window VGG-16 demonstration and synthetic 8×8 regression model run efficiently on CPU. However, the RetinaNet production models referenced in Stage 3 require GPU acceleration for reasonable training times on real datasets.

### How does the sliding-window approach differ from RetinaNet?

The sliding-window method in Stage 1 evaluates multiple image crops independently using a classifier, resulting in redundant computations. RetinaNet (Stage 3) uses a single-shot detector architecture that predicts bounding boxes and class probabilities in one forward pass through a feature pyramid network.

### Where can I find the dataset used in the bounding-box regression example?

The curriculum generates synthetic data on-the-fly using the `generate_images` function, which creates random rectangles on 8×8 grids. This eliminates external dependencies and allows learners to focus purely on the regression mechanics without data preprocessing overhead.