# Python Libraries Used for Computer Vision in AI for Beginners: Complete Guide

> Explore essential Python libraries for computer vision in AI for beginners. Learn TensorFlow Keras PyTorch OpenCV Pillow and scikit-image for image processing and AI projects.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: getting-started
- Published: 2026-08-28

---

**The Computer Vision module in Microsoft's AI-for-Beginners curriculum relies on a dual-framework approach using TensorFlow/Keras and PyTorch alongside specialized image-processing libraries like OpenCV, Pillow, and scikit-image.**

The repository provides hands-on notebooks and helper scripts that teach foundational computer vision concepts through practical implementation. According to the source code in `microsoft/AI-For-Beginners`, the curriculum deliberately combines modern deep-learning ecosystems with classic computer vision tools to give beginners comprehensive exposure to both high-level model building and low-level image manipulation.

## Core Deep Learning Frameworks

The curriculum adopts a framework-agnostic philosophy, teaching parallel implementations in both TensorFlow and PyTorch to ensure learners understand core concepts regardless of their preferred stack.

### TensorFlow and Keras

**TensorFlow** serves as the primary deep-learning framework for CNN implementations, transfer learning, and semantic segmentation tasks. The high-level **Keras API** (accessed via `from tensorflow import keras`) simplifies model definition and training loops.

In [`tfcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/tfcv.py), the curriculum encapsulates TensorFlow vision utilities, while notebooks like `TransferLearningTF.ipynb` demonstrate production-ready patterns. The typical TensorFlow workflow involves defining sequential models with convolutional layers:

```python
import tensorflow as tf
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Conv2D(32, 3, activation="relu", input_shape=(28, 28, 1)),
    keras.layers.MaxPooling2D(),
    keras.layers.Flatten(),
    keras.layers.Dense(10, activation="softmax")
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])

```

### PyTorch and Torchvision

**PyTorch** provides the alternative computational backbone, used extensively for custom CNNs, GANs, auto-encoders, and transfer learning examples. The companion **torchvision** library handles dataset loading (MNIST, ImageNet) and standard transformations.

The file [`pytorchcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/pytorchcv.py) contains PyTorch-specific vision helpers, while `TransferLearningPyTorch.ipynb` showcases advanced architectures. The curriculum demonstrates PyTorch's imperative style through simple CNN definitions:

```python
import torch
import torch.nn as nn
import torch.nn.functional as F

class SimpleCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(1, 32, 3)
        self.pool = nn.MaxPool2d(2)
        self.fc = nn.Linear(32 * 13 * 13, 10)

    def forward(self, x):
        x = F.relu(self.conv(x))
        x = self.pool(x)
        x = x.view(x.size(0), -1)
        return F.log_softmax(self.fc(x), dim=1)

model = SimpleCNN()

```

## Image Processing and I/O Libraries

Beyond neural network frameworks, the curriculum integrates specialized libraries for image manipulation and preprocessing.

### Pillow and OpenCV

**Pillow** (`from PIL import Image`) handles basic image I/O operations including resizing, cropping, and format conversion across both TensorFlow and PyTorch examples. It appears consistently in [`pytorchcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/pytorchcv.py) for loading training data.

**OpenCV** (`import cv2`) provides low-level computer vision capabilities used specifically in `ObjectDetection.ipynb` for video processing and visualization. The library handles frame capture and annotation:

```python
import cv2

cap = cv2.VideoCapture("video.mp4")
ret, frame = cap.read()
if ret:
    cv2.rectangle(frame, (50, 50), (200, 200), (0, 255, 0), 2)
    cv2.imshow("Frame", frame)
    cv2.waitKey(0)
cap.release()
cv2.destroyAllWindows()

```

### scikit-image

**scikit-image** supplements the preprocessing pipeline in semantic segmentation notebooks. `SemanticSegmentationTF.ipynb` imports specific utilities for image resizing and transformation:

```python
from skimage.io import imread
from skimage.transform import resize

img = imread("data/sample.jpg")
img_resized = resize(img, (224, 224))

```

## Visualization and Training Utilities

Supporting libraries manage numerical operations, progress tracking, and result visualization throughout the Computer Vision lessons.

### NumPy and Matplotlib

**NumPy** provides the foundational array structure for interoperability between frameworks, converting Pillow images to normalized arrays and handling tensor manipulation. **Matplotlib** generates training curves and visualizes convolutional layer outputs in [`tfcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/tfcv.py):

```python
import matplotlib.pyplot as plt

def plot_results(hist):
    plt.figure(figsize=(12, 4))
    plt.subplot(1, 2, 1)
    plt.plot(hist["train_acc"], label="train")
    plt.plot(hist["val_acc"], label="val")
    plt.title("Accuracy")
    plt.legend()
    plt.subplot(1, 2, 2)
    plt.plot(hist["train_loss"], label="train")
    plt.plot(hist["val_loss"], label="val")
    plt.title("Loss")
    plt.legend()
    plt.show()

```

### tqdm and torchinfo

**tqdm** adds interactive progress bars to long training loops, improving the notebook experience in `SemanticSegmentationTF.ipynb`. **torchinfo** (used in `TransferLearningPyTorch.ipynb`) generates detailed model summaries showing layer shapes and parameter counts via `from torchinfo import summary`.

## Key Implementation Files

The following source files demonstrate specific library combinations:

- **[`pytorchcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/pytorchcv.py)** – PyTorch and torchvision utilities for dataset handling and model definitions
- **[`tfcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/tfcv.py)** – TensorFlow/Keras helper functions for CNN construction and training
- **`ObjectDetection.ipynb`** – OpenCV integration for video processing and bounding box visualization
- **`SemanticSegmentationTF.ipynb`** – TensorFlow segmentation pipelines with scikit-image preprocessing and tqdm progress tracking
- **`SemanticSegmentationPytorch.ipynb`** – PyTorch implementation of segmentation with torchvision transforms
- **`TransferLearningPyTorch.ipynb`** – PyTorch transfer learning featuring torchinfo model summaries
- **`TransferLearningTF.ipynb`** – TensorFlow transfer learning utilizing the custom `tfcv` module

## Summary

- The AI-for-Beginners Computer Vision curriculum employs **both TensorFlow/Keras and PyTorch** to teach framework-agnostic deep learning concepts
- **Pillow** and **OpenCV** handle image I/O and low-level processing alongside **scikit-image** for specialized transformations
- **NumPy** serves as the interoperability layer between image libraries and deep learning frameworks
- **Matplotlib**, **tqdm**, and **torchinfo** provide essential visualization and monitoring capabilities during training
- All libraries appear in specific lesson files such as [`pytorchcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/pytorchcv.py), [`tfcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/tfcv.py), and `ObjectDetection.ipynb`, confirming their integral role in the learning path

## Frequently Asked Questions

### What are the main deep learning frameworks taught in the Computer Vision lessons?

The curriculum teaches both **TensorFlow/Keras** and **PyTorch** as equal alternatives. TensorFlow appears in [`tfcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/tfcv.py) and segmentation notebooks, while PyTorch dominates [`pytorchcv.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/pytorchcv.py) and transfer learning examples. This dual approach ensures beginners understand convolutional neural network concepts regardless of their production framework choice.

### Does the curriculum use OpenCV or Pillow for image processing?

Both libraries serve distinct purposes. **Pillow** handles general image loading and preprocessing across most notebooks, while **OpenCV** appears specifically in `ObjectDetection.ipynb` for video frame capture and real-time bounding box drawing. The curriculum uses them complementarily rather than interchangeably.

### Which utility libraries improve the training experience in the notebooks?

**tqdm** provides progress bars for long-running training loops in `SemanticSegmentationTF.ipynb`, while **torchinfo** generates detailed architecture summaries in `TransferLearningPyTorch.ipynb`. **Matplotlib** visualizes training metrics and intermediate feature maps throughout the Computer Vision module, creating immediate feedback for learners.