# Debugging and Profiling Techniques in Phase 0: A Complete Guide to AI Diagnostics

> Master Phase 0 AI diagnostics with 9 essential debugging and profiling techniques. Learn GPU memory tracking and more to build robust neural networks. Get the complete toolkit now.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: tutorial
- Published: 2026-07-30

---

**Phase 0 of the AI‑Engineering‑From‑Scratch curriculum teaches nine essential debugging and profiling techniques—from simple print statements to GPU memory tracking—that form a complete diagnostic toolkit for neural‑network development.**

The **rohitg00/ai-engineering-from-scratch** repository structures its curriculum into distinct phases, with Phase 0 (*Setup & Tooling*) dedicated to foundational workflows. Lesson **12‑debugging‑and‑profiling** provides a systematic introduction to identifying performance bottlenecks and silent bugs in PyTorch code before students advance to model building.

## The Nine Core Debugging and Profiling Techniques

The lesson [`phases/00-setup-and-tooling/12-debugging-and-profiling/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/00-setup-and-tooling/12-debugging-and-profiling/docs/en.md) breaks diagnostics into nine practical categories, each illustrated with runnable Python examples.

### Print Debugging

**Print debugging** remains the fastest way to inspect tensor states during training. The curriculum demonstrates using simple `print` statements and custom `debug_print` helpers to validate tensor shapes, dtypes, and detect NaN values mid‑training.

### Python Debugger (pdb and breakpoint)

For interactive investigation, the lesson covers the **Python Debugger (pdb)** and the modern `breakpoint()` API. Students learn to step through execution line‑by‑line, inspect local variables, and set **conditional breakpoints** that trigger only when specific tensor conditions are met.

### Python Logging

Moving beyond ad‑hoc prints, the curriculum teaches the **`logging` module** for production‑grade diagnostics. This includes configuring severity levels (DEBUG, INFO, WARNING), formatting log messages, and capturing output to rotating files for long training runs.

### Timing Code Sections

To locate performance bottlenecks, students use **`time.perf_counter()`** and custom `Timer` context managers. These utilities measure elapsed time for critical sections like forward passes or data loading pipelines.

### cProfile and line_profiler

For systematic profiling, the lesson introduces **cProfile** for whole‑program statistical profiling and **line_profiler** for line‑level granularity. These tools identify hot spots by showing exact execution counts and time spent per line of model code.

### Memory Profiling

Memory diagnostics cover three distinct layers:
- **CPU memory** tracking with **`tracemalloc`** for locating reference leaks
- **CPU line‑level** profiling with **`memory_profiler`** 
- **GPU memory** monitoring using PyTorch’s **`torch.cuda.memory_allocated`** and `torch.cuda.max_memory_allocated`

### Common AI Bugs and Detection Methods

The curriculum dedicates a section to **AI‑specific failure modes**:
- **Shape mismatches** between layers or batch dimensions
- **NaN losses** from gradient explosions or bad initialization
- **Data leakage** between train/validation splits
- **Wrong device placement** (CPU/GPU mismatch)

### TensorBoard Basics

For visual debugging, students learn **TensorBoard** integration: logging scalars for loss curves, histograms for weight distributions, and image grids for batch visualization. This replaces console‑only debugging with time‑series analysis.

### VS Code Debugger Integration

Finally, the lesson covers IDE integration by configuring **[`launch.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/launch.json)** for VS Code using **`debugpy`**. This enables seamless breakpoint debugging inside the editor without terminal pdb commands.

## Reference Implementation: debug_tools.py

All nine techniques are implemented in a single utility script located at [`phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py). The file contains runnable demonstrations for immediate use:

```python

# phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py

def demo_print_debugging():
    """Simple tensor inspection prints."""
    pass

def demo_timing():
    """Context manager timing examples."""
    pass

def demo_memory_tracking():
    """CPU memory with tracemalloc."""
    pass

def demo_shape_checking():
    """Validate tensor dimensions."""
    pass

def demo_nan_detection():
    """Check for NaN in losses."""
    pass

def demo_device_checking():
    """Verify tensor device placement."""
    pass

def demo_gradient_health():
    """Inspect gradient norms."""
    pass

def demo_gpu_memory():
    """Monitor torch.cuda allocations."""
    pass

def demo_logging():
    """Structured logging setup."""
    pass

def demo_conditional_breakpoint():
    """Smart breakpoint insertion."""
    pass

def main():
    demo_print_debugging()
    demo_timing()
    demo_memory_tracking()
    # Sequential demonstration of all techniques

```

Running `python phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py` executes a step‑by‑step walkthrough, outputting diagnostic markers like:

```text
▶️  Print debugging – tensor shape: torch.Size([32, 128])
🕒  Timing – forward pass took 0.0123 s
📊  Memory – current CPU allocation: 12.4 MiB, GPU allocation: 45.2 MiB
⚠️  NaN detection – loss contains NaN at step 27

```

## Key Files and Resources

The debugging lesson ships with three primary resources:

- **[`phases/00-setup-and-tooling/12-debugging-and-profiling/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/00-setup-and-tooling/12-debugging-and-profiling/docs/en.md)** — Narrative documentation detailing usage patterns, common pitfalls, and when to apply each technique
- **[`phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/00-setup-and-tooling/12-debugging-and-profiling/code/debug_tools.py)** — Reference implementations of all nine debugging utilities
- **[`phases/00-setup-and-tooling/12-debugging-and-profiling/outputs/prompt-debug-ai-code.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/00-setup-and-tooling/12-debugging-and-profiling/outputs/prompt-debug-ai-code.md)** — A pre‑crafted prompt for generating AI‑assisted debugging suggestions on custom code snippets

## Summary

Phase 0 of the AI‑Engineering‑From‑Scratch curriculum provides a layered debugging and profiling toolkit:

- **Quick diagnostics** via print statements and Python’s built‑in debugger
- **Systematic observation** through the logging module and TensorBoard visualization
- **Performance analysis** using cProfile, line_profiler, and custom timing contexts
- **Resource monitoring** with tracemalloc, memory_profiler, and PyTorch CUDA APIs
- **AI‑specific checks** for shape mismatches, NaN values, data leakage, and device placement

These techniques, implemented in [`debug_tools.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/debug_tools.py), create a portable diagnostic kit students reuse throughout the curriculum.

## Frequently Asked Questions

### How do I start the interactive debugger in the middle of training?

Insert **`breakpoint()`** at any line in your training loop. When Python reaches that line, it drops into the pdb interactive shell where you can inspect variables, step through execution, or continue with `c`. For conditional stopping, wrap it in an `if` statement checking for NaN losses or abnormal tensor values.

### What is the difference between cProfile and line_profiler?

**cProfile** provides function‑level statistics showing call counts and total time per function, making it ideal for finding which model components consume the most time. **line_profiler** requires decoration with `@profile` and shows time spent on individual lines within functions, perfect for optimizing specific hot loops without guessing which line causes slowdowns.

### How do I detect GPU memory leaks in PyTorch training?

Use **`torch.cuda.memory_allocated()`** and **`torch.cuda.max_memory_allocated()`** at strategic points in your training loop—after forward pass, backward pass, and optimizer step. If allocated memory increases every epoch without resetting, you likely have retained tensors or circular references. Pair this with **`tracemalloc`** for CPU leaks to cover both memory spaces.