# How to Run Tests for a Specific Lesson Using the Stdlib Test Runners in AI Engineering From Scratch

> Learn to run specific tests for AI Engineering From Scratch lessons. Use the stdlib test runners or the lesson_run.py script for efficient testing and debugging.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: testing
- Published: 2026-06-16

---

**To run tests for a specific lesson in the ai-engineering-from-scratch curriculum, navigate to the lesson directory and execute `python3 -m unittest discover -s code/tests -v` from the lesson root, or use the [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) helper for automated syntax checking and execution.**

The **ai-engineering-from-scratch** repository enforces a strict "stdlib-only" testing policy that prohibits external frameworks like **pytest** or **Jest**. Each lesson contains unit tests designed to run with the language's standard library runner, ensuring zero-dependency validation across the curriculum.

## Locating Lesson Tests in the Repository

Each lesson lives under the path `phases/<phase-number>-<phase-slug>/<lesson-number>-<lesson-slug>/`. The executable code resides in the `code/` subdirectory, while unit tests are stored in `code/tests/` (or adjacent to it). For example, a reinforcement learning lesson on Markov Decision Processes would be located at `phases/09-reinforcement-learning/01-mdps-states-actions-rewards/`.

To verify the test structure exists:

```bash
cd phases/09-reinforcement-learning/01-mdps-states-actions-rewards
ls code/tests

```

Expected output includes `test_*.py` files such as [`test_main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/test_main.py) or [`test_utils.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/test_utils.py).

## Running Tests with the Stdlib Runner

According to the official guidance in **[`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md)** (lines 111-112), all test suites must run via the language's stdlib runner. This design choice eliminates external dependencies and ensures lessons remain portable across environments.

### Python unittest Discovery

For Python lessons, use the built-in `unittest` module with discovery mode. From the lesson root (the directory containing `code/`), execute:

```bash
python3 -m unittest discover -s code/tests -v

```

The `-s code/tests` flag specifies the discovery start directory, while `-v` enables verbose output for debugging. This command automatically finds all `test_*.py` files and executes them using the standard library harness.

### TypeScript and Compiled Languages

TypeScript lessons use `npx tsx --test`, while **Rust** and **Julia** execute tests inline during compilation. These follow the same stdlib-only philosophy but require different commands based on their respective language specifications.

## Automating Tests with lesson_run.py

The repository provides **[`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py)**, a helper script that automates syntax checking and optional execution across the curriculum. As documented in the script's docstring (lines 16-21) and implemented in its `main` function (lines 24-28), it supports targeting specific phases or lessons with strict validation.

### Syntax-Only Validation

To validate all lessons in a phase without executing them:

```bash
python3 scripts/lesson_run.py --phase 09 --strict

```

The script walks the `phases/` tree, compiles all `.py` files for syntax verification, and reports a concise summary of passed, failed, and skipped lessons. Failures include the specific file path and error type, such as `SyntaxError: invalid syntax`.

### Executing Specific Lessons

To run both syntax checks and execution for a single lesson:

```bash
cd phases/09-reinforcement-learning/01-mdps-states-actions-rewards
python3 ../../../../scripts/lesson_run.py --execute

```

The script checks for `# requires:` headers in entry files and skips execution if external dependencies are declared, preventing runtime errors while reporting the missing requirements.

## Interpreting Test Results

The stdlib runners and helper script use standardized exit codes and output formats:

- **PASS**: All tests succeeded with exit code `0`.
- **FAIL**: Exceptions raised during testing display the offending test name and full traceback.
- **SKIP**: Lessons without Python files or with external dependency declarations are ignored during syntax-only runs.

## Summary

- Navigate to `phases/<phase>/<lesson>/` and run `python3 -m unittest discover -s code/tests -v` to execute Python tests directly using the standard library.
- Use `python3 scripts/lesson_run.py --phase <number> --strict` for automated syntax validation across multiple lessons.
- The [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) policy mandates stdlib-only testing with no external frameworks required.
- Lessons with external dependencies declare them via `# requires:` headers and are automatically skipped by the helper script.

## Frequently Asked Questions

### Where are the test files located in the ai-engineering-from-scratch repository?

Test files reside in the `code/tests/` subdirectory within each lesson directory, following the path pattern `phases/<phase-number>-<phase-slug>/<lesson-number>-<lesson-slug>/code/tests/`. These contain standard library unit tests named with the `test_*.py` convention.

### What testing framework does the curriculum use?

The repository enforces a strict stdlib-only policy as documented in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md). Python lessons use the built-in `unittest` module, TypeScript uses `tsx` with Node.js test runners, and Rust/Julia use inline compilation tests. No external testing frameworks like pytest or Jest are permitted.

### How do I skip lessons with external dependencies?

Lessons declaring external dependencies include a `# requires:` comment in their entry [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) file. The [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py) script automatically detects these headers and skips execution for those lessons, reporting them as skipped in the summary rather than failing with an import error.

### Can I run tests for an entire phase at once?

Yes. Use `python3 scripts/lesson_run.py --phase <number> --strict` to validate all lessons within a specific phase. Add the `--execute` flag to run entry files after syntax checks, though lessons with external dependencies will still be skipped automatically to prevent runtime failures.