# How to Debug Issues When Running Code in AI Engineering from Scratch > Debug runtime errors in AI Engineering from Scratch by validating structure with audit_lessons.py and running lesson tests via main.py. Fix issues in the rohitg00/ai-engineering-from-scratch curriculum efficiently. - Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch) - Tags: how-to-guide - Published: 2026-06-06 --- **Start with [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) to validate repository structure, then run the lesson's [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) and its `code/tests/` suite to isolate and fix runtime errors in the rohitg00/ai-engineering-from-scratch curriculum.** The *ai-engineering-from-scratch* repository is a 503-lesson curriculum where every phase contains raw-first implementations, narrative documentation, and self-contained test suites. Because each lesson is designed to run independently, debugging issues when running the code follows a repeatable workflow that relies on built-in invariant checks before you ever inspect a traceback. This guide walks through the exact scripts, file paths, and validation steps defined in the source code. ## Debug Structural Issues First with [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) The fastest way to surface problems is to run the repository-wide audit. According to the source in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), the script performs a full-stack lint of every lesson directory and exits with code 0 when the repository is clean. The audit validates five invariants directly from its implementation: - **Folder naming convention** ([`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), lines 25-27) — Each lesson directory must match the `NN-slug` pattern. - **Documentation requirements** (lines 97-115) — Every [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must exist, be valid UTF-8, contain a top-level heading, and be at least 200 bytes long. - **Source presence** (lines 19-26) — The `code/` subtree must contain at least one source file, ignoring known placeholders. - **Quiz schema integrity** (lines 30-88) — Quizzes are checked for required keys, option count, and a correct-index range inside allowed bounds. - **Internal link validity** (lines 96-112) — Broken internal Markdown links inside [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) are detected and reported with exact paths. Because the output includes the relative path that failed — for example, [`phases/14-agent-engineering/03-reflexion-and-verbal-rl/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/03-reflexion-and-verbal-rl/docs/en.md) — you can jump straight to the offending file. ```bash # Full audit of the entire curriculum python scripts/audit_lessons.py # Targeted audit while working on a single phase python scripts/audit_lessons.py --phase 14 ``` ## Debug Runtime Issues When Running the Lesson Code After the audit passes, the next step is to execute the individual lesson. Every lesson exposes a runnable entry point inside `phases///code/`, typically named [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) or a language-specific equivalent such as [`main.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.ts), [`main.rs`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.rs), or `main.jl`. ### Run the Lesson [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) Entry Point Running the file directly should either print results and exit with status 0, or raise an exception that names the exact failure mode. ```bash # Example: run "The Agent Loop" lesson (Phase 14, Lesson 01) python phases/14-agent-engineering/01-the-agent-loop/code/main.py ``` If the script aborts, the Python traceback gives you the precise line number and module in the lesson's `code/` directory. Because the algorithms are implemented from scratch before any framework is introduced, most errors are logic-level and visible at this layer. ### Isolate Failures with the `code/tests/` Unit Test Suite Each lesson ships a `code/tests/` folder containing at least five `unittest` cases. Invoking the test runner narrows the failure to a single function or class, which is far easier to debug than the whole script. ```bash # Discover and run all tests for a lesson python -m unittest discover phases/14-agent-engineering/01-the-agent-loop/code/tests -v ``` A passing suite confirms the core logic satisfies the lesson specification. A failing test pinpoints the exact contract that is broken. ## Debug Missing Artefact Issues in the `outputs/` Directory Every lesson's *Ship It* step writes prompts, skills, agents, or MCP servers into `outputs/`. If a downstream lesson cannot locate a required asset, the error often surfaces as a missing file inside `phases///outputs/`. Verify the presence of the expected artefact and compare it against the requirements in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md). ```bash # List artefacts produced by the "The Agent Loop" lesson ls phases/14-agent-engineering/01-the-agent-loop/outputs/ ``` If the directory is empty or missing a file, re-run the upstream lesson's [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) until the artefact is generated. ## Debug Site-Wide Link Issues with [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) The file [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) parses the markdown links in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) to generate [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js). If a lesson's link is malformed, the site build fails and the CI job reports a link-check error. Fixing the markdown entry in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) — using the format `[Lesson Title](phases/14-agent-engineering/01-the-agent-loop/)` — resolves this class of problem before it affects other learners. ## Debug Cross-Lesson Workflows with Helper Scripts Two repository-wide helpers reduce friction when debugging across multiple lessons: - **[`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py)** — Acts as a uniform entry point that selects the appropriate language runtime, configures the environment, and displays any stdout or stderr. Using this wrapper can expose path or configuration issues that do not appear when calling [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) directly. - **[`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py)** — Installs generated [`SKILL.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/SKILL.md) files into a local skill registry so you can test a skill from any compatible agent. If an agent cannot load a skill you built, confirm the [`SKILL.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/SKILL.md) exists in the lesson's `outputs/` directory and rerun this installer. When available, pass a `--dry-run` flag to these helpers to surface configuration errors before a full execution. ## Summary - **Run `python scripts/audit_lessons.py`** to catch structural, documentation, quiz-schema, and link violations before executing any code. - **Execute `phases///code/main.py`** to reveal runtime tracebacks in the raw-first implementation. - **Run `python -m unittest discover`** inside `code/tests/` to isolate broken logic to a specific function or class contract. - **Inspect `phases///outputs/`** for missing prompts, skills, agents, or MCP servers that downstream lessons require. - **Repair markdown links** in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) when [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) or CI reports link-check failures. - **Use [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) and [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py)** to standardise execution and skill registration across the 503-lesson curriculum. ## Frequently Asked Questions ### What is the first step to debug any lesson in AI Engineering from Scratch? Run `python scripts/audit_lessons.py`. It validates folder naming conventions (`NN-slug`), UTF-8 documentation, source-file presence, quiz schemas, and internal Markdown links across all 503 lessons, returning the exact file path and rule that failed. ### Where are the unit tests located for each lesson? Every lesson stores its `unittest` suite inside `phases///code/tests/`. Invoke them with `python -m unittest discover -v` to run at least five test cases that verify the lesson's core logic against its specification. ### How can I check if a lesson produced the required output artefacts? List the contents of `phases///outputs/`. If a downstream lesson fails because a prompt, skill, agent, or MCP server is missing, re-run the upstream lesson's [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) and confirm the expected file appears in that directory. ### What script validates markdown links and quiz schemas across the entire curriculum? [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) enforces quiz schema rules — including required keys, option counts, and correct-index ranges in lines 30-88 — and detects broken internal Markdown links in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) through its link-validation logic, as implemented in the rohitg00/ai-engineering-from-scratch repository.