How to Debug Issues When Running Code in AI Engineering from Scratch

Start with scripts/audit_lessons.py to validate repository structure, then run the lesson's main.py and its code/tests/ suite to isolate and fix runtime errors in the rohitg00/ai-engineering-from-scratch curriculum.

The ai-engineering-from-scratch repository is a 503-lesson curriculum where every phase contains raw-first implementations, narrative documentation, and self-contained test suites. Because each lesson is designed to run independently, debugging issues when running the code follows a repeatable workflow that relies on built-in invariant checks before you ever inspect a traceback. This guide walks through the exact scripts, file paths, and validation steps defined in the source code.

Debug Structural Issues First with scripts/audit_lessons.py

The fastest way to surface problems is to run the repository-wide audit. According to the source in scripts/audit_lessons.py, the script performs a full-stack lint of every lesson directory and exits with code 0 when the repository is clean.

The audit validates five invariants directly from its implementation:

• Folder naming convention (scripts/audit_lessons.py, lines 25-27) — Each lesson directory must match the NN-slug pattern.
• Documentation requirements (lines 97-115) — Every docs/en.md must exist, be valid UTF-8, contain a top-level heading, and be at least 200 bytes long.
• Source presence (lines 19-26) — The code/ subtree must contain at least one source file, ignoring known placeholders.
• Quiz schema integrity (lines 30-88) — Quizzes are checked for required keys, option count, and a correct-index range inside allowed bounds.
• Internal link validity (lines 96-112) — Broken internal Markdown links inside docs/en.md are detected and reported with exact paths.

Because the output includes the relative path that failed — for example, phases/14-agent-engineering/03-reflexion-and-verbal-rl/docs/en.md — you can jump straight to the offending file.

# Full audit of the entire curriculum

python scripts/audit_lessons.py

# Targeted audit while working on a single phase

python scripts/audit_lessons.py --phase 14

Debug Runtime Issues When Running the Lesson Code

After the audit passes, the next step is to execute the individual lesson. Every lesson exposes a runnable entry point inside phases/<phase>/<lesson>/code/, typically named main.py or a language-specific equivalent such as main.ts, main.rs, or main.jl.

Run the Lesson main.py Entry Point

Running the file directly should either print results and exit with status 0, or raise an exception that names the exact failure mode.

# Example: run "The Agent Loop" lesson (Phase 14, Lesson 01)

python phases/14-agent-engineering/01-the-agent-loop/code/main.py

If the script aborts, the Python traceback gives you the precise line number and module in the lesson's code/ directory. Because the algorithms are implemented from scratch before any framework is introduced, most errors are logic-level and visible at this layer.

Isolate Failures with the code/tests/ Unit Test Suite

Each lesson ships a code/tests/ folder containing at least five unittest cases. Invoking the test runner narrows the failure to a single function or class, which is far easier to debug than the whole script.

# Discover and run all tests for a lesson

python -m unittest discover phases/14-agent-engineering/01-the-agent-loop/code/tests -v

A passing suite confirms the core logic satisfies the lesson specification. A failing test pinpoints the exact contract that is broken.

Debug Missing Artefact Issues in the outputs/ Directory

Every lesson's Ship It step writes prompts, skills, agents, or MCP servers into outputs/. If a downstream lesson cannot locate a required asset, the error often surfaces as a missing file inside phases/<NN-phase>/<NN-lesson>/outputs/.

Verify the presence of the expected artefact and compare it against the requirements in docs/en.md.

# List artefacts produced by the "The Agent Loop" lesson

ls phases/14-agent-engineering/01-the-agent-loop/outputs/

If the directory is empty or missing a file, re-run the upstream lesson's main.py until the artefact is generated.

Debug Site-Wide Link Issues with site/build.js

The file site/build.js parses the markdown links in README.md to generate site/data.js. If a lesson's link is malformed, the site build fails and the CI job reports a link-check error.

Fixing the markdown entry in README.md — using the format [Lesson Title](phases/14-agent-engineering/01-the-agent-loop/) — resolves this class of problem before it affects other learners.

Debug Cross-Lesson Workflows with Helper Scripts

Two repository-wide helpers reduce friction when debugging across multiple lessons:

• scripts/lesson_run.py — Acts as a uniform entry point that selects the appropriate language runtime, configures the environment, and displays any stdout or stderr. Using this wrapper can expose path or configuration issues that do not appear when calling main.py directly.
• scripts/install_skills.py — Installs generated SKILL.md files into a local skill registry so you can test a skill from any compatible agent. If an agent cannot load a skill you built, confirm the SKILL.md exists in the lesson's outputs/ directory and rerun this installer.

When available, pass a --dry-run flag to these helpers to surface configuration errors before a full execution.

Summary

• Run python scripts/audit_lessons.py to catch structural, documentation, quiz-schema, and link violations before executing any code.
• Execute phases/<phase>/<lesson>/code/main.py to reveal runtime tracebacks in the raw-first implementation.
• Run python -m unittest discover inside code/tests/ to isolate broken logic to a specific function or class contract.
• Inspect phases/<phase>/<lesson>/outputs/ for missing prompts, skills, agents, or MCP servers that downstream lessons require.
• Repair markdown links in README.md when site/build.js or CI reports link-check failures.
• Use scripts/lesson_run.py and scripts/install_skills.py to standardise execution and skill registration across the 503-lesson curriculum.

Frequently Asked Questions

What is the first step to debug any lesson in AI Engineering from Scratch?

Run python scripts/audit_lessons.py. It validates folder naming conventions (NN-slug), UTF-8 documentation, source-file presence, quiz schemas, and internal Markdown links across all 503 lessons, returning the exact file path and rule that failed.

Where are the unit tests located for each lesson?

Every lesson stores its unittest suite inside phases/<NN-phase>/<NN-lesson>/code/tests/. Invoke them with python -m unittest discover <path> -v to run at least five test cases that verify the lesson's core logic against its specification.

How can I check if a lesson produced the required output artefacts?

List the contents of phases/<NN-phase>/<NN-lesson>/outputs/. If a downstream lesson fails because a prompt, skill, agent, or MCP server is missing, re-run the upstream lesson's main.py and confirm the expected file appears in that directory.

What script validates markdown links and quiz schemas across the entire curriculum?

scripts/audit_lessons.py enforces quiz schema rules — including required keys, option counts, and correct-index ranges in lines 30-88 — and detects broken internal Markdown links in docs/en.md through its link-validation logic, as implemented in the rohitg00/ai-engineering-from-scratch repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}