# How the AI Engineering Curriculum's Lesson Audit Validation System Works

> Discover how the AI engineering curriculum's lesson audit validation system automatically checks lesson directories for compliance blocking merges in CI. Learn about this Python static analyzer.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-06-18

---

**The lesson audit validation system in rohitg00/ai-engineering-from-scratch is a Python-based static analyzer that automatically verifies every lesson directory for structural compliance, required documentation, and valid internal links, exiting with a non-zero status code when violations are detected to block merges in CI.**

The rohitg00/ai-engineering-from-scratch repository maintains strict quality standards for its self-paced AI engineering curriculum through an automated lesson audit validation system. This validation layer ensures that every lesson in the `phases/` directory contains mandatory documentation, functional code examples, and properly structured metadata before changes can be merged into the main branch.

## Core Architecture and Entry Points

The validation logic resides primarily in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), which serves as the main entry point for both local development checks and continuous integration enforcement. The script accepts optional flags to filter by phase, output machine-readable JSON, or enable strict mode.

```bash
python scripts/audit_lessons.py [--phase N] [--json] [--strict]

```

*This command structure is defined at line 5 of the main script.*

On execution, the validator performs a glob search across `phases/**/` to discover all lesson directories containing [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) files. Each parent folder of a discovered markdown file becomes a *lesson* candidate for validation.

## The Per-Lesson Validation Pipeline

For every discovered lesson, the system invokes `audit_lesson(audit, lesson_path)` at line 214, passing a central `Audit` object that aggregates violations. The function executes a series of discrete checks, each registering an `Issue` when rules fail.

### Directory Structure and Naming Compliance

**Rule L001** enforces the `<phase>/<lesson>` naming convention through the `check_lesson_dir_pattern` function (line 85). The validator ensures each lesson directory follows the established organizational pattern within the phases hierarchy.

### Documentation Requirements

**Rule L002** and **Rule L004** validate the presence and format of [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md). The `check_docs_en_md` function (lines 100–115) verifies that the file exists, uses UTF-8 encoding, and begins with a top-level H1 header (`# Title`) at line 115.

### Code Artifact Validation

**Rule L005** ensures the `code/` directory contains non-empty source or configuration files. The `check_code_main` function at line 126 confirms that every lesson includes executable reference material.

### Quiz Schema Enforcement

**Rule L006** validates [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) through the `check_quiz` function (lines 137–188). This check verifies that the file exists, contains valid JSON, and conforms to a strict schema requiring object entries with correct fields and exactly six questions per lesson.

### Internal Link Resolution

**Rule L010** validates Markdown hyperlinks using `check_internal_links` (lines 196–214). The system resolves both anchor links (`[text](#anchor)`) and cross-lesson references (`[text](../other-lesson)`) against the repository file system, utilizing the helper [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py) referenced in the companion comment at line 25.

## Issue Collection and Reporting

Each failed check creates an `Issue` object containing a rule code (e.g., `L002`), the lesson path, the offending file, and a human-readable message. The `Audit` object tracks the total count of `lessons_checked` and accumulates all discovered issues.

The `render_report(audit)` function (lines 225–235) generates a human-readable text summary of the validation run. When the `--json` flag is provided, the script emits a structured JSON payload (lines 257–263) containing the lesson count and a list of issue dictionaries suitable for programmatic consumption.

The script exits with status `1` when any issues are present, otherwise `0` (line 272). This exit code behavior enables CI systems to treat audit failures as blocking checks.

## CI/CD Integration and Merge Enforcement

The GitHub Actions workflow defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) executes the audit on every push and pull request:

```yaml
audit:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v3
    - name: Run lesson audit
      run: python scripts/audit_lessons.py

```

If the audit detects violations, the job fails immediately, preventing the pull request from merging until contributors resolve the highlighted issues—such as adding a missing [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) or correcting broken internal links. Once the audit passes, subsequent workflow jobs for README synchronization and site rebuilding proceed.

## Running the Audit Locally

Developers can execute targeted audits during curriculum development to validate specific phases or generate machine-readable reports:

```bash

# Validate only phase 04 (Computer Vision) and output JSON

python scripts/audit_lessons.py --phase 04 --json > audit-report.json

```

This local execution mirrors the CI environment, allowing contributors to identify compliance issues before submitting pull requests.

## Extending the Validation Rules

The modular architecture allows curriculum maintainers to add custom compliance checks by implementing new functions and registering them in the `audit_lesson` workflow. For example, to enforce a custom header format:

```python
def check_custom_header(audit: Audit, lesson: Path) -> None:
    header = lesson / "docs" / "en.md"
    first_line = header.read_text().splitlines()[0]
    if not first_line.startswith("# "):

        audit.add("L999", lesson, header, "Missing required H1 header")

```

Adding this call within `audit_lesson` causes the new `L999` rule to appear in reports alongside built-in checks, maintaining consistency with the existing issue code format.

## Summary

- The **lesson audit validation system** is implemented in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) with helper utilities in [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py).
- It validates **six specific compliance rules** (L001–L006, L010) covering directory naming, documentation presence, code content, quiz schema, and link resolution.
- The system uses a central `Audit` object to collect `Issue` instances, exiting with code `1` when violations exist to block CI merges.
- **JSON and text reporting** modes support both human review and automated bot integrations.
- The validator is integrated into [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) to enforce quality gates on every pull request.

## Frequently Asked Questions

### What happens if a lesson is missing the required quiz.json file?

The audit will fail with **Rule L006**, generating an issue in the report indicating that [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) is missing or invalid. The CI workflow will block the merge until the file is added with the correct schema, including exactly six questions with properly formatted object entries.

### How does the audit system handle internal links between lessons?

**Rule L010** invokes `check_internal_links` (lines 196–214) to resolve Markdown hyperlinks against the repository filesystem. The validator checks both anchor references within files and relative paths to other lessons, utilizing the [`link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/link_check.py) helper to ensure all internal navigation targets exist.

### Can I run the audit on a single lesson instead of the entire curriculum?

Yes. Use the `--phase` flag to limit validation to a specific phase directory, or filter the output manually. While there is no single-lesson flag, targeting a phase reduces the scope significantly. The `--json` flag outputs structured data that can be filtered by lesson path if needed.

### Where does the audit store its results when run in CI?

The audit outputs results to stdout, which GitHub Actions captures in the workflow logs. When using the `--json` flag in CI, the JSON report is printed to stdout and can be redirected to a file or parsed by subsequent workflow steps for PR comments or artifact storage.