# How to Run Local Validation Scripts Like audit_lessons.py Before Pushing

> Run audit_lessons.py locally before pushing to validate your AI Engineering From Scratch code. Ensure your changes pass CI checks for lesson structure, docs, and quizzes.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-09-04

---

**Run `python3 scripts/audit_lessons.py` from the repository root to validate lesson structure, documentation, and quiz schemas locally before pushing, ensuring your changes pass the same checks enforced by CI.**

The rohitg00/ai-engineering-from-scratch repository maintains curriculum integrity through deterministic **binary-level validation scripts** that check every lesson before it reaches the CI pipeline. Running local validation scripts like [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) before pushing catches structural violations immediately and prevents failed GitHub Actions workflows. Because the validator uses only Python standard libraries, you can execute comprehensive checks without installing additional dependencies.

## How the Validation Engine Works

### Entry Point and Command-Line Interface

The primary validation script resides at [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py). When invoked, it parses command-line flags—including `--phase`, `--json`, and `--strict`—and instantiates an `Audit` object to track violations across the curriculum. The script is entirely self-contained, requiring no external packages beyond a standard Python 3 interpreter.

### Lesson Discovery

The `iter_lesson_dirs` function walks the `phases/` directory tree, optionally filtering by a phase number passed via `--phase`, and yields each lesson directory for inspection. This ensures that validation scales from individual phases to the entire repository without manual file globbing.

### Rule Checks (L001–L009)

For every discovered lesson, the script executes a series of deterministic rule functions:

- **`check_lesson_dir_pattern`** (L001): Validates that the directory name follows the required naming convention.
- **`check_docs_en_md`** (L002–L004): Verifies that [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists, is valid UTF-8, meets minimum size requirements, and contains a top-level H1 heading.
- **`check_code_main`** (L005): Guarantees that the `code/` subdirectory contains at least one non-ignored file.
- **`check_quiz`** (L006–L009): Validates JSON schema compliance, ensures correct option counts, and checks that `correct` indices point to valid choices.
- **`check_internal_links`**: Confirms that every Markdown link references an existing file within the repository boundary.

Each violation is recorded against its rule identifier in the `Audit` instance for final reporting.

### Reporting and Exit Codes

After processing all lessons, the script renders results through either `render_report` (human-readable) or a JSON emitter (`--json`). The exit code is `0` for a clean run and `1` if any issues are discovered, making it suitable for scripting and CI gates.

### CI Integration

The [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) workflow executes `python scripts/audit_lessons.py` on every pull request. Because the CI environment runs the same code you run locally, local validation guarantees that merges will not fail due to structural errors.

## Running audit_lessons.py Locally

### Validate All Phases

Execute the following from the repository root to audit every lesson across all phases:

```bash
python3 scripts/audit_lessons.py

```

This produces a human-readable report of any L001–L009 violations. An exit status of `0` confirms the curriculum structure is valid.

### Target a Specific Phase

To limit validation to a single phase—useful when iterating on new content—pass the `--phase` flag with the phase number:

```bash
python3 scripts/audit_lessons.py --phase 14

```

This restricts the audit to `phases/14-agent-engineering/`, reducing execution time and focusing output on the work in progress.

### Generate JSON Reports for Automation

For integration with custom dashboards or CI parsers, output the results as JSON:

```bash
python3 scripts/audit_lessons.py --json > audit-report.json

```

The resulting document contains `lessons_checked` and an array of `issues` with file paths and rule identifiers, enabling automated triage.

### Enable Strict Mode

To future-proof your validation against upcoming stricter policies, include the `--strict` flag:

```bash
python3 scripts/audit_lessons.py --strict

```

Currently, `--strict` behaves identically to the default mode, but it signals intent to adopt stricter validation as the curriculum evolves.

### Validate Certification Curricula

The repository includes a separate validator for certification tracks. Run the certification audit to check Claude-specific curriculum rules:

```bash
python3 scripts/audit_certifications.py --json > cert-audit.json

```

This script validates lesson naming conventions and track consistency within `certifications/claude/`.

### Automate with Git Pre-Push Hooks

Prevent accidental pushes of broken curriculum content by installing a pre-push hook. Create `.git/hooks/pre-push` with the following content and make it executable (`chmod +x .git/hooks/pre-push`):

```bash
#!/usr/bin/env bash

# Run lesson validation before every push

python3 scripts/audit_lessons.py
if [ $? -ne 0 ]; then
  echo "❌ Local audit failed – fix the reported issues before pushing."
  exit 1
fi

```

With this hook active, every `git push` aborts if [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) detects any violations, ensuring the remote repository never receives structurally invalid lessons.

## Summary

- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** serves as the primary entry point for local validation, implementing rules L001 through L009.
- The script requires only Python 3 standard libraries, eliminating environment setup friction.
- Use `--phase` to scope validation to specific directories, and `--json` for machine-readable output.
- Exit code `1` indicates validation failures that would block CI merges.
- Installing a Git pre-push hook automates validation, preventing broken commits from reaching the remote.

## Frequently Asked Questions

### What does audit_lessons.py actually check?

The script validates lesson directories against nine structural rules (L001–L009): directory naming patterns, [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) existence and formatting, code file presence, quiz JSON schema compliance, and internal link integrity. It also verifies that all Markdown links point to existing files within the repository.

### Do I need to install dependencies to run the validation?

No. [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) relies exclusively on Python's standard library. As long as you have Python 3 installed, you can execute the script immediately after cloning the repository without creating a virtual environment or running `pip install`.

### How does local validation differ from CI checks?

There is no difference in the validation logic. The [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) workflow executes the same `python scripts/audit_lessons.py` command you run locally. Local execution simply provides faster feedback by catching errors before the code reaches GitHub Actions.

### Can I run validation on a single lesson instead of an entire phase?

The current implementation filters at the phase level using `--phase`. To validate a specific lesson, you must either temporarily relocate it to its own phase directory or run the full phase audit and filter the output manually. The `iter_lesson_dirs` generator in the source code processes entire phase directories collectively.