How to Run Local Validation Scripts Like audit_lessons.py Before Pushing

Run python3 scripts/audit_lessons.py from the repository root to validate lesson structure, documentation, and quiz schemas locally before pushing, ensuring your changes pass the same checks enforced by CI.

The rohitg00/ai-engineering-from-scratch repository maintains curriculum integrity through deterministic binary-level validation scripts that check every lesson before it reaches the CI pipeline. Running local validation scripts like audit_lessons.py before pushing catches structural violations immediately and prevents failed GitHub Actions workflows. Because the validator uses only Python standard libraries, you can execute comprehensive checks without installing additional dependencies.

How the Validation Engine Works

Entry Point and Command-Line Interface

The primary validation script resides at scripts/audit_lessons.py. When invoked, it parses command-line flags—including --phase, --json, and --strict—and instantiates an Audit object to track violations across the curriculum. The script is entirely self-contained, requiring no external packages beyond a standard Python 3 interpreter.

Lesson Discovery

The iter_lesson_dirs function walks the phases/ directory tree, optionally filtering by a phase number passed via --phase, and yields each lesson directory for inspection. This ensures that validation scales from individual phases to the entire repository without manual file globbing.

Rule Checks (L001–L009)

For every discovered lesson, the script executes a series of deterministic rule functions:

• check_lesson_dir_pattern (L001): Validates that the directory name follows the required naming convention.
• check_docs_en_md (L002–L004): Verifies that docs/en.md exists, is valid UTF-8, meets minimum size requirements, and contains a top-level H1 heading.
• check_code_main (L005): Guarantees that the code/ subdirectory contains at least one non-ignored file.
• check_quiz (L006–L009): Validates JSON schema compliance, ensures correct option counts, and checks that correct indices point to valid choices.
• check_internal_links: Confirms that every Markdown link references an existing file within the repository boundary.

Each violation is recorded against its rule identifier in the Audit instance for final reporting.

Reporting and Exit Codes

After processing all lessons, the script renders results through either render_report (human-readable) or a JSON emitter (--json). The exit code is 0 for a clean run and 1 if any issues are discovered, making it suitable for scripting and CI gates.

CI Integration

The .github/workflows/curriculum.yml workflow executes python scripts/audit_lessons.py on every pull request. Because the CI environment runs the same code you run locally, local validation guarantees that merges will not fail due to structural errors.

Running audit_lessons.py Locally

Validate All Phases

Execute the following from the repository root to audit every lesson across all phases:

python3 scripts/audit_lessons.py

This produces a human-readable report of any L001–L009 violations. An exit status of 0 confirms the curriculum structure is valid.

Target a Specific Phase

To limit validation to a single phase—useful when iterating on new content—pass the --phase flag with the phase number:

python3 scripts/audit_lessons.py --phase 14

This restricts the audit to phases/14-agent-engineering/, reducing execution time and focusing output on the work in progress.

Generate JSON Reports for Automation

For integration with custom dashboards or CI parsers, output the results as JSON:

python3 scripts/audit_lessons.py --json > audit-report.json

The resulting document contains lessons_checked and an array of issues with file paths and rule identifiers, enabling automated triage.

Enable Strict Mode

To future-proof your validation against upcoming stricter policies, include the --strict flag:

python3 scripts/audit_lessons.py --strict

Currently, --strict behaves identically to the default mode, but it signals intent to adopt stricter validation as the curriculum evolves.

Validate Certification Curricula

The repository includes a separate validator for certification tracks. Run the certification audit to check Claude-specific curriculum rules:

python3 scripts/audit_certifications.py --json > cert-audit.json

This script validates lesson naming conventions and track consistency within certifications/claude/.

Automate with Git Pre-Push Hooks

Prevent accidental pushes of broken curriculum content by installing a pre-push hook. Create .git/hooks/pre-push with the following content and make it executable (chmod +x .git/hooks/pre-push):

#!/usr/bin/env bash

# Run lesson validation before every push

python3 scripts/audit_lessons.py
if [ $? -ne 0 ]; then
  echo "❌ Local audit failed – fix the reported issues before pushing."
  exit 1
fi

With this hook active, every git push aborts if audit_lessons.py detects any violations, ensuring the remote repository never receives structurally invalid lessons.

Summary

• scripts/audit_lessons.py serves as the primary entry point for local validation, implementing rules L001 through L009.
• The script requires only Python 3 standard libraries, eliminating environment setup friction.
• Use --phase to scope validation to specific directories, and --json for machine-readable output.
• Exit code 1 indicates validation failures that would block CI merges.
• Installing a Git pre-push hook automates validation, preventing broken commits from reaching the remote.

Frequently Asked Questions

What does audit_lessons.py actually check?

The script validates lesson directories against nine structural rules (L001–L009): directory naming patterns, docs/en.md existence and formatting, code file presence, quiz JSON schema compliance, and internal link integrity. It also verifies that all Markdown links point to existing files within the repository.

Do I need to install dependencies to run the validation?

No. audit_lessons.py relies exclusively on Python's standard library. As long as you have Python 3 installed, you can execute the script immediately after cloning the repository without creating a virtual environment or running pip install.

How does local validation differ from CI checks?

There is no difference in the validation logic. The .github/workflows/curriculum.yml workflow executes the same python scripts/audit_lessons.py command you run locally. Local execution simply provides faster feedback by catching errors before the code reaches GitHub Actions.

Can I run validation on a single lesson instead of an entire phase?

The current implementation filters at the phase level using --phase. To validate a specific lesson, you must either temporarily relocate it to its own phase directory or run the full phase audit and filter the output manually. The iter_lesson_dirs generator in the source code processes entire phase directories collectively.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}