What Testing Requirements Must a Lesson’s code/tests Directory Meet?

Each lesson in the AI Engineering from Scratch curriculum must contain at least 5 unit tests in its code/tests/ directory, execute with exit code 0 using the language’s standard test runner, and pass validation by both CI pipelines and local audit scripts.

The ai-engineering-from-scratch repository enforces strict quality standards across its lesson structure. According to the AGENTS.md contract—specifically lines 108-112—every lesson must include a robust test suite under its code/tests/ folder to ensure code correctness and educational integrity.

Testing Requirements Defined in AGENTS.md

The canonical requirements are documented in the code/tests subsection of AGENTS.md. These standards apply uniformly across all phases and lessons in the curriculum.

Minimum Test Count

A lesson must contain at least 5 unit tests. This minimum threshold ensures sufficient coverage of the lesson's core functionality. The tests should validate the behavior of the implementation files located in the lesson's code/ directory, such as main.py or equivalent entry points.

Test Execution Standards

Tests must run using the language’s standard-library runner and terminate with exit code 0 to indicate success:

• Python: python3 -m unittest discover
• TypeScript: npx tsx --test
• Rust / Julia: Using the language’s inline test facilities

The test command must be run from within the lesson's code/ directory or configured to discover tests in the code/tests/ subdirectory.

File Placement and Naming Conventions

All test files belong in the code/tests/ folder within the lesson directory. Files should follow the pattern test_*.py, test_*.ts, or the appropriate naming convention for the target language. This structure ensures the discovery command locates and executes all test suites correctly.

Validation and Enforcement Mechanisms

The repository implements automated checks to ensure compliance with these testing requirements.

CI Pipeline Checks

The Per-PR validation section of AGENTS.md mandates that the continuous integration pipeline verify test compliance. Every pull request is checked to confirm that lessons include the required non-empty code/tests/ folder and that the test suite executes successfully with exit code 0.

Local Auditing with scripts/audit_lessons.py

For local development, the scripts/audit_lessons.py script provides immediate feedback. This tool checks for the presence of a non-empty code/tests/ directory and validates that the lesson meets the minimum test count requirement. Running this script before submitting changes ensures the lesson will pass CI checks.

Example Test Implementation

Below is a minimal Python test file that satisfies the "at least 5 unit tests" rule. Place this file at phases/xx-phase-slug/yy-lesson-slug/code/tests/test_main.py:

import unittest
from main import add, mul  # example functions from the lesson implementation

class TestLessonFunctions(unittest.TestCase):
    def test_add_positive(self):
        self.assertEqual(add(2, 3), 5)

    def test_add_negative(self):
        self.assertEqual(add(-1, -1), -2)

    def test_mul_positive(self):
        self.assertEqual(mul(2, 3), 6)

    def test_mul_zero(self):
        self.assertEqual(mul(0, 5), 0)

    def test_mul_negative(self):
        self.assertEqual(mul(-2, 3), -6)

if __name__ == "__main__":
    unittest.main()

Running Tests Locally

To verify your test suite meets requirements before submission, execute the standard runner from the lesson's code/ directory:

cd phases/xx-phase-slug/yy-lesson-slug/code
python3 -m unittest discover tests -v   # should exit with code 0

A successful run displays the test results and terminates with exit code 0, confirming that the code/tests/ directory meets all repository standards.

Summary

• Each lesson must provide at least 5 unit tests in its code/tests/ directory.
• Tests must run with the language's standard runner and exit with code 0.
• Place tests in code/tests/ using naming patterns like test_*.py or test_*.ts.
• Compliance is enforced by the CI pipeline (per AGENTS.md) and the local scripts/audit_lessons.py script.

Frequently Asked Questions

How many tests are required per lesson?

Each lesson must contain at least 5 unit tests according to the AGENTS.md specification at lines 108-112. This minimum ensures adequate coverage of the lesson's learning objectives and implementation details.

What is the required test runner for Python lessons?

Python lessons must use the standard library's unittest framework, executed via python3 -m unittest discover. This ensures compatibility with the CI pipeline and the local audit script.

Where should test files be located within a lesson directory?

All test files must reside in the code/tests/ subdirectory of the lesson. Name files following the pattern test_*.py (or equivalent for other languages) to ensure automatic discovery by the test runners.

How can I validate tests locally before submitting?

Run the scripts/audit_lessons.py script from the repository root. This tool checks for the presence of a non-empty code/tests/ folder and validates the minimum test count, allowing you to fix issues before submitting a pull request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}