How to Customize or Extend Existing Lesson Code in AI Engineering from Scratch

To customize or extend existing lesson code in AI Engineering from Scratch, modify the code/main.py file within the specific lesson directory, validate changes using scripts/lesson_run.py, and verify functionality with the included unit test suite.

The AI Engineering from Scratch curriculum is organized as a collection of isolated, self-contained lessons designed for hands-on experimentation. Each lesson resides in its own directory under the phases/ folder, containing runnable implementations, documentation, and automated tests. This modular architecture allows you to safely customize or extend existing lesson code—whether adding new activation functions, adjusting neural network architectures, or experimenting with hyper-parameters—without risking interference with other lessons in the repository.

Lesson Structure and Code Organization

Every lesson follows a strict directory convention: phases/<phase-id>-<phase-name>/<lesson-id>-<lesson-slug>/. Within each lesson folder, you will find three critical sub-directories:

• code/ – Contains the primary implementation files such as main.py, written in Python, TypeScript, Rust, or Julia.
• docs/ – Stores the human-readable lesson narrative in en.md and other localized documentation.
• outputs/ – Holds generated assets including prompts, skills, agents, and MCP servers produced by the lesson implementation.

This separation ensures that source code remains distinct from documentation and generated artefacts, making version control and customization straightforward.

Step-by-Step Workflow for Customizing Lessons

1. Locate the Target Lesson Directory

Identify the specific lesson you want to modify using the directory naming convention. For example, the back-propagation lesson resides at:

phases/03-deep-learning-core/03-backpropagation/

2. Modify the Core Implementation

Open phases/03-deep-learning-core/03-backpropagation/code/main.py to edit the primary implementation. The codebase is intentionally minimal, relying only on the standard library plus small numpy-style primitives shipped with the lesson. Common customizations include:

• Adding functionality – Implement new methods in core classes (e.g., adding a leaky_relu method to the Value class).
• Tuning hyper-parameters – Adjust learning rates, layer dimensions, or regularization strengths directly in the initialization code.
• Extending architecture – Insert additional hidden layers or modify the loss function in the Network class definition.

3. Run the Lesson Locally

Execute your modified code using the repository's helper script to ensure it runs in the correct context:

python scripts/lesson_run.py phases/03-deep-learning-core/03-backpropagation

This script configures the temporary PYTHONPATH and invokes the lesson's entry point, displaying training progress and final results in your terminal.

4. Execute the Unit Test Suite

Every lesson includes a tests/ directory containing five or more unit tests. Validate your changes by running:

python -m unittest discover -s phases/03-deep-learning-core/03-backpropagation/tests -v

If you added new functionality, create corresponding test cases in the lesson's tests/ folder to ensure the CI pipeline remains green.

5. Regenerate Outputs and Update Documentation

If your lesson generates prompts, skills, or agent configurations stored in outputs/, rebuild these assets with the --generate-outputs flag:

python scripts/lesson_run.py --generate-outputs phases/03-deep-learning-core/03-backpropagation

Additionally, update phases/03-deep-learning-core/03-backpropagation/docs/en.md if your changes alter the lesson's fundamental behavior or learning objectives.

6. Commit Changes Using Conventional Commits

Follow the repository's commit convention to maintain curriculum integrity. Use the format feat(<scope>): <description> with a maximum 72-character subject line:

feat(phase-03/03-backpropagation): add leaky-relu activation

Push your branch and open a pull request; the CI pipeline will automatically run audit_lessons.py and readme-counts checks to verify structural compliance.

Practical Extension Examples

Adding a Leaky ReLU Activation Function

Extend the Value class in phases/03-deep-learning-core/03-backpropagation/code/main.py to support Leaky ReLU, which mitigates the "dying ReLU" problem by allowing a small gradient when the unit is inactive:

class Value:
    # ... existing methods ...

    def leaky_relu(self, negative_slope: float = 0.01):
        """Leaky ReLU activation (α·x for x < 0, x for x ≥ 0)."""
        out = Value(self.data if self.data > 0 else negative_slope * self.data,
                    (self,), 'leaky_relu')

        def _backward():
            grad = out.grad * (1 if self.data > 0 else negative_slope)
            self.grad += grad

        out._backward = _backward
        return out

To utilize this activation, modify the Neuron.__call__ method to invoke leaky_relu() instead of the default sigmoid or ReLU:

def __call__(self, x):
    act = sum((wi * xi for wi, xi in zip(self.weights, x)), self.bias)
    return act.leaky_relu()

Extending the Network Architecture with Additional Layers

Modify the Network class constructor to insert an extra hidden layer between existing layers. This example adds a size-6 hidden layer after the first hidden layer:

class Network:
    def __init__(self, sizes):
        extended = []
        for i in range(len(sizes) - 1):
            extended.append(sizes[i])
            # Insert extra layer after the first hidden layer

            if i == 1:
                extended.append(6)
        extended.append(sizes[-1])
        
        self.layers = []
        for i in range(len(extended) - 1):
            self.layers.append(Layer(extended[i], extended[i + 1]))

The network topology now follows [input, hidden-1, new-hidden-2, output] without requiring changes to the training loop.

Implementing Unit Tests for New Features

Create phases/03-deep-learning-core/03-backpropagation/tests/test_activation.py to validate your Leaky ReLU implementation:

import unittest
from phases.03_deep_learning_core.03_backpropagation.code.main import Value

class TestLeakyReLU(unittest.TestCase):
    def test_negative(self):
        v = Value(-2.0)
        out = v.leaky_relu(0.1)
        out.backward()
        self.assertAlmostEqual(out.data, -0.2)   # α·x

        self.assertAlmostEqual(v.grad, 0.1)      # gradient respects α

    def test_positive(self):
        v = Value(3.0)
        out = v.leaky_relu()
        out.backward()
        self.assertAlmostEqual(out.data, 3.0)
        self.assertAlmostEqual(v.grad, 1.0)

if __name__ == '__main__':
    unittest.main()

Run this test suite alongside existing tests to verify backward pass correctness and numerical stability.

Summary

• Lesson Isolation: Each lesson operates as a micro-project under phases/<phase-id>-<phase-name>/<lesson-id>-<lesson-slug>/, preventing cross-contamination between experiments.
• Primary Entry Point: Customize logic by editing code/main.py within your target lesson directory.
• Validation Workflow: Use python scripts/lesson_run.py <lesson-path> for execution and python -m unittest discover -s <lesson-path>/tests for verification.
• Output Regeneration: Rebuild lesson artefacts using the --generate-outputs flag when modifying code that generates prompts or agent skills.
• Dependency Constraints: Limit extensions to the standard library and lesson-provided primitives to avoid framework bloat.
• Version Control: Commit changes using conventional commit prefixes like feat(phase-03/03-backpropagation): to maintain CI/CD compatibility.

Frequently Asked Questions

Where is the main implementation file located for each lesson?

The primary implementation resides in code/main.py within the specific lesson directory. For example, the back-propagation lesson's core logic is located at phases/03-deep-learning-core/03-backpropagation/code/main.py. This file contains the Value class, network definitions, and training loops that you will modify.

How do I run tests after customizing lesson code?

Execute the lesson-specific test suite using Python's built-in unittest discovery command: python -m unittest discover -s phases/03-deep-learning-core/03-backpropagation/tests -v. This command automatically finds and runs all test cases in the lesson's tests/ folder, validating that your customizations maintain expected behavior.

Can I add external dependencies when extending lesson code?

The curriculum is designed to use only the standard library plus small numpy-style primitives shipped with each lesson. You should avoid importing heavy frameworks like TensorFlow or PyTorch in the "build-it" phase of lessons. This constraint ensures that the educational implementation remains transparent and runnable without complex environment setup.

How do I regenerate lesson outputs after code changes?

Run the helper script with the --generate-outputs flag: python scripts/lesson_run.py --generate-outputs phases/03-deep-learning-core/03-backpropagation. This rebuilds prompts, skills, agents, and MCP servers stored in the lesson's outputs/ directory, ensuring that generated artefacts remain synchronized with your code modifications.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}