How AI Engineering Is Implemented in the AI Engineering From Scratch Curriculum

The repository teaches AI engineering through a strict "Build-It / Use-It" methodology where learners first implement algorithms from raw math using only standard libraries, then recreate the same logic using production frameworks like PyTorch or JAX.

The AI Engineering From Scratch repository by rohitg00 is a structured curriculum spanning 20 phases and 503 lessons. It implements AI engineering as a progressive disclosure of complexity, forcing learners to understand the underlying mechanics before touching abstraction layers. Every lesson follows a repeatable architectural pattern that generates reusable artifacts while maintaining pedagogical clarity.

The Build-It/Use-It Pedagogical Pattern

The core implementation philosophy separates every lesson into two distinct phases. Build-It requires implementing the algorithm from first principles using only standard library functions plus a constrained allow-list (numpy, torch, h5py, zstandard, safetensors). Use-It takes the same algorithm and rewrites it using the corresponding production framework, revealing exactly what the library abstracts away.

This pattern appears consistently across the repository. For example, in phases/07-transformers-deep-dive/02-self-attention-from-scratch/, the code/ directory contains both a pure NumPy implementation and a PyTorch nn.MultiheadAttention version. This side-by-side comparison eliminates the "magic" of high-level frameworks by showing the direct transformation from mathematical notation to API calls.

Directory Structure and Lesson Organization

The repository organizes content under phases/<phase-id>-<phase-name>/<lesson-id>-<lesson-name>/, creating a navigable filesystem curriculum. Each lesson folder contains four standardized components:

• docs/en.md – The narrative explanation with YAML front-matter containing metadata
• code/ – Runnable source files implementing both Build-It and Use-It versions
• outputs/ – Generated artifacts including prompts, skills, agents, or MCP servers
• quiz.json – Automated assessment questions validating comprehension

This structure ensures that AI engineering concepts remain isolated and testable. The outputs/ directory is particularly significant because it treats educational content as infrastructure—every lesson produces an installable artifact rather than just documentation.

Implementing Core Algorithms From Scratch

The Build-It implementations demonstrate how AI engineering works at the tensor level. Consider the self-attention mechanism implemented in phases/07-transformers-deep-dive/02-self-attention-from-scratch/code/self_attention.py:

import numpy as np

def self_attention(Q, K, V):
    """Compute attention = softmax(Q·Kᵀ / √d_k) · V"""
    dk = Q.shape[-1]
    scores = Q @ K.T / np.sqrt(dk)           # [seq_len, seq_len]

    weights = np.exp(scores - np.max(scores, axis=-1, keepdims=True))
    weights /= weights.sum(axis=-1, keepdims=True)
    return weights @ V

This implementation uses only NumPy operations to demonstrate the mathematical core: scaled dot-product attention with numerical stability handling. The corresponding Use-It version in the same lesson replaces this with PyTorch's optimized implementation:

import torch.nn as nn
import torch

class SelfAttention(nn.Module):
    def __init__(self, embed_dim, heads):
        super().__init__()
        self.attn = nn.MultiheadAttention(embed_dim, heads)

    def forward(self, x):
        # x: (seq_len, batch, embed_dim)

        attn_output, _ = self.attn(x, x, x)
        return attn_output

Both versions live in the lesson's code/ folder, allowing learners to compare the educational implementation against the production-ready API.

Agent Engineering and Production Patterns

Phase 14 transitions from static algorithms to dynamic AI engineering systems implementing agent loops. The curriculum includes a ReAct-style agent implementation in phases/14-agent-engineering/01-the-agent-loop/code/agent_loop.py that demonstrates how an LLM can generate tool calls, handle execution results, and continue reasoning.

This phase extends beyond simple inference to include:

• Memory systems for context management
• Tool calling schemas and validation
• MCP (Model Context Protocol) server implementations
• Observability hooks for production monitoring

The agent implementations follow the same Build-It/Use-It pattern, first creating a pure-Python reasoning loop, then integrating with production agent frameworks.

Cross-Language Implementation Strategy

True AI engineering requires understanding that algorithms transcend specific languages. The repository maintains parallel implementations across Python, TypeScript, Rust, and Julia for most lessons. This cross-language parity reinforces algorithmic understanding by forcing learners to separate language-specific syntax from mathematical concepts.

The dependency policy remains strict across all languages: only standard libraries plus minimal external dependencies allowed during the Build-It phase. This constraint guarantees that learners understand memory layout, type systems, and computational complexity before relying on garbage collection or automatic differentiation.

Artifact Generation and Reusability

Every lesson generates concrete artifacts in the outputs/ directory. These are not merely example files but installable components managed by scripts/install_skills.py. The script parses the generated skills, prompts, and agent definitions, installing them into the local environment for composition into larger projects.

This artifact-first mindset transforms the curriculum from passive reading into active infrastructure building. Learners accumulate a personal toolkit of vetted implementations that can be composed without re-implementing fundamentals.

Automation and Quality Assurance

The repository implements rigorous CI/CD for educational content through .github/workflows/curriculum.yml. This pipeline:

1. Runs scripts/audit_lessons.py to validate directory structure and required files
2. Executes scripts/check_readme_counts.py to synchronize the README's lesson table with actual content
3. Rebuilds the static site using site/build.js to regenerate site/data.js
4. Executes unit tests in code/tests/ for each lesson

The automation ensures that the repository never stores generated files—the catalog.json and site/data.js are regenerated on every merge, maintaining a single source of truth in the lesson source files.

Summary

• AI engineering in this repository means building from first principles before touching frameworks, implemented through the Build-It/Use-It pattern.
• Lessons follow a strict filesystem structure under phases/ with standardized docs/, code/, outputs/, and quiz.json components.
• Core algorithms like self-attention are implemented in pure NumPy before transitioning to PyTorch nn.MultiheadAttention.
• Phase 14 demonstrates production agent engineering with ReAct loops, memory systems, and MCP servers.
• Cross-language parity across Python, TypeScript, Rust, and Julia reinforces algorithmic understanding.
• Generated artifacts in outputs/ are installable via scripts/install_skills.py, creating reusable infrastructure.
• Continuous integration via .github/workflows/curriculum.yml validates lesson structure and regenerates the static site automatically.

Frequently Asked Questions

What is the Build-It/Use-It pattern in AI Engineering From Scratch?

The Build-It/Use-It pattern is a pedagogical strategy where learners first implement an algorithm using only standard libraries and basic math (Build-It), then reimplement the same functionality using production frameworks like PyTorch or JAX (Use-It). This approach reveals exactly what high-level APIs abstract away, ensuring deep understanding of AI engineering fundamentals before introducing convenience layers.

How does the repository handle agent implementation and tool calling?

Phase 14 of the curriculum focuses specifically on agent engineering, implementing a ReAct-style agent loop in phases/14-agent-engineering/01-the-agent-loop/code/agent_loop.py. This demonstrates how an LLM can generate tool calls, execute them, and incorporate results back into its reasoning context. Subsequent lessons extend this with memory systems, planning algorithms, and MCP (Model Context Protocol) server integration for production deployment.

Can I reuse the code generated from the lessons in real projects?

Yes. Every lesson generates artifacts in the outputs/ directory that are designed for reuse. The repository includes scripts/install_skills.py, which parses these generated artifacts—including prompts, skills, and agent definitions—and installs them into your local environment. This allows you to compose lessons into larger AI engineering workflows without re-implementing the underlying algorithms.

What languages are supported in the AI Engineering From Scratch curriculum?

The curriculum primarily targets Python but maintains cross-language parity with parallel implementations in TypeScript, Rust, and Julia for most lessons. This multilanguage approach ensures that AI engineering concepts are understood as algorithmic truths rather than language-specific syntax, though Python remains the primary language for the Build-It implementations due to its prominence in the AI ecosystem.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}