What Is the Stdlib‑First Philosophy and Dependency Allowlist in AI Engineering from Scratch?

The stdlib‑first philosophy mandates that every lesson in the AI Engineering from Scratch curriculum be implementable using only the language’s standard library, while a curated dependency allowlist permits external packages only when the standard library cannot express the required functionality. This design principle, enforced throughout the rohitg00/ai-engineering-from-scratch repository, eliminates hidden abstractions and ensures that learners comprehend every line of code they execute.

The Stdlib‑First Philosophy Explained

The stdlib‑first rule is codified in the repository’s AGENTS.md file, which explicitly states: “Dependency allowlist (see Dependencies below). Stdlib‑first.” (lines 45–47). This policy requires contributors to attempt a pure standard library implementation before introducing any third‑party dependency.

Core Educational Benefits

Adherence to this philosophy serves three primary pedagogical goals:

1. Pedagogical Clarity – Learners see every implementation detail without “magic” from hidden library internals, ensuring they understand exactly what each component does.
2. Portability – Pure‑stdlib code runs on any environment shipping the language runtime, making lessons reproducible across minimal containers, cloud VMs, and local machines without dependency hell.
3. Determinism for Evaluation – Automated grading and benchmarking avoid version skew or external dependency failures that cause flaky tests, as the code relies solely on the stable runtime distribution.

The Dependency Allowlist Strategy

To enforce the stdlib‑first rule while acknowledging practical constraints, the repository maintains a strict dependency allowlist defined in AGENTS.md (lines 50–58). Each language has a minimal set of approved external packages that may be used only when the standard library is insufficient.

Language‑Specific Allowlists

The permitted dependencies for each supported language are:

• Python: numpy, torch, h5py, zstandard, safetensors (plus the Python standard library)
• TypeScript: hono, zod, ws (only when WebSockets are required), @hono/node-server (plus Node.js 20+ standard library)
• Rust: Standard library only (single‑file compilation via rustc --edition 2021)
• Julia: Random, Statistics, LinearAlgebra, Printf (these are actually Julia standard library modules, effectively allowing only the stdlib)

Enforcement and Justification

If a lesson would require a banned dependency, the author must skip it and provide a clear justification such as “stays stdlib‑first for educational clarity” (line 59). This enforcement mechanism ensures contributors first exhaust standard library capabilities before falling back to the approved packages for complex algorithms like heavy numeric kernels.

Real‑World Implementation Examples

The following code snippets demonstrate the stdlib‑first constraint in production lessons from the curriculum.

Pure‑Stdlib Tokenizer Implementation

In phases/01-math-foundations/21-graph-theory/code/main.py, the curriculum implements a trivial whitespace tokenizer using only Python’s built‑in string methods, avoiding external libraries like regex or nltk:

# phases/01-math-foundations/21-graph-theory/code/main.py

# ------------------------------------------------------

# Implements a trivial whitespace tokenizer using only Python's stdlib.

# No external libraries such as `regex` or `nltk` are used.

#
def simple_tokenizer(text: str) -> list[str]:
    """Split `text` on whitespace and return a list of tokens."""
    return text.strip().split()

if __name__ == "__main__":
    sample = "The quick brown fox jumps over the lazy dog."
    print(simple_tokenizer(sample))

The lesson’s documentation explicitly marks the language as “Python (stdlib)” (phases/01-math-foundations/21-graph-theory/docs/en.md, line 6), confirming the exclusion of third‑party packages.

Stdlib‑Only HTTP Server for A2A Protocol

For the multi‑agent A2A protocol demonstration, the curriculum avoids web frameworks like FastAPI or Flask. Instead, phases/16-multi-agent-and-swarms/12-a2a-protocol/code/main.py uses only http.server and json from the standard library:

# phases/16-multi-agent-and-swarms/12-a2a-protocol/code/main.py

# -----------------------------------------------------------

# Demonstrates a minimal HTTP server using only the stdlib `http.server`.

# No external web frameworks (e.g., FastAPI) are required.

#
import json
from http.server import BaseHTTPRequestHandler, HTTPServer

class EchoHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get('Content‑Length', 0))
        payload = self.rfile.read(length).decode()
        response = {"echo": json.loads(payload)}
        self.send_response(200)
        self.send_header("Content‑Type", "application/json")
        self.end_headers()
        self.wfile.write(json.dumps(response).encode())

if __name__ == "__main__":
    server = HTTPServer(("localhost", 8000), EchoHandler)
    print("Running stdlib HTTP server on http://localhost:8000")
    server.serve_forever()

The lesson’s front‑matter lists “Python (stdlib, http.server, json)” (phases/16-multi-agent-and-swarms/12-a2a-protocol/docs/en.md, line 6), demonstrating a full‑stack example that executes with the command python3 main.py and satisfies the repository’s code contract (exit 0, self‑terminating demo) without any external baggage.

Summary

• The stdlib‑first philosophy requires every lesson to be implementable using only the language’s standard library, as documented in AGENTS.md (lines 45–47).
• A dependency allowlist restricts external packages to a minimal set per language (e.g., numpy and torch for Python; stdlib‑only for Rust), ensuring pedagogical clarity and runtime portability.
• Contributors must skip lessons requiring banned dependencies or justify why the lesson stays stdlib‑first (line 59).
• Real implementations, such as the tokenizer in phases/01-math-foundations/21-graph-theory and the HTTP server in phases/16-multi-agent-and-swarms/12-a2a-protocol, prove that complex AI engineering concepts can be taught without third‑party frameworks.

Frequently Asked Questions

What is the stdlib‑first philosophy in AI Engineering from Scratch?

The stdlib‑first philosophy is a hard constraint requiring that all lesson code be implementable using only the language’s built‑in standard library. This ensures learners understand every line of execution without hidden abstractions from third‑party packages, while maximizing portability across different computing environments.

Which external packages are allowed in the Python allowlist?

According to AGENTS.md (lines 50–58), the Python dependency allowlist permits numpy, torch, h5py, zstandard, and safetensors in addition to the standard library. These packages are approved only for algorithms requiring heavy numeric kernels that would be impractical to write from scratch, such as tensor operations or compression codecs.

How does the dependency allowlist handle Rust implementations?

Rust lessons must use stdlib only and compile as a single file using rustc --edition 2021. No external crates are permitted in the Rust allowlist, forcing contributors to demonstrate systems programming and AI concepts using only the language’s built‑in capabilities and standard library modules.

Why does the curriculum enforce stdlib‑first for automated grading?

By eliminating external dependencies, the stdlib‑first rule removes version skew and network‑related dependency failures that cause flaky tests. This creates determinism for evaluation, ensuring that automated grading scripts and benchmarking suites run reliably across minimal containers and different runtime versions without requiring complex package management.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}