Dependency Allowlist Policy and Stdlib-First Approach in AI Engineering From Scratch

The ai-engineering-from-scratch curriculum enforces a strict dependency allowlist that restricts lessons to language standard libraries and a minimal, vetted set of packages (such as NumPy, Torch, Hono, and Zod) to ensure educational clarity, portability, and reduced cognitive load for learners.

The rohitg00/ai-engineering-from-scratch repository follows a rigorous dependency allowlist policy designed to keep algorithmic lessons focused on fundamental concepts rather than third-party abstractions. This stdlib-first approach ensures that every code example remains portable, transparent, and accessible to beginners while preventing dependency creep across the curriculum.

What Is the Dependency Allowlist Policy?

The dependency allowlist is a curated registry of permitted packages defined in the repository’s AGENTS.md file. Each lesson must import only from the language’s standard library or from the explicitly allowed third-party packages. Any attempt to import a disallowed dependency results in an automatic rejection by the CI pipeline.

According to AGENTS.md lines 55-63, the permitted dependencies are:

• Python: numpy, torch, h5py, zstandard, safetensors, plus the stdlib
• TypeScript: hono, zod, ws (WebSocket only), @hono/node-server, plus Node 20+ stdlib
• Rust: stdlib only (compiled with rustc --edition 2021 as single-file scripts)
• Julia: Random, Statistics, LinearAlgebra, Printf (Julia stdlib modules)

This whitelist is deliberately narrow. If a contributor requires functionality not available through these channels, they must either implement it manually or provide a rigorous justification for adding the package to the allowlist.

Why a Stdlib-First Approach Matters

Educational Clarity

Lessons in this repository aim to teach how algorithms work from the ground up. Relying on heavy external libraries obscures the underlying mathematics and implementation details. By using only the standard library (or a minimal, vetted set of packages), learners can trace every step of the computation without navigating complex abstraction layers.

Portability and Consistency

The curriculum must compile and run on the CI infrastructure used by every contributor. Standard libraries are guaranteed to be present on the supported runtimes, eliminating version-skew problems that arise with optional dependencies. This ensures that code examples in phases/*/code/ directories execute identically across Linux, macOS, and Windows environments.

Reduced Cognitive Load

Beginners can focus on one new concept at a time. Introducing a new third-party API would require learning its surface area, installation quirks, and potential deprecations, distracting from the core lesson objectives. The stdlib-first mandate keeps the learning curve shallow by leveraging APIs that are permanently available and well-documented in the host language.

Future-Proofing and Dependency Creep

The allowlist guards the repository against accidental "dependency creep," where lessons drift toward full-stack applications rather than focused algorithmic exposition. As noted in AGENTS.md lines 64-65, if a new package becomes essential, it must be evaluated for educational value and added deliberately rather than imported ad hoc.

How the Allowlist Is Enforced

The repository uses scripts/audit_lessons.py to validate every contribution. This CI script scans lesson files for import statements and cross-references them against the allowlist. When a contribution tries to import a disallowed package—such as pandas in Python or lodash in TypeScript—the script automatically rejects the change and prompts the contributor to remove the dependency or justify its inclusion.

Practical Examples: Allowed vs. Disallowed Imports

The following examples demonstrate compliant and non-compliant code patterns across the supported languages.

Python Examples

Correct usage with an allowed dependency (numpy):

import numpy as np

def softmax(x: np.ndarray) -> np.ndarray:
    """Compute the softmax of a vector using NumPy."""
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum()

Incorrect usage attempting to import a banned package (pandas):

import pandas as pd  # ❌ pandas is not on the allowlist; CI will reject this

TypeScript Examples

Correct usage with an allowed dependency (zod):

import { z } from "zod";

export const Point = z.object({
  x: z.number(),
  y: z.number(),
});

Incorrect usage importing an unapproved library (lodash):

import _ from "lodash";  // ❌ lodash is not permitted by the allowlist

Rust Examples

Correct usage relying only on the standard library:

fn factorial(n: u64) -> u64 {
    (1..=n).product()
}

Incorrect usage pulling in an external crate (rand) without justification:

extern crate rand; // ❌ Randomness is provided by stdlib; external crate not allowed

Summary

• The dependency allowlist policy restricts lessons to specific vetted packages (NumPy, Torch, Hono, Zod) plus language standard libraries, as defined in AGENTS.md.
• A stdlib-first approach ensures educational transparency by exposing algorithmic implementation details rather than hiding them behind third-party abstractions.
• The scripts/audit_lessons.py CI script automatically enforces these rules, rejecting pull requests that import disallowed dependencies.
• This policy prevents dependency creep, maintains cross-platform portability, and reduces cognitive load for beginners learning AI engineering fundamentals from scratch.

Frequently Asked Questions

What happens if I need a package that is not on the allowlist?

You have two options: implement the required functionality using only the standard library or allowed packages, or submit a pull request modifying AGENTS.md with a detailed justification explaining the educational necessity of the new dependency. The maintainers will evaluate whether the package aligns with the stdlib-first philosophy before merging.

Why is pandas banned but NumPy is allowed in Python lessons?

NumPy is permitted because it provides essential n-dimensional array operations that are foundational to neural network implementations, while remaining relatively close to the metal. Pandas introduces high-level data frame abstractions that obscure the underlying tensor manipulations the curriculum aims to teach, and its API adds significant cognitive overhead for beginners.

Can I use external crates in Rust lessons if I compile with Cargo?

No. According to the repository rules in AGENTS.md, Rust lessons must compile as single-file scripts using rustc --edition 2021 with stdlib only. This constraint ensures that learners see raw implementations of algorithms without relying on ecosystem crates that might hide allocation strategies or optimization details.

Does the allowlist apply to test files and utility scripts?

Yes. The audit_lessons.py script enforces the policy across the entire phases/ directory structure. All code committed to the repository, including tests and utilities, must adhere to the dependency restrictions to maintain consistency and ensure that every file in the curriculum remains executable in the standardized CI environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}