Python Dependency Allowlist in AI Engineering From Scratch: Complete Package Guide

The ai-engineering-from-scratch repository restricts Python lessons to only five third-party packages—numpy, torch, h5py, zstandard, and safetensors—plus the standard library, enforcing a "stdlib-first" educational philosophy through automated CI audits.

The curriculum maintains a strict Python dependency allowlist to ensure students master fundamental algorithms before relying on high-level abstractions. This policy is documented in AGENTS.md and enforced by scripts/audit_lessons.py, keeping contributions lightweight and focused on core AI engineering concepts.

The Complete Python Dependency Allowlist

The repository permits only the following third-party packages in lesson implementations:

• numpy – Numerical arrays and linear algebra operations
• torch – PyTorch deep-learning framework and tensor operations
• h5py – HDF5 file format handling for dataset storage
• zstandard – Zstandard compression algorithms
• safetensors – Safe, fast tensor serialization format

Additionally, all modules from the Python standard library (stdlib) are automatically permitted without restriction.

Where the Policy is Defined

The official dependency table is located in AGENTS.md at lines 52-55, which serves as the single source of truth for allowed imports. This documentation explicitly forbids popular data science libraries like pandas, scikit-learn, and tensorflow to maintain curriculum consistency.

Why the Repository Enforces a Strict Allowlist

The allowlist supports the curriculum's "stdlib-first" educational philosophy. Students must implement core algorithms using only Python's built-in capabilities before optionally leveraging the permitted scientific libraries.

This constraint ensures:

• Deep understanding of underlying mathematical operations
• Lightweight lesson environments without dependency bloat
• Consistent code review standards across all submissions

CI Enforcement and Audit Mechanisms

The repository validates compliance through automated continuous integration. The scripts/audit_lessons.py script runs after merges to detect policy violations, flagging any imports outside the allowed set before pull requests can be approved.

If a lesson attempts to import a forbidden package like import pandas as pd, the CI audit fails and contributors must remove the unauthorized dependency before merging.

Valid Import Patterns and Code Examples

Below are minimal, legal implementations demonstrating each allowed package. These patterns pass the repository's automated checks.

NumPy for Numerical Operations

import numpy as np

def matmul(a: np.ndarray, b: np.ndarray) -> np.ndarray:
    return np.dot(a, b)

PyTorch for Deep Learning

import torch

def tensor_sum(x: torch.Tensor) -> torch.Tensor:
    return x.sum()

HDF5 File Handling

import h5py

def read_h5(path: str, dataset: str):
    with h5py.File(path, "r") as f:
        return f[dataset][...]

Zstandard Compression

import zstandard as zs

def compress(data: bytes) -> bytes:
    cctx = zs.ZstdCompressor()
    return cctx.compress(data)

Safe Tensor Serialization

import safetensors.torch

def save_tensor(t: torch.Tensor, file: str):
    safetensors.torch.save_file({"tensor": t}, file)

Standard Library Only

from pathlib import Path
import json

def load_config(p: Path) -> dict:
    return json.loads(p.read_text())

Summary

• The Python dependency allowlist permits only numpy, torch, h5py, zstandard, safetensors, and the standard library in lesson code.
• Policy documentation resides in AGENTS.md (lines 52-55), which defines the complete list of authorized third-party packages.
• The scripts/audit_lessons.py CI script automatically validates imports and blocks pull requests containing forbidden dependencies.
• This "stdlib-first" approach ensures students implement core algorithms manually before using high-level scientific libraries.
• Common packages like pandas, scikit-learn, and tensorflow are explicitly prohibited to maintain curriculum focus.

Frequently Asked Questions

Can I request additional packages to be added to the Python dependency allowlist?

The allowlist is intentionally restrictive to maintain the curriculum's educational philosophy. While the repository welcomes suggestions, new additions are rare and require justification showing they provide capabilities unavailable in the current allowed set or standard library.

Why are popular libraries like pandas and scikit-learn excluded from the allowlist?

These libraries abstract away the underlying algorithmic implementations that the curriculum aims to teach. By restricting lessons to numpy for numerical operations and torch for deep learning, students must manually implement algorithms like matrix multiplication and backpropagation, reinforcing fundamental AI engineering concepts.

What happens if my pull request includes an import not on the allowlist?

The CI pipeline will fail during the audit phase. The scripts/audit_lessons.py script checks for policy violations and flags any unauthorized imports. You must remove the offending dependency and refactor your code to use only allowed packages before the PR can be merged.

Is the Python standard library completely unrestricted under this policy?

Yes, all modules included in the Python standard library (stdlib) are permitted without restriction. The allowlist specifically governs third-party packages that require external installation via pip, ensuring that lessons remain feasible in standard Python environments without additional dependencies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}