# Python Dependency Allowlist in AI Engineering From Scratch: Complete Package Guide

> Learn about Python dependency allowlist in AI Engineering. Discover the stdlib-first approach and essential packages like numpy and torch used in ai-engineering-from-scratch for focused learning.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: complete-package-guide
- Published: 2026-06-21

---

**The `ai-engineering-from-scratch` repository restricts Python lessons to only five third-party packages—`numpy`, `torch`, `h5py`, `zstandard`, and `safetensors`—plus the standard library, enforcing a "stdlib-first" educational philosophy through automated CI audits.**

The curriculum maintains a strict **Python dependency allowlist** to ensure students master fundamental algorithms before relying on high-level abstractions. This policy is documented in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) and enforced by [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), keeping contributions lightweight and focused on core AI engineering concepts.

## The Complete Python Dependency Allowlist

The repository permits only the following third-party packages in lesson implementations:

- **`numpy`** – Numerical arrays and linear algebra operations
- **`torch`** – PyTorch deep-learning framework and tensor operations  
- **`h5py`** – HDF5 file format handling for dataset storage
- **`zstandard`** – Zstandard compression algorithms
- **`safetensors`** – Safe, fast tensor serialization format

Additionally, all modules from the **Python standard library** (`stdlib`) are automatically permitted without restriction.

### Where the Policy is Defined

The official dependency table is located in **[`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md)** at lines 52-55, which serves as the single source of truth for allowed imports. This documentation explicitly forbids popular data science libraries like `pandas`, `scikit-learn`, and `tensorflow` to maintain curriculum consistency.

## Why the Repository Enforces a Strict Allowlist

The allowlist supports the curriculum's **"stdlib-first"** educational philosophy. Students must implement core algorithms using only Python's built-in capabilities before optionally leveraging the permitted scientific libraries.

This constraint ensures:

- Deep understanding of underlying mathematical operations
- Lightweight lesson environments without dependency bloat  
- Consistent code review standards across all submissions

## CI Enforcement and Audit Mechanisms

The repository validates compliance through automated continuous integration. The **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** script runs after merges to detect policy violations, flagging any imports outside the allowed set before pull requests can be approved.

If a lesson attempts to import a forbidden package like `import pandas as pd`, the CI audit fails and contributors must remove the unauthorized dependency before merging.

## Valid Import Patterns and Code Examples

Below are minimal, legal implementations demonstrating each allowed package. These patterns pass the repository's automated checks.

### NumPy for Numerical Operations

```python
import numpy as np

def matmul(a: np.ndarray, b: np.ndarray) -> np.ndarray:
    return np.dot(a, b)

```

### PyTorch for Deep Learning

```python
import torch

def tensor_sum(x: torch.Tensor) -> torch.Tensor:
    return x.sum()

```

### HDF5 File Handling

```python
import h5py

def read_h5(path: str, dataset: str):
    with h5py.File(path, "r") as f:
        return f[dataset][...]

```

### Zstandard Compression

```python
import zstandard as zs

def compress(data: bytes) -> bytes:
    cctx = zs.ZstdCompressor()
    return cctx.compress(data)

```

### Safe Tensor Serialization

```python
import safetensors.torch

def save_tensor(t: torch.Tensor, file: str):
    safetensors.torch.save_file({"tensor": t}, file)

```

### Standard Library Only

```python
from pathlib import Path
import json

def load_config(p: Path) -> dict:
    return json.loads(p.read_text())

```

## Summary

- The **Python dependency allowlist** permits only `numpy`, `torch`, `h5py`, `zstandard`, `safetensors`, and the standard library in lesson code.
- Policy documentation resides in **[`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md)** (lines 52-55), which defines the complete list of authorized third-party packages.
- The **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** CI script automatically validates imports and blocks pull requests containing forbidden dependencies.
- This "stdlib-first" approach ensures students implement core algorithms manually before using high-level scientific libraries.
- Common packages like `pandas`, `scikit-learn`, and `tensorflow` are explicitly prohibited to maintain curriculum focus.

## Frequently Asked Questions

### Can I request additional packages to be added to the Python dependency allowlist?

The allowlist is intentionally restrictive to maintain the curriculum's educational philosophy. While the repository welcomes suggestions, new additions are rare and require justification showing they provide capabilities unavailable in the current allowed set or standard library.

### Why are popular libraries like pandas and scikit-learn excluded from the allowlist?

These libraries abstract away the underlying algorithmic implementations that the curriculum aims to teach. By restricting lessons to `numpy` for numerical operations and `torch` for deep learning, students must manually implement algorithms like matrix multiplication and backpropagation, reinforcing fundamental AI engineering concepts.

### What happens if my pull request includes an import not on the allowlist?

The CI pipeline will fail during the audit phase. The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) script checks for policy violations and flags any unauthorized imports. You must remove the offending dependency and refactor your code to use only allowed packages before the PR can be merged.

### Is the Python standard library completely unrestricted under this policy?

Yes, all modules included in the Python standard library (`stdlib`) are permitted without restriction. The allowlist specifically governs third-party packages that require external installation via `pip`, ensuring that lessons remain feasible in standard Python environments without additional dependencies.