# Allowed Dependencies for Python in AI Engineering from Scratch: The Complete Allowlist

> Master AI engineering fundamentals by exploring the complete allowlist of five core Python dependencies for the ai-engineering-from-scratch project. Learn without high-level abstractions.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: getting-started
- Published: 2026-08-26

---

**The AI Engineering from Scratch curriculum restricts Python lessons to five specific third-party libraries—`numpy`, `torch`, `h5py`, `zstandard`, and `safetensors`—plus the standard library to ensure learners master fundamental algorithms without relying on high-level abstractions.**

The `rohitg00/ai-engineering-from-scratch` repository maintains a strict dependency allowlist documented in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md). This policy forces hands-on implementation of core AI concepts like back-propagation and tokenization rather than depending on pre-built frameworks that obscure underlying mechanics.

## The Official Python Dependency Allowlist

According to the source code in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) (lines 55-60), the project permits only the following external packages for Python lesson implementations:

- **`numpy`** – Numerical array operations and linear algebra foundations
- **`torch`** – PyTorch for tensor computations and automatic differentiation  
- **`h5py`** – HDF5 file I/O for efficient handling of large datasets
- **`zstandard`** – Fast compression and decompression of binary data streams
- **`safetensors`** – Safe, zero-copy tensor serialization format

All other functionality must use Python's built-in standard library. This constraint ensures that every lesson in the curriculum remains self-contained and pedagogically focused.

## Why Restrict Dependencies?

The dependency policy serves three critical educational purposes:

**Eliminates Hidden Complexity** – By banning high-level frameworks like `pandas` or `transformers`, learners must implement algorithms from scratch, understanding back-propagation, attention mechanisms, and tokenization at the mathematical level.

**Ensures Reproducibility** – Minimal dependencies guarantee that code runs identically across Linux, macOS, and Windows environments without version conflicts or system-specific package issues.

**Enforces Curriculum Integrity** – The continuous integration pipeline validates every contribution against the allowlist, preventing accidental introduction of external dependencies that would break the pedagogical model.

## Working with Allowed Dependencies: Code Examples

Each permitted library addresses specific technical needs in AI engineering. Below are canonical implementations showing proper usage within the project's constraints.

### NumPy for Vector Operations

Use `numpy` for all numerical array manipulations and linear algebra:

```python
import numpy as np

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
dot_product = np.dot(a, b)
print("Dot product:", dot_product)

```

### PyTorch for Automatic Differentiation

The `torch` library provides tensor computation with gradient tracking:

```python
import torch

x = torch.tensor([2.0, 3.0], requires_grad=True)
y = x ** 2 + 5
loss = y.mean()
loss.backward()
print("Gradients:", x.grad)

```

### h5py for Dataset Persistence

Store large NumPy arrays efficiently using HDF5 format:

```python
import h5py
import numpy as np

data = np.random.rand(10, 10)
with h5py.File('example.h5', 'w') as f:
    f.create_dataset('random_matrix', data=data)

```

### zstandard for Data Compression

Compress binary data streams when preprocessing large corpora:

```python
import zstandard as zstd

original = b"The quick brown fox jumps over the lazy dog" * 10
cctx = zstd.ZstdCompressor()
compressed = cctx.compress(original)
dctx = zstd.ZstdDecompressor()
decompressed = dctx.decompress(compressed)
assert original == decompressed

```

### safetensors for Model Serialization

Safely save and load PyTorch tensors without pickle vulnerabilities:

```python
import torch
from safetensors.torch import save_file, load_file

tensor = torch.randn(3, 3)
save_file({'weight': tensor}, 'model.safetensors')
loaded = load_file('model.safetensors')
print("Loaded tensor shape:", loaded['weight'].shape)

```

## Dependency Enforcement and Validation

The project automates compliance checks through [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py). This CI script parses every Python file in `phases/*/*/code/main.py` and validates imports against the allowlist during pull request reviews.

If a lesson imports `pandas`, `scikit-learn`, or any non-allowed package, the CI audit rejects the change with a specific error citing the offending import and referencing the [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) dependency table.

## Summary

- The AI Engineering from Scratch project permits exactly five third-party Python packages: `numpy`, `torch`, `h5py`, `zstandard`, and `safetensors`
- All lesson code must use these specific libraries alongside Python's standard library
- The policy is codified in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) and enforced automatically via [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)
- These restrictions ensure pedagogical clarity by forcing manual implementation of AI fundamentals
- Code examples demonstrate proper import patterns for numerical computing, deep learning, data storage, compression, and safe serialization

## Frequently Asked Questions

### Can I use pandas or scikit-learn in my lesson submissions?

No. The [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) allowlist explicitly excludes `pandas`, `scikit-learn`, and similar high-level libraries. Any submission importing these packages fails the CI audit in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py). You must implement data manipulation using only `numpy` and Python built-ins.

### Is the Python standard library fully available within the project?

Yes. While third-party dependencies are restricted to the five allowed packages, you have complete access to Python's standard library including `json`, `pathlib`, `collections`, `itertools`, and `math`. These modules provide sufficient functionality for file I/O, data structures, and mathematical operations without external dependencies.

### How do I request adding a new dependency to the allowlist?

The curriculum maintains a frozen allowlist to preserve educational integrity. Exceptions are rarely granted and require demonstrating that the library addresses a fundamental capability gap impossible to implement manually. Submit a proposal to the repository maintainers citing the specific technical limitation and pedagogical justification, though proposals for convenience libraries are typically rejected.

### What happens if my code uses a non-allowed dependency?

The continuous integration pipeline running [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) automatically blocks pull requests containing unauthorized imports. The build fails with an error message identifying the disallowed package and referencing the [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) dependency section. You must refactor the code to use only allowed dependencies or standard library equivalents before merging.