How to Run AI Models from ai-engineering-from-scratch Locally: A Complete Setup Guide

You can run AI models from the ai-engineering-from-scratch curriculum locally by cloning the repository, installing dependencies from requirements.txt, and executing individual lesson scripts such as phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py using Python 3.10+.

The rohitg00/ai-engineering-from-scratch repository delivers a comprehensive curriculum containing stand-alone implementations of every algorithm in the AI stack, from foundational linear algebra to full LLM pipelines. Each lesson ships as self-contained code under phases/.../code/ directories, allowing you to run AI models from ai-engineering-from-scratch locally without external API dependencies. This guide walks you through the exact steps to execute these implementations on your machine.

Prerequisites and Environment Setup

Before running any models, ensure your environment meets the baseline requirements. The curriculum requires Python 3.10+ and only uses third-party packages listed in the root requirements.txt, including numpy, torch, zstandard, and safetensors.

Create and activate a virtual environment, then install the dependencies:

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Verify Your Installation with the Curriculum Runner

The repository includes a validation script at scripts/lesson_run.py that discovers all lesson code and verifies integrity. This tool serves as your first sanity check before executing heavy computational workloads.

Run a syntax-only check (fast, no heavy dependencies):

python scripts/lesson_run.py

For a full execution test that runs the entry file of each lesson (skipping those with extra requirements):

python scripts/lesson_run.py --execute

The script recursively walks every phases/**/code/ folder, compiles Python files using py_compile, and when --execute is provided, runs files starting with main..

Running Specific AI Models Locally

Once verified, you can execute individual models by targeting their specific lesson entry points.

Pre-Train Mini-GPT (124M Parameters)

To run the minimal transformer implementation from Phase 10:

python phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py

This script builds a 124M parameter transformer from scratch, generates a synthetic dataset, executes a single training epoch, and prints the loss. According to the source code in phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py, this provides a complete pre-training pipeline without external data downloads.

Optimize Inference

After training, test the model with the inference optimization lesson:

python phases/10-llms-from-scratch/12-inference-optimization/code/main.py \
    --model-path checkpoints/mini_gpt.pt \
    --prompt "The quick brown fox"

This entry point loads the trained checkpoint, tokenizes input, and demonstrates efficient forward pass execution as implemented in phases/10-llms-from-scratch/12-inference-optimization/code/main.py.

Quantize the Model

To reduce model size using INT8 quantization:

python phases/10-llms-from-scratch/11-quantization/code/main.py \
    --model-path checkpoints/mini_gpt.pt \
    --output quantized.pt

The quantization script implements post-training quantization techniques, converting the full-precision model to a compact format suitable for edge deployment.

Install Reusable Artifacts

The curriculum generates reusable prompts, skills, and agent configurations stored as markdown artifacts. Use the scripts/install_skills.py helper to copy these into your target directory:

python scripts/install_skills.py ./my_artifacts --layout skills

This script discovers all outputs/*-*.md files, parses their front-matter, and writes a manifest.json for quick integration into your own projects.

Debug Models with the Agent Workbench

For troubleshooting model failures, navigate to the agent workbench in Phase 14:

python phases/14-agent-engineering/31-agent-workbench-why-models-fail/code/main.py

This minimal Python application loads lesson code, runs structured sanity tests, and prints detailed failure traces to help identify why specific models fail.

Summary

• Clone the rohitg00/ai-engineering-from-scratch repository and install dependencies from requirements.txt using Python 3.10+.
• Verify the curriculum using scripts/lesson_run.py before executing individual models.
• Execute specific lessons by running their main.py files, such as phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py for Mini-GPT training.
• Optimize trained models using the quantization and inference scripts in Phase 10.
• Reuse artifacts via scripts/install_skills.py to incorporate curriculum outputs into external projects.

Frequently Asked Questions

What Python version is required to run the AI models locally?

The curriculum requires Python 3.10 or higher to ensure compatibility with modern torch features and syntax used in the lesson implementations. All dependencies are pinned in requirements.txt to guarantee reproducible execution across environments.

Can I run the models without GPU acceleration?

Yes, all models in the curriculum run on CPU-only environments, though training times will be significantly slower for transformer architectures. The Mini-GPT implementation and other Phase 10 lessons use torch CPU tensors by default unless you explicitly move tensors to CUDA devices in the code.

How do I know which script is the entry point for each lesson?

The curriculum follows a consistent naming convention where the primary executable is always named main.py (or starts with main.). The scripts/lesson_run.py utility specifically looks for these patterns when executing with the --execute flag, making it easy to identify runnable entry points within any phases/**/code/ directory.

Where are the trained model checkpoints saved?

By default, training scripts like phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py save checkpoints to a checkpoints/ directory relative to the repository root. You can customize this path using command-line arguments (e.g., --model-path) when running inference or quantization scripts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}