# How to Run AI Models from ai-engineering-from-scratch Locally: A Complete Setup Guide

> Easily run AI models from ai-engineering-from-scratch locally. Follow our guide to clone the repo, install dependencies, and execute Python scripts for hands-on AI.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-19

---

**You can run AI models from the ai-engineering-from-scratch curriculum locally by cloning the repository, installing dependencies from [`requirements.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/requirements.txt), and executing individual lesson scripts such as [`phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py) using Python 3.10+.**

The **rohitg00/ai-engineering-from-scratch** repository delivers a comprehensive curriculum containing stand-alone implementations of every algorithm in the AI stack, from foundational linear algebra to full LLM pipelines. Each lesson ships as self-contained code under `phases/.../code/` directories, allowing you to run AI models from ai-engineering-from-scratch locally without external API dependencies. This guide walks you through the exact steps to execute these implementations on your machine.

## Prerequisites and Environment Setup

Before running any models, ensure your environment meets the baseline requirements. The curriculum requires **Python 3.10+** and only uses third-party packages listed in the root [`requirements.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/requirements.txt), including `numpy`, `torch`, `zstandard`, and `safetensors`.

Create and activate a virtual environment, then install the dependencies:

```bash
git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

```

## Verify Your Installation with the Curriculum Runner

The repository includes a validation script at [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) that discovers all lesson code and verifies integrity. This tool serves as your first sanity check before executing heavy computational workloads.

Run a **syntax-only check** (fast, no heavy dependencies):

```bash
python scripts/lesson_run.py

```

For a **full execution test** that runs the entry file of each lesson (skipping those with extra requirements):

```bash
python scripts/lesson_run.py --execute

```

The script recursively walks every `phases/**/code/` folder, compiles Python files using `py_compile`, and when `--execute` is provided, runs files starting with `main.`.

## Running Specific AI Models Locally

Once verified, you can execute individual models by targeting their specific lesson entry points.

### Pre-Train Mini-GPT (124M Parameters)

To run the minimal transformer implementation from Phase 10:

```bash
python phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py

```

This script builds a 124M parameter transformer from scratch, generates a synthetic dataset, executes a single training epoch, and prints the loss. According to the source code in [`phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py), this provides a complete pre-training pipeline without external data downloads.

### Optimize Inference

After training, test the model with the inference optimization lesson:

```bash
python phases/10-llms-from-scratch/12-inference-optimization/code/main.py \
    --model-path checkpoints/mini_gpt.pt \
    --prompt "The quick brown fox"

```

This entry point loads the trained checkpoint, tokenizes input, and demonstrates efficient forward pass execution as implemented in [`phases/10-llms-from-scratch/12-inference-optimization/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/12-inference-optimization/code/main.py).

### Quantize the Model

To reduce model size using INT8 quantization:

```bash
python phases/10-llms-from-scratch/11-quantization/code/main.py \
    --model-path checkpoints/mini_gpt.pt \
    --output quantized.pt

```

The quantization script implements post-training quantization techniques, converting the full-precision model to a compact format suitable for edge deployment.

## Install Reusable Artifacts

The curriculum generates reusable prompts, skills, and agent configurations stored as markdown artifacts. Use the [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py) helper to copy these into your target directory:

```bash
python scripts/install_skills.py ./my_artifacts --layout skills

```

This script discovers all `outputs/*-*.md` files, parses their front-matter, and writes a [`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json) for quick integration into your own projects.

## Debug Models with the Agent Workbench

For troubleshooting model failures, navigate to the agent workbench in Phase 14:

```bash
python phases/14-agent-engineering/31-agent-workbench-why-models-fail/code/main.py

```

This minimal Python application loads lesson code, runs structured sanity tests, and prints detailed failure traces to help identify why specific models fail.

## Summary

- **Clone** the rohitg00/ai-engineering-from-scratch repository and install dependencies from [`requirements.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/requirements.txt) using Python 3.10+.
- **Verify** the curriculum using [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) before executing individual models.
- **Execute** specific lessons by running their [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) files, such as [`phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py) for Mini-GPT training.
- **Optimize** trained models using the quantization and inference scripts in Phase 10.
- **Reuse** artifacts via [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py) to incorporate curriculum outputs into external projects.

## Frequently Asked Questions

### What Python version is required to run the AI models locally?

The curriculum requires **Python 3.10 or higher** to ensure compatibility with modern `torch` features and syntax used in the lesson implementations. All dependencies are pinned in [`requirements.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/requirements.txt) to guarantee reproducible execution across environments.

### Can I run the models without GPU acceleration?

Yes, all models in the curriculum run on CPU-only environments, though training times will be significantly slower for transformer architectures. The Mini-GPT implementation and other Phase 10 lessons use `torch` CPU tensors by default unless you explicitly move tensors to CUDA devices in the code.

### How do I know which script is the entry point for each lesson?

The curriculum follows a consistent naming convention where the primary executable is always named [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) (or starts with `main.`). The [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) utility specifically looks for these patterns when executing with the `--execute` flag, making it easy to identify runnable entry points within any `phases/**/code/` directory.

### Where are the trained model checkpoints saved?

By default, training scripts like [`phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py) save checkpoints to a `checkpoints/` directory relative to the repository root. You can customize this path using command-line arguments (e.g., `--model-path`) when running inference or quantization scripts.