# LLM Engineering Phases 9-12: From Reinforcement Learning to Multimodal AI > Explore LLM Engineering phases 9-12 in the ai-engineering-from-scratch repo: Reinforcement Learning, LLMs from Scratch, LLM Engineering, and Multimodal AI. Master advanced concepts with practical code. - Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch) - Tags: deep-dive - Published: 2026-08-26 --- **The rohitg00/ai-engineering-from-scratch repository structures its advanced curriculum into four consecutive LLM Engineering phases—Phase 9 (Reinforcement Learning), Phase 10 (LLMs from Scratch), Phase 11 (LLM Engineering), and Phase 12 (Multimodal AI)—each located in the `phases/` directory with dedicated README files and runnable code entry points.** These LLM Engineering phases bridge the gap between theoretical foundations and production deployment, guiding practitioners from Markov Decision Processes to multimodal system architecture. According to the repository source code, each phase builds upon the previous to create a complete pipeline for designing, training, and deploying large language model systems. ## Phase 9 – Reinforcement Learning: Foundation of RLHF Phase 9 establishes the mathematical underpinnings necessary for modern alignment techniques like RLHF (Reinforcement Learning from Human Feedback). Located in [`phases/09-reinforcement-learning/README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/09-reinforcement-learning/README.md), this phase focuses on **Markov Decision Processes**, value functions, and policy evaluation. ### Core Concepts and Implementation The curriculum emphasizes agents that learn by interaction. You will implement **MDPs** (Markov Decision Processes), compute value grids, and analyze how policy quality relates to expected return. These concepts form the bedrock for understanding how language models are fine-tuned using reinforcement learning signals. The primary entry point for this phase demonstrates state-action-reward loops: ```bash python3 phases/09-reinforcement-learning/01-mdps-states-actions-rewards/code/main.py ``` Running this script initializes the reinforcement learning environment and executes the foundational algorithms implemented in the repository. ## Phase 10 – LLMs from Scratch: Tokenization to Training Phase 10 transitions from theoretical RL to practical language model construction. As documented in [`phases/10-llms-from-scratch/README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/README.md), this phase covers **tokenization**, data preprocessing, model architecture design, and training loops for tiny LLMs. ### Building the Tokenizer and Model You will write a **tokenizer from the ground up**, construct the neural architecture, and train a miniature LLM. The phase also includes analysis of compression ratios and data efficiency metrics critical for understanding modern transformer scaling laws. Execute the tokenizer implementation using: ```bash python3 phases/10-llms-from-scratch/01-tokenizers/code/main.py ``` This script in [`phases/10-llms-from-scratch/01-tokenizers/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/01-tokenizers/code/main.py) serves as the gateway to understanding how raw text transforms into model-ready tensors. ## Phase 11 – LLM Engineering: Production Deployment Phase 11, detailed in [`phases/11-llm-engineering/README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/README.md), focuses specifically on **LLM Engineering**—the discipline of moving from pretrained checkpoints to production-ready services. This phase covers prompt engineering strategies, fine-tuning methodologies, inference scaling techniques, and deployment patterns. ### Prompt Engineering and Fine-tuning The curriculum teaches structured approaches to **prompt engineering** and supervised fine-tuning (SFT). You will evaluate different prompt strategies for accuracy and latency, then implement parameter-efficient fine-tuning on pretrained models to adapt them for specific downstream tasks. ### Inference Scaling and API Deployment Practical deployment skills include creating **API-served inference services** and optimizing batch sizes for throughput. The phase addresses the engineering challenges of serving large models at scale, including model sharding and request queuing strategies. Start the prompt engineering exercises with: ```bash python3 phases/11-llm-engineering/01-prompt-engineering/code/main.py ``` This entry point in [`phases/11-llm-engineering/01-prompt-engineering/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/01-prompt-engineering/code/main.py) demonstrates the core inference patterns used throughout the phase. ## Phase 12 – Multimodal AI: Cross-Modal Integration Phase 12 culminates the curriculum with **Multimodal AI**, located in [`phases/12-multimodal-ai/README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/12-multimodal-ai/README.md). This phase integrates text, vision, and audio into unified model architectures using multimodal embeddings and cross-modal attention mechanisms. ### Multimodal Architectures You will build a **multimodal encoder-decoder** capable of processing heterogeneous data types. The curriculum covers zero-shot image-captioning tasks and audio-text alignment techniques, teaching how to align embedding spaces across different modalities. The practical implementation begins with: ```bash python3 phases/12-multimodal-ai/01-multimodal-embeddings/code/main.py ``` Located at [`phases/12-multimodal-ai/01-multimodal-embeddings/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/12-multimodal-ai/01-multimodal-embeddings/code/main.py), this script initializes the multimodal embedding layers that bridge vision and language representations. ## Repository Navigation and Execution All four LLM Engineering phases follow a consistent directory structure within the `phases/` folder. Each phase contains a [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) file explaining theoretical concepts and a `code/` subdirectory containing runnable implementations. To execute the complete curriculum sequentially: 1. **Phase 9**: Run [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) in `01-mdps-states-actions-rewards/code/` to master reinforcement learning foundations 2. **Phase 10**: Execute the tokenizer script in `01-tokenizers/code/` to understand LLM internals 3. **Phase 11**: Launch the prompt engineering module in `01-prompt-engineering/code/` for deployment skills 4. **Phase 12**: Start with `01-multimodal-embeddings/code/` to build cross-modal systems ## Summary The LLM Engineering phases 9-12 in rohitg00/ai-engineering-from-scratch provide a systematic progression from theoretical fundamentals to production systems: - **Phase 9** establishes reinforcement learning foundations necessary for RLHF alignment techniques - **Phase 10** teaches end-to-end language model construction, from tokenization to training loops - **Phase 11** covers production engineering, including prompt optimization, fine-tuning, and API deployment - **Phase 12** integrates multimodal capabilities, combining text with vision and audio through cross-modal attention Each phase contains specific entry-point scripts runnable from the repository root, enabling hands-on experimentation with the core concepts. ## Frequently Asked Questions ### What prerequisites are needed for LLM Engineering Phase 11? According to the repository structure, learners should complete **Phase 10 (LLMs from Scratch)** before advancing to Phase 11. Understanding tokenization and basic training loops from `phases/10-llms-from-scratch/` provides the necessary context for prompt engineering and fine-tuning operations covered in Phase 11. ### How does Phase 9 relate to modern LLM training? Phase 9 covers **Markov Decision Processes and policy evaluation**, which form the mathematical foundation for RLHF (Reinforcement Learning from Human Feedback). The value functions and policy optimization techniques implemented in `phases/09-reinforcement-learning/` directly translate to the alignment methods used in production language models like GPT-4 and Claude. ### What distinguishes Phase 10 from Phase 11 in the curriculum? **Phase 10** focuses on *building* language models from scratch, covering architecture and training in `phases/10-llms-from-scratch/`, while **Phase 11** focuses on *engineering* existing models—optimizing inference, designing prompts, and deploying services. Phase 10 is about creation; Phase 11 is about productionization and scaling. ### Which phase should I start with for multimodal AI applications? Start with **Phase 12**, but ensure you have completed **Phase 11** first. The multimodal implementations in `phases/12-multimodal-ai/` assume familiarity with transformer architectures and inference patterns established in the earlier LLM Engineering phases, particularly the embedding handling and attention mechanisms covered in Phase 11.