# aie-book | Chip Huyen | Knowledge Base | Instagit

[WIP] Resources for AI engineers. Also contains supporting materials for the book AI Engineering (Chip Huyen, 2025)

GitHub Stars: 15.3k

Repository: https://github.com/chiphuyen/aie-book

---

## Articles

### [Optimizing Inference Latency in Production AI Systems: A Complete Guide from AI Engineering](/chiphuyen/aie-book/optimize-inference-latency-production-ai)

Master production AI inference latency optimization. Reduce time to first token (TTFT) with quantized prefilling and optimize time per output token (TPOT) using dynamic batching and KV-cache.

- Tags: how-to-guide
- Published: 2026-04-24

### [How to Select the Right Foundation Model for Your Use Case: A 3-Stage Engineering Approach](/chiphuyen/aie-book/select-right-foundation-model-use-case)

Select the right foundation model for your use case with a 3-stage engineering approach. Define evaluation axes, gather candidates, and benchmark for latency, cost, and accuracy trade-offs.

- Tags: how-to-guide
- Published: 2026-04-24

### [How to Solve Memory Bottlenecks in Training Foundation Models: 6 Proven Strategies](/chiphuyen/aie-book/memory-bottlenecks-training-foundation-models)

Solve memory bottlenecks in foundation model training with 6 expert strategies. Learn PEFT, gradient checkpointing, quantization, and more to optimize your training.

- Tags: best-practices
- Published: 2026-04-24

### [How to Build Feedback Loops for AI Applications: Architecture, Implementation, and Best Practices](/chiphuyen/aie-book/build-feedback-loops-ai-applications)

Learn how to build feedback loops for AI applications. Connect user interfaces, observability, storage, and model updates for continuous improvement and better AI.

- Tags: best-practices
- Published: 2026-04-24

### [How to Implement Structured Outputs from LLMs Reliably: A Multi-Layered Architecture](/chiphuyen/aie-book/implement-structured-outputs-llms-reliably)

Learn to implement reliable structured outputs from LLMs with a multi-layered architecture including prompt guardrails, server-side validation, and grammar constraints for consistent JSON.

- Tags: architecture
- Published: 2026-04-24

### [AI Engineering vs Traditional ML Engineering: 5 Key Differences Explained](/chiphuyen/aie-book/differences-ai-engineering-vs-ml-engineering)

Discover the 5 key differences between AI engineering and traditional ML engineering. Understand the shift to foundation models, prompt engineering, and feedback loops.

- Tags: deep-dive
- Published: 2026-04-24

### [How Test Time Compute Improves LLM Performance: 6 Architectural Methods](/chiphuyen/aie-book/how-test-time-compute-improves-llm-performance)

Discover how test time compute boosts LLM performance with 6 architectural methods like beam search and retrieval augmentation. Enhance output and inject knowledge without retraining.

- Tags: deep-dive
- Published: 2026-04-24

### [Prompt Injection Defense: 6 Proven Strategies for Securing LLM Applications](/chiphuyen/aie-book/best-strategies-prompt-injection-defense)

Discover 6 proven prompt injection defense strategies to secure LLM applications. Learn to implement layered protection against jailbreak attacks for robust AI security.

- Tags: best-practices
- Published: 2026-04-24

### [How to Design an AI Engineering Architecture with Guardrails: A Layered Approach](/chiphuyen/aie-book/design-ai-engineering-architecture-guardrails)

Design an AI engineering architecture with guardrails using a layered approach. Implement safety checks at the model gateway, inference service, and standalone policy service for robust AI systems.

- Tags: architecture
- Published: 2026-04-24

### [How to Reduce Inference Cost with Quantization: A Complete Technical Guide](/chiphuyen/aie-book/reduce-inference-cost-with-quantization)

Reduce inference cost with quantization. Learn how INT8 and 4-bit models cut memory by 75% and boost throughput 2-4x. Optimize models without code changes.

- Tags: how-to-guide
- Published: 2026-04-24

### [Quantization Methods for Reducing LLM Model Size: 5 Techniques Explained](/chiphuyen/aie-book/quantization-methods-reducing-llm-size)

Discover 5 quantization methods to reduce LLM model size by up to 75%. Learn how converting 32-bit weights to lower-precision formats like INT8 or FP16 maintains model capability.

- Tags: deep-dive
- Published: 2026-04-24

### [When to Use LoRA for Fine-Tuning: 6 Scenarios for Parameter-Efficient LLM Training](/chiphuyen/aie-book/when-to-use-lora-for-fine-tuning)

Discover when to use LoRA for fine-tuning LLMs efficiently. Train models with limited GPU memory and enable rapid prototyping and modular deployment.

- Tags: best-practices
- Published: 2026-04-24

### [What Are Parameter-Efficient Fine-Tuning Techniques? A Complete Guide](/chiphuyen/aie-book/parameter-efficient-fine-tuning-techniques)

Discover parameter-efficient fine-tuning PEFT techniques to adapt large models by updating just a small fraction of parameters. Fine-tune on a single GPU, saving resources.

- Tags: deep-dive
- Published: 2026-04-24

### [Common Failure Modes When Building AI Agents: Lessons from the AIE Book](/chiphuyen/aie-book/common-failure-modes-ai-agents)

Discover common failure modes when building AI agents including tool selection, reasoning, security, and observability issues. Learn from AIE book insights to avoid costly mistakes.

- Tags: best-practices
- Published: 2026-04-24

### [How to Build AI Agents: Architecture, Tools, and Implementation Guide](/chiphuyen/aie-book/how-to-build-ai-agents)

Learn how to build AI agents by implementing an iterative loop: planning, tool selection, execution, and synthesis. This guide covers architecture, tools, and implementation.

- Tags: how-to-guide
- Published: 2026-04-24

### [How RAG Retrieval Algorithms Compare: Sparse, Dense, and Hybrid Search Explained](/chiphuyen/aie-book/how-rag-retrieval-algorithms-compare)

Compare RAG retrieval algorithms sparse dense and hybrid search. Learn how BM25 semantic similarity and combined approaches boost RAG performance for better search results.

- Tags: deep-dive
- Published: 2026-04-24

### [Retrieval Algorithms in RAG: Term-Based, Embedding-Based, and Hybrid Methods](/chiphuyen/aie-book/different-retrieval-algorithms-in-rag)

Explore RAG retrieval algorithms including term-based, embedding-based, and hybrid methods. Understand how these techniques enhance search accuracy for LLMs.

- Tags: deep-dive
- Published: 2026-04-24

### [When to Choose RAG Over Fine-Tuning for LLMs: A Complete Decision Framework](/chiphuyen/aie-book/when-to-choose-rag-over-fine-tuning)

Decide between RAG and fine-tuning LLMs. Select RAG for dynamic knowledge and cost savings, or fine-tuning for specific behaviors with high-quality data.

- Tags: deep-dive
- Published: 2026-04-24

### [Benefits and Limitations of Using AI as a Judge for Evaluation: A Complete Technical Guide](/chiphuyen/aie-book/benefits-and-limitations-of-ai-as-judge-evaluation)

Explore the benefits and limitations of using AI as a judge for evaluation in generative tasks. Learn about scalable assessment, subjectivity, and instability to optimize your AI models.

- Tags: deep-dive
- Published: 2026-04-24

### [How to Evaluate AI Systems Using AI as a Judge: A 5-Step Technical Implementation](/chiphuyen/aie-book/how-to-evaluate-ai-systems-using-ai-as-a-judge)

Learn to evaluate AI systems using AI as a judge. Implement a 5-step technical process with LLMs for reproducible quality metrics. Enhance your AI development.

- Tags: how-to-guide
- Published: 2026-04-24

### [Zero-Shot vs Few-Shot Prompting: When to Use Each Strategy](/chiphuyen/aie-book/when-to-use-zero-shot-vs-few-shot-prompting)

Learn when to use zero-shot vs few-shot prompting. Optimize for familiar tasks or token limits with zero-shot and enforce formats or safety with few-shot.

- Tags: deep-dive
- Published: 2026-04-24

### [How Does In-Context Learning Work in LLMs? A Technical Deep Dive](/chiphuyen/aie-book/how-does-in-context-learning-work-in-llms)

Discover how in-context learning enables LLMs to tackle new tasks using prompt examples without weight updates. A technical deep dive into ICL.

- Tags: deep-dive
- Published: 2026-04-24