# Agent Loop Implementation in AI Engineering from Scratch: A Complete Technical Guide

> Master Agent Loop implementation in AI Engineering from Scratch. Learn how AI models generate thoughts, actions, and use tools iteratively in this complete technical guide.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-08-28

---

**The Agent Loop implementation in *AI Engineering from Scratch* provides a deterministic, ReAct-style control flow where a language model iteratively generates thoughts and actions, executes registered tools, and feeds observations back into the conversation buffer until reaching a stop condition or turn budget.**

The `rohitg00/ai-engineering-from-scratch` repository offers a minimal, standard-library-only reference architecture for building autonomous agents. At its foundation lies the **Agent Loop**, located in [`phases/14-agent-engineering/01-the-agent-loop/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/main.py), which demonstrates the core cognitive cycle that powers modern frameworks like LangGraph, AutoGen, and the OpenAI Agents SDK. This implementation emphasizes clarity over complexity, making it an ideal starting point for understanding how AI agents reason and act.

## Core Architecture Components

The Agent Loop implementation consists of five mandatory ingredients orchestrated through clean, composable classes.

### ToolRegistry

The **`ToolRegistry`** maintains a mapping between string tool names and their callable implementations. It exposes `register()`, `names()`, and a safe `dispatch()` method that handles argument validation and error isolation.

```python
from phases.14-agent-engineering.01-the-agent-loop.code.main import ToolRegistry, ToolCall

registry = ToolRegistry()
registry.register("calculator", lambda expr: eval(expr))  # Simplified example

observation = registry.dispatch(ToolCall(name="calculator", args={"expr": "2+2"}))

```

### ToyLLM

The **`ToyLLM`** class provides a scripted, deterministic "language model" that returns pre-programmed sequences of dictionaries containing `kind`, `thought`, `action`, and `args` keys. This deterministic behavior guarantees repeatable traces during development while maintaining interface compatibility with real API providers.

### AgentLoop

The **`AgentLoop`** class serves as the central orchestrator. It initializes with a **message buffer** (`history`), a `ToolRegistry` instance, and a configurable `max_turns` budget. The `run()` method implements the classic ReAct cycle: appending user messages, querying the LLM, recording thoughts, dispatching tool calls, and storing observations.

### Supporting Tools

The implementation includes example tools that demonstrate stateful interactions:

- **`KVStore`**: A simple key-value store (`kv_get`, `kv_set`) illustrating how tools maintain mutable state across turns
- **`calculator`**: A safe arithmetic evaluator that validates inputs contain only numeric characters and operators, showing defensive tool design

## The ReAct Execution Flow

The Agent Loop implementation follows the strict **Observe → Think → Act → Observe** pattern. Inside [`phases/14-agent-engineering/01-the-agent-loop/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/main.py), the `AgentLoop.run()` method implements this flow:

```python
def run(self, user_message: str) -> str:
    self.history.append(Turn(kind="user", content=user_message))
    
    for step in range(self.max_turns):
        reply = self.llm.respond(self.history)          # LLM produces Thought + Action

        if reply["kind"] == "finish":                  # Stop condition

            self.history.append(Turn(kind="final", content=reply["content"]))
            return reply["content"]
            
        self.history.append(Turn(kind="thought", content=reply.get("thought", "")))
        call = ToolCall(name=reply["action"], args=reply.get("args", {}))
        observation = self.tools.dispatch(call)         # Tool execution

        
        self.history.append(
            Turn(kind="action", content=call.name,
                 tool_call=call, observation=observation)
        )
    
    # Turn budget exhausted

    self.history.append(Turn(kind="final", content="budget exhausted"))
    return "budget exhausted"

```

Each iteration appends structured `Turn` objects to the history buffer, creating a complete trace of the agent's reasoning process.

## Working with the Agent Loop

### Creating a Simple Agent

Use the provided `build_demo_agent()` factory to instantiate an agent with pre-registered tools:

```python
from phases.14-agent-engineering.01-the-agent-loop.code.main import (
    build_demo_agent, pretty_trace
)

agent = build_demo_agent()                     # registers calculator, kv_get, kv_set

result = agent.run("What is 120 plus 15% tax, stored in kv?")

pretty_trace(agent.history)                     # prints the full ReAct trace

print("final answer:", result)

```

This produces a structured trace showing each thought, action, and observation until the final answer is reached.

### Adding Custom Tools

Extend the agent's capabilities by registering new functions with type hints:

```python
def reverse_string(s: str) -> str:
    """Reverse the input string."""
    return s[::-1]

agent.tools.register("reverse", reverse_string)

# The ToyLLM can now dispatch: 

# {"kind":"action","thought":"reverse name","action":"reverse","args":{"s":"Alice"}}

# Observation: "ecilA"

```

### Integrating Production LLMs

The loop's architecture allows swapping `ToyLLM` for real providers without modifying the core logic:

```python
from openai import OpenAI  # Conceptual example

real_llm = OpenAI(api_key="YOUR_KEY")

# Pass real_llm to AgentLoop constructor; the respond() interface remains consistent

```

## Summary

- The **Agent Loop** implements a ReAct control flow in pure Python using only standard library modules.
- The five mandatory ingredients are: a message buffer (history), a **ToolRegistry**, a stop condition (`finish` signal), a configurable **turn budget** (`max_turns`), and an observation formatter.
- **`ToolRegistry.dispatch()`** provides safe tool execution with built-in error handling and argument validation.
- **`ToyLLM`** enables deterministic, repeatable testing while maintaining interface parity with production LLM APIs.
- The implementation emphasizes that modern agent frameworks (LangGraph, Claude Agent SDK, OpenAI Agents SDK) build upon this identical loop structure, differing primarily in state persistence, actor-model messaging, and tracing infrastructure.

## Frequently Asked Questions

### What is the Agent Loop pattern in AI engineering?

The **Agent Loop** is an architectural pattern that enables autonomous decision-making in AI systems. It follows the ReAct (Reasoning + Acting) methodology where a language model repeatedly observes the environment, generates an internal thought, selects an action, executes that action via tools, and incorporates the resulting observation back into its context. This cycle continues until the task is complete or resources are exhausted.

### How does the ToyLLM differ from production LLM providers?

**`ToyLLM`** is a deterministic, scripted simulator that returns pre-defined sequences of actions and thoughts for testing and educational purposes. Unlike production providers such as OpenAI or Anthropic, it lacks stochastic generation and instead follows a fixed script, ensuring repeatable outputs that help developers debug agent logic without API costs or rate limits. However, it implements the same `respond(history)` interface, allowing seamless substitution with real LLM clients.

### What are the five mandatory ingredients of the Agent Loop?

According to the source code in `rohitg00/ai-engineering-from-scratch`, every Agent Loop implementation requires: (1) a **message buffer** or history to maintain conversational state, (2) a **tool registry** for dispatching actions, (3) a **stop condition** (typically a `finish` signal), (4) a **turn budget** to prevent infinite loops, and (5) an **observation formatter** to structure tool outputs for the LLM context.

### How do I add custom tools to the Agent Loop?

Register new tools using the **`ToolRegistry.register()`** method, passing a unique string name and a callable function. The function should accept typed arguments and return a string observation. Once registered, the LLM (whether `ToyLLM` or a production model) can invoke the tool by name with appropriate arguments via the standard action dispatch mechanism.