# How the Agent Loop Function Works in AI Engineering: Core Architecture Explained

> Understand the Agent Loop function in AI engineering. Discover how think-act-observe cycles create autonomous agents from language models to meet stop conditions.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: architecture
- Published: 2026-07-31

---

**The Agent Loop function is the fundamental control-flow mechanism that transforms a language model into an autonomous agent by cycling through think-act-observe iterations while maintaining a persistent message buffer until a stop condition is met.**

The Agent Loop serves as the universal backbone for modern autonomous AI systems. In the `rohitg00/ai-engineering-from-scratch` repository, specifically within Phase 14 (Agent Engineering), this pattern demonstrates how minimal components can create self-reinforcing feedback loops that enable multi-step reasoning and tool use. Understanding this bare-bones implementation provides a solid mental model for any higher-level framework.

## The Five Mandatory Components of the Agent Loop

Every robust Agent Loop implementation requires five specific ingredients to function correctly. These components are explicitly defined in the curriculum documentation at [`phases/14-agent-engineering/01-the-agent-loop/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/docs/en.md).

### Message Buffer

The **message buffer** stores the complete conversation transcript, including user turns, assistant thoughts, tool calls, and observations. It grows with each iteration so the LLM can access the full history for context-aware decision making. According to the documentation at lines 55-60, this buffer is essential for the self-reinforcing feedback that distinguishes agents from simple chat bots.

### Tool Registry

The **tool registry** maps string tool names to executable callables. When the model emits an `Action`, the registry dispatches the call and returns a string result. In the reference implementation at [`phases/14-agent-engineering/01-the-agent-loop/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/main.py) (lines 34-44), the registry handles functions like `calculator`, `kv_get`, and `kv_set`.

### Stop Condition

The **stop condition** terminates the loop when the LLM emits a `finish` message, when no tool call appears, or when the turn budget exhausts. The implementation at lines 94-102 checks the reply kind and returns the final content immediately upon completion.

### Turn Budget

The **turn budget** caps the maximum number of iterations to prevent infinite loops. The default configuration sets this to 12 turns, as implemented at lines 101-103 of the main module. When the budget exhausts without completion, the agent returns a "budget exhausted" signal.

### Observation Formatter

The **observation formatter** converts raw tool output into plain-text strings suitable for LLM consumption. In the toy implementation, this is simply the return value of the tool function (lines 44-53), though production agents may wrap binary data or errors into readable representations.

## Step-by-Step Execution Flow

The Agent Loop follows a strict six-step cycle that repeats until completion:

1. **Initialize** – The `AgentLoop` receives an LLM client and `ToolRegistry`, appending the user's prompt to the buffer as a `user` turn.

2. **LLM Turn** – The method `llm.respond(history)` produces a dictionary containing either a `thought` plus an `action` with arguments, or a `finish` payload.

3. **Record Thought** – The system appends the thought string to the buffer as a `thought` turn, preserving the reasoning chain.

4. **Dispatch Action** – A `ToolCall` object is constructed from the LLM's `action`. The registry looks up the callable, executes it, and returns an observation string.

5. **Record Action & Observation** – The `action` turn stores both the tool call and its observation, making this context available for subsequent iterations.

6. **Loop or Stop** – The process repeats until the LLM signals `finish`, the buffer reaches `max_turns`, or another guardrail triggers.

## Reference Implementation from ai-engineering-from-scratch

The repository provides a complete pure-stdlib implementation demonstrating these concepts. Here is the core loop structure from [`phases/14-agent-engineering/01-the-agent-loop/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/main.py):

```python
def run(self, user_message: str) -> str:
    self.history.append(Turn(kind="user", content=user_message))
    for step in range(self.max_turns):
        reply = self.llm.respond(self.history)
        if reply["kind"] == "finish":
            self.history.append(Turn(kind="final", content=reply["content"]))
            return reply["content"]
        # record thought

        self.history.append(Turn(kind="thought", content=reply.get("thought", "")))
        # dispatch action

        call = ToolCall(name=reply["action"], args=reply.get("args", {}))
        observation = self.tools.dispatch(call)
        self.history.append(
            Turn(kind="action", content=call.name,
                 tool_call=call, observation=observation)
        )
    # budget exhausted

    self.history.append(Turn(kind="final", content="budget exhausted"))
    return "budget exhausted"

```

The `Turn` dataclass structures each interaction with appropriate metadata:

```python
@dataclass
class Turn:
    kind: str          # user, thought, action, final …

    content: str
    tool_call: ToolCall | None = None
    observation: str | None = None

```

Tool registration follows a simple pattern:

```python
tools = ToolRegistry()
tools.register("calculator", calculator)
tools.register("kv_get", kv.get)
tools.register("kv_set", kv.set)

```

Running the demo agent produces a trace showing the loop in action:

```python
from phases_14_agent_engineering_01_the_agent_loop.code.main import build_demo_agent

agent = build_demo_agent()
final_answer = agent.run("What is 120 plus 15% tax, stored in kv?")
print("final answer:", final_answer)

```

Output:

```

[00   user] What is 120 plus 15% tax, stored in kv?
[01 thought] store the base price
[02 action] kv_set({'key': 'base', 'value': '120'}) -> stored base
[03 thought] compute 15% tax
[04 action] calculator({'expr': '120 * 0.15'}) -> 18.0
[05 thought] store the tax
[06 action] kv_set({'key': 'tax', 'value': '18.0'}) -> stored tax
[07 thought] compute total
[08 action] calculator({'expr': '120 + 18.0'}) -> 138.0
final answer: the total including 15% tax is 138.0

```

## Why Every Major Agent Framework Uses This Pattern

Modern agent SDKs are essentially wrappers around this same loop. **Claude Agent SDK** injects built-in tools and lifecycle hooks around the identical ReAct cycle. **OpenAI Agents SDK** adds guardrails and session tracking but maintains the same `respond → dispatch → observe` sequence. **LangGraph** treats each turn as a node in a stateful graph while preserving the buffer semantics. **AutoGen v0.4** runs the loop in async actors, yet the core "think-act-observe" pattern remains unchanged.

Mastering this bare-bones implementation from `rohitg00/ai-engineering-from-scratch` provides the foundational mental model necessary to understand any higher-level framework.

## Summary

- The **Agent Loop function** requires five mandatory components: a message buffer, tool registry, stop condition, turn budget, and observation formatter.
- Execution follows a cyclical pattern: **LLM generates thought → tool dispatches → observation records → loop repeats** until completion.
- The reference implementation at [`phases/14-agent-engineering/01-the-agent-loop/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/main.py) demonstrates a complete working example using only Python standard library components.
- Modern frameworks like LangGraph and AutoGen build ergonomic abstractions atop this identical control-flow pattern.

## Frequently Asked Questions

### What distinguishes an Agent Loop from a simple chat completion loop?

A simple chat loop only alternates between user and assistant messages, while the **Agent Loop** explicitly handles tool calls by dispatching actions, capturing observations, and feeding those results back into the context window. This creates a self-reinforcing feedback mechanism that enables multi-step reasoning and autonomous task completion.

### Why is the turn budget necessary if the stop condition handles completion?

The **turn budget** (default 12 turns) serves as a safety guardrail to prevent infinite loops when the LLM enters circular reasoning or repeatedly calls tools without reaching a valid finish state. Without this cap, a malfunctioning agent could consume unlimited API tokens or compute resources.

### How does the Agent Loop handle tool execution errors?

In the reference implementation, the **observation formatter** receives the raw return value from the tool call, which may include error strings. The loop appends these observations to the message buffer regardless of success or failure, allowing the LLM to see the error in the next iteration and potentially recover or report the failure.

### Can the Agent Loop function work with any large language model?

Yes, the pattern is model-agnostic. The `ToyLLM` class in the curriculum demonstrates the interface contract: any model that accepts a message history and returns structured output (thought/action or finish) can power the loop. Production implementations typically use OpenAI, Anthropic, or open-weight models via the same `respond(history)` interface.