How the Agent Loop Function Works in AI Engineering: Core Architecture Explained

The Agent Loop function is the fundamental control-flow mechanism that transforms a language model into an autonomous agent by cycling through think-act-observe iterations while maintaining a persistent message buffer until a stop condition is met.

The Agent Loop serves as the universal backbone for modern autonomous AI systems. In the rohitg00/ai-engineering-from-scratch repository, specifically within Phase 14 (Agent Engineering), this pattern demonstrates how minimal components can create self-reinforcing feedback loops that enable multi-step reasoning and tool use. Understanding this bare-bones implementation provides a solid mental model for any higher-level framework.

The Five Mandatory Components of the Agent Loop

Every robust Agent Loop implementation requires five specific ingredients to function correctly. These components are explicitly defined in the curriculum documentation at phases/14-agent-engineering/01-the-agent-loop/docs/en.md.

Message Buffer

The message buffer stores the complete conversation transcript, including user turns, assistant thoughts, tool calls, and observations. It grows with each iteration so the LLM can access the full history for context-aware decision making. According to the documentation at lines 55-60, this buffer is essential for the self-reinforcing feedback that distinguishes agents from simple chat bots.

Tool Registry

The tool registry maps string tool names to executable callables. When the model emits an Action, the registry dispatches the call and returns a string result. In the reference implementation at phases/14-agent-engineering/01-the-agent-loop/code/main.py (lines 34-44), the registry handles functions like calculator, kv_get, and kv_set.

Stop Condition

The stop condition terminates the loop when the LLM emits a finish message, when no tool call appears, or when the turn budget exhausts. The implementation at lines 94-102 checks the reply kind and returns the final content immediately upon completion.

Turn Budget

The turn budget caps the maximum number of iterations to prevent infinite loops. The default configuration sets this to 12 turns, as implemented at lines 101-103 of the main module. When the budget exhausts without completion, the agent returns a "budget exhausted" signal.

Observation Formatter

The observation formatter converts raw tool output into plain-text strings suitable for LLM consumption. In the toy implementation, this is simply the return value of the tool function (lines 44-53), though production agents may wrap binary data or errors into readable representations.

Step-by-Step Execution Flow

The Agent Loop follows a strict six-step cycle that repeats until completion:

1. Initialize – The AgentLoop receives an LLM client and ToolRegistry, appending the user's prompt to the buffer as a user turn.

2. LLM Turn – The method llm.respond(history) produces a dictionary containing either a thought plus an action with arguments, or a finish payload.

3. Record Thought – The system appends the thought string to the buffer as a thought turn, preserving the reasoning chain.

4. Dispatch Action – A ToolCall object is constructed from the LLM's action. The registry looks up the callable, executes it, and returns an observation string.

5. Record Action & Observation – The action turn stores both the tool call and its observation, making this context available for subsequent iterations.

6. Loop or Stop – The process repeats until the LLM signals finish, the buffer reaches max_turns, or another guardrail triggers.

Reference Implementation from ai-engineering-from-scratch

The repository provides a complete pure-stdlib implementation demonstrating these concepts. Here is the core loop structure from phases/14-agent-engineering/01-the-agent-loop/code/main.py:

def run(self, user_message: str) -> str:
    self.history.append(Turn(kind="user", content=user_message))
    for step in range(self.max_turns):
        reply = self.llm.respond(self.history)
        if reply["kind"] == "finish":
            self.history.append(Turn(kind="final", content=reply["content"]))
            return reply["content"]
        # record thought

        self.history.append(Turn(kind="thought", content=reply.get("thought", "")))
        # dispatch action

        call = ToolCall(name=reply["action"], args=reply.get("args", {}))
        observation = self.tools.dispatch(call)
        self.history.append(
            Turn(kind="action", content=call.name,
                 tool_call=call, observation=observation)
        )
    # budget exhausted

    self.history.append(Turn(kind="final", content="budget exhausted"))
    return "budget exhausted"

The Turn dataclass structures each interaction with appropriate metadata:

@dataclass
class Turn:
    kind: str          # user, thought, action, final …

    content: str
    tool_call: ToolCall | None = None
    observation: str | None = None

Tool registration follows a simple pattern:

tools = ToolRegistry()
tools.register("calculator", calculator)
tools.register("kv_get", kv.get)
tools.register("kv_set", kv.set)

Running the demo agent produces a trace showing the loop in action:

from phases_14_agent_engineering_01_the_agent_loop.code.main import build_demo_agent

agent = build_demo_agent()
final_answer = agent.run("What is 120 plus 15% tax, stored in kv?")
print("final answer:", final_answer)

Output:

[00   user] What is 120 plus 15% tax, stored in kv?
[01 thought] store the base price
[02 action] kv_set({'key': 'base', 'value': '120'}) -> stored base
[03 thought] compute 15% tax
[04 action] calculator({'expr': '120 * 0.15'}) -> 18.0
[05 thought] store the tax
[06 action] kv_set({'key': 'tax', 'value': '18.0'}) -> stored tax
[07 thought] compute total
[08 action] calculator({'expr': '120 + 18.0'}) -> 138.0
final answer: the total including 15% tax is 138.0

Why Every Major Agent Framework Uses This Pattern

Modern agent SDKs are essentially wrappers around this same loop. Claude Agent SDK injects built-in tools and lifecycle hooks around the identical ReAct cycle. OpenAI Agents SDK adds guardrails and session tracking but maintains the same respond → dispatch → observe sequence. LangGraph treats each turn as a node in a stateful graph while preserving the buffer semantics. AutoGen v0.4 runs the loop in async actors, yet the core "think-act-observe" pattern remains unchanged.

Mastering this bare-bones implementation from rohitg00/ai-engineering-from-scratch provides the foundational mental model necessary to understand any higher-level framework.

Summary

• The Agent Loop function requires five mandatory components: a message buffer, tool registry, stop condition, turn budget, and observation formatter.
• Execution follows a cyclical pattern: LLM generates thought → tool dispatches → observation records → loop repeats until completion.
• The reference implementation at phases/14-agent-engineering/01-the-agent-loop/code/main.py demonstrates a complete working example using only Python standard library components.
• Modern frameworks like LangGraph and AutoGen build ergonomic abstractions atop this identical control-flow pattern.

Frequently Asked Questions

What distinguishes an Agent Loop from a simple chat completion loop?

A simple chat loop only alternates between user and assistant messages, while the Agent Loop explicitly handles tool calls by dispatching actions, capturing observations, and feeding those results back into the context window. This creates a self-reinforcing feedback mechanism that enables multi-step reasoning and autonomous task completion.

Why is the turn budget necessary if the stop condition handles completion?

The turn budget (default 12 turns) serves as a safety guardrail to prevent infinite loops when the LLM enters circular reasoning or repeatedly calls tools without reaching a valid finish state. Without this cap, a malfunctioning agent could consume unlimited API tokens or compute resources.

How does the Agent Loop handle tool execution errors?

In the reference implementation, the observation formatter receives the raw return value from the tool call, which may include error strings. The loop appends these observations to the message buffer regardless of success or failure, allowing the LLM to see the error in the next iteration and potentially recover or report the failure.

Can the Agent Loop function work with any large language model?

Yes, the pattern is model-agnostic. The ToyLLM class in the curriculum demonstrates the interface contract: any model that accepts a message history and returns structured output (thought/action or finish) can power the loop. Production implementations typically use OpenAI, Anthropic, or open-weight models via the same respond(history) interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}