Agent Loop Implementation in AI Engineering from Scratch: A Complete Technical Guide

The Agent Loop implementation in AI Engineering from Scratch provides a deterministic, ReAct-style control flow where a language model iteratively generates thoughts and actions, executes registered tools, and feeds observations back into the conversation buffer until reaching a stop condition or turn budget.

The rohitg00/ai-engineering-from-scratch repository offers a minimal, standard-library-only reference architecture for building autonomous agents. At its foundation lies the Agent Loop, located in phases/14-agent-engineering/01-the-agent-loop/code/main.py, which demonstrates the core cognitive cycle that powers modern frameworks like LangGraph, AutoGen, and the OpenAI Agents SDK. This implementation emphasizes clarity over complexity, making it an ideal starting point for understanding how AI agents reason and act.

Core Architecture Components

The Agent Loop implementation consists of five mandatory ingredients orchestrated through clean, composable classes.

ToolRegistry

The ToolRegistry maintains a mapping between string tool names and their callable implementations. It exposes register(), names(), and a safe dispatch() method that handles argument validation and error isolation.

from phases.14-agent-engineering.01-the-agent-loop.code.main import ToolRegistry, ToolCall

registry = ToolRegistry()
registry.register("calculator", lambda expr: eval(expr))  # Simplified example

observation = registry.dispatch(ToolCall(name="calculator", args={"expr": "2+2"}))

ToyLLM

The ToyLLM class provides a scripted, deterministic "language model" that returns pre-programmed sequences of dictionaries containing kind, thought, action, and args keys. This deterministic behavior guarantees repeatable traces during development while maintaining interface compatibility with real API providers.

AgentLoop

The AgentLoop class serves as the central orchestrator. It initializes with a message buffer (history), a ToolRegistry instance, and a configurable max_turns budget. The run() method implements the classic ReAct cycle: appending user messages, querying the LLM, recording thoughts, dispatching tool calls, and storing observations.

Supporting Tools

The implementation includes example tools that demonstrate stateful interactions:

• KVStore: A simple key-value store (kv_get, kv_set) illustrating how tools maintain mutable state across turns
• calculator: A safe arithmetic evaluator that validates inputs contain only numeric characters and operators, showing defensive tool design

The ReAct Execution Flow

The Agent Loop implementation follows the strict Observe → Think → Act → Observe pattern. Inside phases/14-agent-engineering/01-the-agent-loop/code/main.py, the AgentLoop.run() method implements this flow:

def run(self, user_message: str) -> str:
    self.history.append(Turn(kind="user", content=user_message))
    
    for step in range(self.max_turns):
        reply = self.llm.respond(self.history)          # LLM produces Thought + Action

        if reply["kind"] == "finish":                  # Stop condition

            self.history.append(Turn(kind="final", content=reply["content"]))
            return reply["content"]
            
        self.history.append(Turn(kind="thought", content=reply.get("thought", "")))
        call = ToolCall(name=reply["action"], args=reply.get("args", {}))
        observation = self.tools.dispatch(call)         # Tool execution

        
        self.history.append(
            Turn(kind="action", content=call.name,
                 tool_call=call, observation=observation)
        )
    
    # Turn budget exhausted

    self.history.append(Turn(kind="final", content="budget exhausted"))
    return "budget exhausted"

Each iteration appends structured Turn objects to the history buffer, creating a complete trace of the agent's reasoning process.

Working with the Agent Loop

Creating a Simple Agent

Use the provided build_demo_agent() factory to instantiate an agent with pre-registered tools:

from phases.14-agent-engineering.01-the-agent-loop.code.main import (
    build_demo_agent, pretty_trace
)

agent = build_demo_agent()                     # registers calculator, kv_get, kv_set

result = agent.run("What is 120 plus 15% tax, stored in kv?")

pretty_trace(agent.history)                     # prints the full ReAct trace

print("final answer:", result)

This produces a structured trace showing each thought, action, and observation until the final answer is reached.

Adding Custom Tools

Extend the agent's capabilities by registering new functions with type hints:

def reverse_string(s: str) -> str:
    """Reverse the input string."""
    return s[::-1]

agent.tools.register("reverse", reverse_string)

# The ToyLLM can now dispatch: 

# {"kind":"action","thought":"reverse name","action":"reverse","args":{"s":"Alice"}}

# Observation: "ecilA"

Integrating Production LLMs

The loop's architecture allows swapping ToyLLM for real providers without modifying the core logic:

from openai import OpenAI  # Conceptual example

real_llm = OpenAI(api_key="YOUR_KEY")

# Pass real_llm to AgentLoop constructor; the respond() interface remains consistent

Summary

• The Agent Loop implements a ReAct control flow in pure Python using only standard library modules.
• The five mandatory ingredients are: a message buffer (history), a ToolRegistry, a stop condition (finish signal), a configurable turn budget (max_turns), and an observation formatter.
• ToolRegistry.dispatch() provides safe tool execution with built-in error handling and argument validation.
• ToyLLM enables deterministic, repeatable testing while maintaining interface parity with production LLM APIs.
• The implementation emphasizes that modern agent frameworks (LangGraph, Claude Agent SDK, OpenAI Agents SDK) build upon this identical loop structure, differing primarily in state persistence, actor-model messaging, and tracing infrastructure.

Frequently Asked Questions

What is the Agent Loop pattern in AI engineering?

The Agent Loop is an architectural pattern that enables autonomous decision-making in AI systems. It follows the ReAct (Reasoning + Acting) methodology where a language model repeatedly observes the environment, generates an internal thought, selects an action, executes that action via tools, and incorporates the resulting observation back into its context. This cycle continues until the task is complete or resources are exhausted.

How does the ToyLLM differ from production LLM providers?

ToyLLM is a deterministic, scripted simulator that returns pre-defined sequences of actions and thoughts for testing and educational purposes. Unlike production providers such as OpenAI or Anthropic, it lacks stochastic generation and instead follows a fixed script, ensuring repeatable outputs that help developers debug agent logic without API costs or rate limits. However, it implements the same respond(history) interface, allowing seamless substitution with real LLM clients.

What are the five mandatory ingredients of the Agent Loop?

According to the source code in rohitg00/ai-engineering-from-scratch, every Agent Loop implementation requires: (1) a message buffer or history to maintain conversational state, (2) a tool registry for dispatching actions, (3) a stop condition (typically a finish signal), (4) a turn budget to prevent infinite loops, and (5) an observation formatter to structure tool outputs for the LLM context.

How do I add custom tools to the Agent Loop?

Register new tools using the ToolRegistry.register() method, passing a unique string name and a callable function. The function should accept typed arguments and return a string observation. Once registered, the LLM (whether ToyLLM or a production model) can invoke the tool by name with appropriate arguments via the standard action dispatch mechanism.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}