Examples of Agents Built Using This Framework: 3 Production-Ready AI Agents

The bojieli/ai-agent-book repository provides three complete agent implementations—UserMemoryAgent, MultimodalAgent, and SelfEvolvingAgent—that demonstrate tool-based ReAct loops, multimodal content processing, and dynamic self-evolution capabilities.

The bojieli/ai-agent-book repository delivers a practical framework for constructing production-grade LLM agents with persistent state, multimodal inputs, and adaptive tool usage. These examples of agents built using this framework showcase architectural patterns for streaming interactions, cross-provider LLM support, and automated tool dispatch. Each implementation follows a consistent design philosophy while addressing distinct operational requirements, from managing user memories to processing diverse media formats.

UserMemoryAgent: Tool-Based Memory Management

The UserMemoryAgent (chapter3/user-memory/agent.py) implements a ReAct (Reasoning and Acting) loop optimized for persistent memory operations. This agent demonstrates how to build stateful assistants that can add, update, or delete user memories across conversation sessions.

Architectural Components

The agent composes several reusable components defined in separate modules:

• Config & MemoryMode (config.py) – Handle API keys, model defaults, and memory operation modes
• BaseMemoryManager (memory_manager.py) – Abstracts storage backends for notes and structured cards
• ConversationHistory (conversation_history.py) – Persists full conversation turns for context injection
• ToolCall – Data class for tracking tool invocations and results

The core loop builds a system prompt that injects current memory context, then iteratively calls the LLM with streaming requests while handling tool calls automatically.

Implementation Example

from chapter3.user_memory.agent import UserMemoryAgent, UserMemoryConfig

# Create a custom config (optional)

cfg = UserMemoryConfig(
    enable_memory_updates=True,
    memory_mode=MemoryMode.ENHANCED_NOTES,
    max_memory_context=5
)

# Initialise the agent for a given user

agent = UserMemoryAgent(
    user_id="user-42",
    provider="openrouter",
    model="google/gemini-3.5-flash",
    config=cfg,
    verbose=True
)

# Ask the agent to perform a task; the agent will add/update memories as needed

result = agent.execute_task(
    task="Remember that my favorite coffee is a flat white with oat milk.",
    max_iterations=10
)

print("Final answer:", result["final_answer"])
print("Tool calls performed:", result["tool_calls"])

MultimodalAgent: Processing Diverse Content Types

The MultimodalAgent (chapter3/multimodal-agent/agent.py) handles PDFs, images, and audio through three distinct extraction strategies. This implementation showcases how to abstract provider-specific handling while maintaining a unified interface for multimodal content.

Extraction Strategies

The agent supports three operational modes:

1. Native multimodal – Direct processing through the LLM's vision/audio capabilities
2. Extract-to-text – Conversion of media to text before processing
3. Tool-enabled mode – Using specialized functions (analyze_image, analyze_audio, analyze_pdf) defined in MultimodalTools

Cross-Provider Compatibility

As implemented in config.py, the agent abstracts differences between OpenAI, Gemini, Doubao, and OpenRouter providers. The MultimodalContent class provides a unified container for various media types, while the Message dataclass mirrors OpenAI's format and supports tool calls. The implementation also transparently adds thinking output for Gemini models when available.

Usage Example

import asyncio
from chapter3.multimodal_agent.agent import MultimodalAgent, MultimodalContent, ExtractionMode

async def demo():
    # Initialise the agent in native multimodal mode with tool support enabled

    agent = MultimodalAgent(
        model="gemini-3.5-flash",
        mode=ExtractionMode.NATIVE,
        enable_tools=True
    )

    # Ask the model to analyse an image directly

    answer = await agent.process_multimodal_content(
        MultimodalContent(type="image", path="sample_chart.png"),
        query="What does this chart show about Q2 revenue?"
    )
    print("Analysis:", answer)

    # Switch to extract-to-text mode and ask a higher-level question

    agent.extraction_mode = ExtractionMode.EXTRACT_TO_TEXT
    await agent.load_and_extract_content(
        MultimodalContent(type="pdf", path="annual_report.pdf")
    )
    async for chunk in agent.chat(
        message="Summarise the key financial highlights from the report.",
        multimodal_content=None,
        stream=True
    ):
        print(chunk, end="")

asyncio.run(demo())

SelfEvolvingAgent: Dynamic Tool Evolution

The SelfEvolvingAgent (chapter8/self-evolving-tools/agent.py) demonstrates advanced autonomy through runtime tool discovery and execution. This agent extends the base Agent pattern to enable self-improvement capabilities.

Graph-Based Coordination

Unlike the simpler memory agent, this implementation utilizes an AgentGraph memory manager for coordinating multiple sub-agents. The graph-based architecture allows the agent to maintain complex relationships between different tool capabilities and execution contexts.

Dynamic Tool Loading

The agent can dynamically load and execute new tool definitions at runtime, extending its capabilities without code changes. It reuses the same ToolCall tracking and streaming logic as the other agents, ensuring consistency across the framework while adding evolution capabilities.

Implementation Sketch

from chapter8.self_evolving_tools.agent import SelfEvolvingAgent

agent = SelfEvolvingAgent(
    user_id="alice",
    provider="openrouter",
    model="anthropic/claude-sonnet-4",
    verbose=True
)

# The agent can dynamically load new tool modules and invoke them.

# Here we start the evolution loop with a scheduling optimization goal.

result = agent.execute_task(
    task="Improve my daily schedule by suggesting time-blocking strategies.",
    max_iterations=5
)

print(result["final_answer"])

Shared Architectural Patterns

All three agents follow a consistent design methodology that simplifies extending the framework:

• Configuration Management – A Config object supplies API keys, model defaults, and per-provider settings through centralized configuration files.
• Message Abstraction – The Message dataclass normalizes role/content fields across providers and optionally carries tool_calls for function invocation tracking.
• Tool Definition & Dispatch – Agents expose tool_definitions lists that the LLM can invoke, with dispatchers like _execute_tool or MultimodalTools handling real execution.
• Streaming Interactions – The core loop uses client.chat.completions.create(..., stream=True) to surface model reasoning in real time.
• State Persistence – Memory managers (BaseMemoryManager, AgentGraph) and conversation history objects maintain context across turns or sessions.

These patterns make it straightforward to craft new agents: compose a system prompt, plug in a MemoryManager, declare desired tools, and let the streaming loop handle the rest.

Summary

The bojieli/ai-agent-book repository demonstrates production-ready agent development through three distinct implementations:

• UserMemoryAgent showcases persistent memory management with tool-based ReAct loops for long-term user context retention.
• MultimodalAgent provides a flexible architecture for processing PDFs, images, and audio across multiple LLM providers with unified abstractions.
• SelfEvolvingAgent illustrates advanced autonomy through dynamic tool loading and graph-based memory coordination.

Each implementation leverages shared components for configuration, streaming, and tool execution, providing a blueprint for building robust, stateful AI agents.

Frequently Asked Questions

What is the primary use case for UserMemoryAgent?

The UserMemoryAgent is designed for applications requiring persistent user context across multiple sessions. According to the source code in chapter3/user-memory/agent.py, it excels at maintaining structured memories—such as user preferences, past interactions, and personal facts—using a tool-based approach where the LLM decides when to add, update, or retrieve memory entries through function calls.

How does MultimodalAgent handle different file types?

The MultimodalAgent processes diverse media through the MultimodalContent class and MultimodalTools functions defined in chapter3/multimodal-agent/config.py. It supports three extraction modes: native multimodal processing for direct LLM analysis, extract-to-text conversion for preprocessing, and tool-enabled mode using specific functions like analyze_image, analyze_audio, and analyze_pdf for provider-agnostic handling.

Can SelfEvolvingAgent modify its own code?

While the SelfEvolvingAgent (chapter8/self-evolving-tools/agent.py) can dynamically load and execute new tool definitions at runtime, it operates within the constraints of the existing framework architecture. The agent extends its capabilities by loading external tool modules through the AgentGraph memory manager rather than modifying its own source code, enabling safe expansion of functionality without risking system integrity.

Which LLM providers are supported by these agents?

The framework supports multiple providers including OpenAI, Google Gemini, Doubao, and OpenRouter. Configuration in files like chapter3/multimodal-agent/config.py handles provider-specific mappings, API key management, and model defaults. The agents can fall back to alternative providers (such as OpenRouter) when primary provider credentials are unavailable, ensuring robust deployment across different environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}