How to Integrate Custom AI Models with the AI Agent Book: A Complete Guide

Integrate custom AI models with the AI Agent Book by setting the OPENAI_MODEL environment variable to your model identifier, ensuring the SDK client supports OpenAI-compatible JSON schema, and optionally extending the mapping helper in pine_voice.py for non-standard provider formats.

The AI Agent Book by bojieli provides a modular framework built on the formula Agent = LLM + Context + Tools. Whether you need to swap GPT-4 for Claude, Gemini, or a self-hosted endpoint, the repository's environment-driven configuration allows you to integrate custom AI models without modifying core agent logic.

Understanding the Model Selection Architecture

The codebase normalizes model selection through a consistent pattern across all runnable experiments. In chapter9/phone-agent/pine_voice.py, the default_model() function reads environment variables to determine which LLM to invoke, while _map_openrouter_model handles translation between OpenAI and OpenRouter identifier formats.

Environment Variable Configuration

The system prioritizes model identifiers through a cascading lookup:

1. OPENAI_MODEL - Primary override variable
2. LLM_MODEL - Secondary fallback
3. Hardcoded defaults (gpt-5.6-luna, gpt-audio) - Final fallback

This hierarchy ensures you can override the default model at runtime without touching source files.

Model Identifier Mapping

The _map_openrouter_model function in chapter9/phone-agent/pine_voice.py normalizes model identifiers. It accepts both OpenAI-style names (gpt-4) and OpenRouter syntax (openai/gpt-4), translating them to the format required by the underlying SDK client.

SDK Client Integration Pattern

Once selected, the model string passes directly to compatible SDK clients. For example, in chapter9/end-to-end-speech/speech_model.py, the code forwards the model identifier to client.audio.speech.create(model=tts_model, ...)【/cache/repos/github.com/bojieli/ai-agent-book/main/chapter9/end-to-end-speech/speech_model.py】. This implementation assumes the client library adheres to OpenAI-compatible JSON schema, enabling seamless swapping between providers like Anthropic, Google, or custom endpoints.

Step-by-Step Integration Guide

Follow these steps to connect your custom AI model to any demo in the repository.

1. Install the Model's Python SDK

Install the appropriate client library for your provider. The book's README provides general instructions in the API Keys section【/cache/repos/github.com/bojieli/ai-agent-book/main/README.md#L68-L80】.

pip install anthropic

2. Configure API Authentication

Add your API key to the environment or a .env file. The repository respects standard dotenv loading patterns.

export ANTHROPIC_API_KEY=sk-...

3. Set the Model Identifier

Export the model name using the OPENAI_MODEL variable. The default_model() helper automatically reads this value.

export OPENAI_MODEL=claude-3-5-sonnet-20240620

4. Extend the Mapping Helper (Optional)

If your provider uses a non-standard identifier format, modify _map_openrouter_model in chapter9/phone-agent/pine_voice.py to recognize the new prefix.

5. Run Existing Demos

Execute any demo that uses default_model(), such as chapter9/phone-agent/demo.py or chapter9/end-to-end-speech/demo.py. The demo will automatically invoke your custom model.

python chapter9/phone-agent/demo.py --model $OPENAI_MODEL

Example: Integrating Claude 3.5 Sonnet

Here is a complete workflow for replacing the default GPT model with Anthropic's Claude 3.5 Sonnet:

# Install the SDK

pip install anthropic

# Configure credentials

export ANTHROPIC_API_KEY=sk-ant-api03-...

# Select the model (mapper automatically adds "anthropic/" prefix if needed)

export OPENAI_MODEL=claude-3-5-sonnet-20240620

# Run the phone-call demo with the custom model

python chapter9/phone-agent/demo.py --model $OPENAI_MODEL

The mapper translates claude-3-5-sonnet-20240620 to anthropic/claude-3-5-sonnet-20240620, and the wrapped OpenAIClient routes the request to Anthropic's endpoint.

Key Files for Custom Model Integration

Understanding these source files helps troubleshoot integration issues:

• chapter9/phone-agent/pine_voice.py - Contains default_model() and _map_openrouter_model functions that handle model selection and identifier normalization.
• chapter9/end-to-end-speech/speech_model.py - Demonstrates how selected model strings propagate to API clients.
• chapter9/phone-agent/demo.py - Entry point showcasing real-time model switching.
• README.md - Documents supported API providers and authentication patterns【/cache/repos/github.com/bojieli/ai-agent-book/main/README.md】.

Summary

• Environment variables drive model selection: Set OPENAI_MODEL to override defaults without code changes.
• Identifier mapping is automatic: The _map_openrouter_model function handles translation between OpenAI and OpenRouter formats.
• Zero core modifications required: Install the SDK, set the key, and run existing demos immediately.
• Compatible clients only: Ensure your custom model exposes an OpenAI-compatible JSON schema interface.
• Optional mapper extensions: Add new provider prefixes in pine_voice.py for non-standard naming conventions.

Frequently Asked Questions

Can I use local models like Ollama or LM Studio?

Yes, provided the local server exposes an OpenAI-compatible API endpoint. Set OPENAI_MODEL to your local model identifier and configure the base URL in the client initialization within pine_voice.py. The default_model() helper will pass your local model name directly to the client.

Do I need to modify the agent logic to switch models?

No. The AI Agent Book abstracts model selection through the default_model() helper in chapter9/phone-agent/pine_voice.py. As long as the new model supports the expected JSON schema for chat completions, existing demos work without modifications to the agent reasoning logic.

Why does the book use OPENAI_MODEL for non-OpenAI providers?

The variable name maintains backward compatibility with existing OpenAI SDK patterns while the _map_openrouter_model function handles provider-specific routing internally. You can use LLM_MODEL as an alternative environment variable if you prefer a provider-agnostic naming convention.

How do I debug model selection issues?

Check the return value of default_model() in chapter9/phone-agent/pine_voice.py. Verify that your environment variable is loaded correctly and that the mapper produces the expected provider-prefixed identifier (e.g., anthropic/claude-3-opus) for your SDK client. Most integration failures occur when the mapper output does not match the client's expected model string format.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}