# How to Integrate Custom AI Models with the AI Agent Book: A Complete Guide

> Integrate custom AI models with the AI Agent Book easily. Learn to set environment variables, configure SDK clients, and extend mapping helpers for seamless integration. Get started now!

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: how-to-guide
- Published: 2026-07-29

---

**Integrate custom AI models with the AI Agent Book by setting the `OPENAI_MODEL` environment variable to your model identifier, ensuring the SDK client supports OpenAI-compatible JSON schema, and optionally extending the mapping helper in [`pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/pine_voice.py) for non-standard provider formats.**

The AI Agent Book by bojieli provides a modular framework built on the formula **Agent = LLM + Context + Tools**. Whether you need to swap GPT-4 for Claude, Gemini, or a self-hosted endpoint, the repository's environment-driven configuration allows you to integrate custom AI models without modifying core agent logic.

## Understanding the Model Selection Architecture

The codebase normalizes model selection through a consistent pattern across all runnable experiments. In [`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py), the `default_model()` function reads environment variables to determine which LLM to invoke, while `_map_openrouter_model` handles translation between OpenAI and OpenRouter identifier formats.

### Environment Variable Configuration

The system prioritizes model identifiers through a cascading lookup:

1. **`OPENAI_MODEL`** - Primary override variable
2. **`LLM_MODEL`** - Secondary fallback
3. **Hardcoded defaults** (`gpt-5.6-luna`, `gpt-audio`) - Final fallback

This hierarchy ensures you can override the default model at runtime without touching source files.

### Model Identifier Mapping

The `_map_openrouter_model` function in [`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py) normalizes model identifiers. It accepts both OpenAI-style names (`gpt-4`) and OpenRouter syntax (`openai/gpt-4`), translating them to the format required by the underlying SDK client.

## SDK Client Integration Pattern

Once selected, the model string passes directly to compatible SDK clients. For example, in [`chapter9/end-to-end-speech/speech_model.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/end-to-end-speech/speech_model.py), the code forwards the model identifier to `client.audio.speech.create(model=tts_model, ...)`【/cache/repos/github.com/bojieli/ai-agent-book/main/chapter9/end-to-end-speech/speech_model.py】. This implementation assumes the client library adheres to OpenAI-compatible JSON schema, enabling seamless swapping between providers like Anthropic, Google, or custom endpoints.

## Step-by-Step Integration Guide

Follow these steps to connect your custom AI model to any demo in the repository.

### 1. Install the Model's Python SDK

Install the appropriate client library for your provider. The book's README provides general instructions in the API Keys section【/cache/repos/github.com/bojieli/ai-agent-book/main/README.md#L68-L80】.

```bash
pip install anthropic

```

### 2. Configure API Authentication

Add your API key to the environment or a `.env` file. The repository respects standard `dotenv` loading patterns.

```bash
export ANTHROPIC_API_KEY=sk-...

```

### 3. Set the Model Identifier

Export the model name using the `OPENAI_MODEL` variable. The `default_model()` helper automatically reads this value.

```bash
export OPENAI_MODEL=claude-3-5-sonnet-20240620

```

### 4. Extend the Mapping Helper (Optional)

If your provider uses a non-standard identifier format, modify `_map_openrouter_model` in [`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py) to recognize the new prefix.

### 5. Run Existing Demos

Execute any demo that uses `default_model()`, such as [`chapter9/phone-agent/demo.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/demo.py) or [`chapter9/end-to-end-speech/demo.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/end-to-end-speech/demo.py). The demo will automatically invoke your custom model.

```bash
python chapter9/phone-agent/demo.py --model $OPENAI_MODEL

```

## Example: Integrating Claude 3.5 Sonnet

Here is a complete workflow for replacing the default GPT model with Anthropic's Claude 3.5 Sonnet:

```bash

# Install the SDK

pip install anthropic

# Configure credentials

export ANTHROPIC_API_KEY=sk-ant-api03-...

# Select the model (mapper automatically adds "anthropic/" prefix if needed)

export OPENAI_MODEL=claude-3-5-sonnet-20240620

# Run the phone-call demo with the custom model

python chapter9/phone-agent/demo.py --model $OPENAI_MODEL

```

The mapper translates `claude-3-5-sonnet-20240620` to `anthropic/claude-3-5-sonnet-20240620`, and the wrapped `OpenAIClient` routes the request to Anthropic's endpoint.

## Key Files for Custom Model Integration

Understanding these source files helps troubleshoot integration issues:

- **[`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py)** - Contains `default_model()` and `_map_openrouter_model` functions that handle model selection and identifier normalization.
- **[`chapter9/end-to-end-speech/speech_model.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/end-to-end-speech/speech_model.py)** - Demonstrates how selected model strings propagate to API clients.
- **[`chapter9/phone-agent/demo.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/demo.py)** - Entry point showcasing real-time model switching.
- **[`README.md`](https://github.com/bojieli/ai-agent-book/blob/main/README.md)** - Documents supported API providers and authentication patterns【/cache/repos/github.com/bojieli/ai-agent-book/main/README.md】.

## Summary

- **Environment variables drive model selection**: Set `OPENAI_MODEL` to override defaults without code changes.
- **Identifier mapping is automatic**: The `_map_openrouter_model` function handles translation between OpenAI and OpenRouter formats.
- **Zero core modifications required**: Install the SDK, set the key, and run existing demos immediately.
- **Compatible clients only**: Ensure your custom model exposes an OpenAI-compatible JSON schema interface.
- **Optional mapper extensions**: Add new provider prefixes in [`pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/pine_voice.py) for non-standard naming conventions.

## Frequently Asked Questions

### Can I use local models like Ollama or LM Studio?

Yes, provided the local server exposes an OpenAI-compatible API endpoint. Set `OPENAI_MODEL` to your local model identifier and configure the base URL in the client initialization within [`pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/pine_voice.py). The `default_model()` helper will pass your local model name directly to the client.

### Do I need to modify the agent logic to switch models?

No. The AI Agent Book abstracts model selection through the `default_model()` helper in [`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py). As long as the new model supports the expected JSON schema for chat completions, existing demos work without modifications to the agent reasoning logic.

### Why does the book use `OPENAI_MODEL` for non-OpenAI providers?

The variable name maintains backward compatibility with existing OpenAI SDK patterns while the `_map_openrouter_model` function handles provider-specific routing internally. You can use `LLM_MODEL` as an alternative environment variable if you prefer a provider-agnostic naming convention.

### How do I debug model selection issues?

Check the return value of `default_model()` in [`chapter9/phone-agent/pine_voice.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter9/phone-agent/pine_voice.py). Verify that your environment variable is loaded correctly and that the mapper produces the expected provider-prefixed identifier (e.g., `anthropic/claude-3-opus`) for your SDK client. Most integration failures occur when the mapper output does not match the client's expected model string format.