# How to Add Custom LLM Model Adapters to Harvey AI: Step-by-Step Guide

> Learn how to add custom LLM model adapters to Harvey AI beyond built-in providers. This guide details using the pluggable ModelAdapter abstraction layer for seamless integration. Integrate any LLM provider today.

- Repository: [Harvey/harvey-labs](https://github.com/harveyai/harvey-labs)
- Tags: how-to-guide
- Published: 2026-08-11

---

**Harvey AI uses a pluggable `ModelAdapter` abstraction layer in [`harness/adapters/base.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/base.py) that lets you integrate any LLM provider by subclassing the base class and implementing four abstract methods.**

Harvey AI ships with built-in support for Anthropic, OpenAI, Google, Mistral, Fireworks, and Baseten. When you need to connect a proprietary or niche LLM service, the adapter pattern in the harveyai/harvey-labs repository provides a clean, documented interface without modifying core agent logic.

## Understanding the ModelAdapter Architecture

The adapter system lives in `harness/adapters/`. The **base class** ([`base.py`](https://github.com/harveyai/harvey-labs/blob/main/base.py)) defines a stable contract that the **agent loop**, **evaluation scripts**, and **tooling layer** all depend on. Each provider implementation isolates API-specific code in its own module.

Key design benefits of this approach:

- **Separation of concerns**: Harvey's core logic works with canonical message formats; adapters handle provider-specific translations.
- **Testability**: Each adapter can be unit-tested independently against recorded API responses.
- **Extensibility**: New providers require no changes to [`harness/agent_loop.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/agent_loop.py) or tool definitions.

## The Four Methods Every Custom Adapter Must Implement

Your subclass must implement these abstract methods from `ModelAdapter`:

### `chat(messages, tools) → ModelResponse`

This is the primary inference method. It accepts:

- `messages`: A list of standardized message dicts (system, user, assistant roles).
- `tools`: Canonical JSON-Schema tool definitions from [`harness/tools.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/tools.py).

It returns a `ModelResponse` containing the assistant message, any tool calls, extracted text, and token usage counts.

### `make_tool_result_messages(results) → list[dict]`

Transforms `(tool_call_id, result)` tuples into provider-specific result messages. Different providers handle tool results differently—Anthropic requires a single batched user message, while OpenAI and Google accept one message per result.

### `make_system_message(content) → dict`

Returns a system-role message formatted for the provider's native schema.

### `make_user_message(content) → dict`

Returns a user-role message formatted for the provider's native schema.

## Step-by-Step: Creating a Custom LLM Adapter

### Step 1: Create the Adapter Module

Create a new file in `harness/adapters/` following the naming convention of existing adapters:

```bash
touch harness/adapters/foo.py

```

### Step 2: Subclass ModelAdapter and Implement Required Methods

Below is a minimal, fully-functional template based on patterns from [`harness/adapters/openai.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/openai.py) and [`harness/adapters/anthropic.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/anthropic.py):

```python

# File: harness/adapters/foo.py

"""FooAI adapter – a template for adding custom LLM providers."""

import json
from harness.adapters.base import ModelAdapter, ModelResponse, ToolCall


class FooAIAdapter(ModelAdapter):
    """Adapter for FooAI's language-model API."""

    def __init__(
        self,
        model: str,
        temperature: float = 0.0,
        max_tokens: int = 4096,
        reasoning_effort: str | None = None,
    ):
        super().__init__(model, temperature, reasoning_effort)
        self.max_tokens = max_tokens
        # Initialize your provider's SDK client here

        self.client = self._initialize_client()
        self._system_prompt: str | None = None

    def chat(self, messages: list[dict], tools: list[dict]) -> ModelResponse:
        """Execute chat completion with tool support."""
        # Extract system prompt from message history

        system = ""
        conversation_messages = []
        for msg in messages:
            if msg["role"] == "system":
                system = msg["content"]
            else:
                conversation_messages.append(msg)

        # Convert canonical tool definitions to provider format

        provider_tools = [self._translate_tool(t) for t in tools]

        # Execute provider-specific API call

        raw_response = self._call_provider_api(
            system=system,
            messages=conversation_messages,
            tools=provider_tools,
        )

        # Parse provider response into canonical structures

        tool_calls: list[ToolCall] = []
        text_parts: list[str] = []

        for item in raw_response["output"]:
            if item["type"] == "tool_call":
                tool_calls.append(
                    ToolCall(
                        id=item["call_id"],
                        name=item["name"],
                        arguments=json.dumps(item["arguments"]),
                    )
                )
            elif item["type"] == "text":
                text_parts.append(item["content"])

        # Construct message for conversation history

        assistant_message = {
            "role": "assistant",
            "content": raw_response["output"],
        }

        return ModelResponse(
            message=assistant_message,
            tool_calls=tool_calls,
            text="\n".join(text_parts),
            input_tokens=raw_response.get("usage", {}).get("input_tokens", 0),
            output_tokens=raw_response.get("usage", {}).get("output_tokens", 0),
        )

    def make_tool_result_messages(
        self, results: list[tuple[str, str]]
    ) -> list[dict]:
        """Format tool execution results for provider's expected structure."""
        messages = []
        for tool_call_id, result in results:
            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": tool_call_id,
                    "content": str(result),
                }
            )
        return messages

    def make_system_message(self, content: str) -> dict:
        """Create system message in canonical format."""
        self._system_prompt = content
        return {"role": "system", "content": content}

    def make_user_message(self, content: str) -> dict:
        """Create user message in canonical format."""
        return {"role": "user", "content": content}

    def _translate_tool(self, tool: dict) -> dict:
        """Convert canonical tool definition to provider-specific schema."""
        return {
            "type": "function",
            "function": {
                "name": tool["name"],
                "description": tool["description"],
                "parameters": tool["parameters"],
            },
        }

    def _initialize_client(self):
        """Initialize provider SDK client. Override with actual implementation."""
        raise NotImplementedError("Configure your provider's client initialization")

    def _call_provider_api(self, system: str, messages: list, tools: list) -> dict:
        """Execute actual API call to provider. Override with implementation."""
        raise NotImplementedError("Implement provider-specific API call")

```

### Step 3: Handle Tool Definition Translation

The **canonical tool format** used throughout Harvey AI follows JSON-Schema conventions defined in [`harness/tools.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/tools.py). Your adapter's `_translate_tool` method maps this to your provider's expected structure.

Reference implementations to study:

- **[`harness/adapters/openai.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/openai.py)**: Maps to OpenAI's function-calling schema with strict mode support.
- **[`harness/adapters/anthropic.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/anthropic.py)**: Maps to Anthropic's tool use format with `tool_choice` handling.

### Step 4: Register Your Adapter

Harvey AI discovers adapters through the `model_provider` configuration field. The harness imports the module matching your provider name and instantiates the corresponding adapter class.

Two registration patterns are supported:

**Automatic discovery** (recommended): Name your file [`foo.py`](https://github.com/harveyai/harvey-labs/blob/main/foo.py) and class `FooAIAdapter`. Set `model_provider: "foo"` in your config.

**Explicit registration**: Add an import mapping in [`harness/__init__.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/__init__.py) or your configuration loader:

```python
from harness.adapters.foo import FooAIAdapter

ADAPTER_REGISTRY = {
    "openai": OpenAIAdapter,
    "anthropic": AnthropicAdapter,
    "foo": FooAIAdapter,  # your custom adapter

}

```

### Step 5: Test Your Adapter

Create tests in [`tests/test_adapters.py`](https://github.com/harveyai/harvey-labs/blob/main/tests/test_adapters.py) following patterns from existing adapter tests:

```python

# tests/test_adapters.py additions

import pytest
from harness.adapters.foo import FooAIAdapter
from harness.adapters.base import ModelResponse, ToolCall


def test_foo_adapter_instantiation():
    """Verify adapter can be created with standard parameters."""
    adapter = FooAIAdapter(model="foo-large-v1", temperature=0.2)
    assert adapter.model == "foo-large-v1"
    assert adapter.temperature == 0.2


def test_foo_adapter_make_messages():
    """Verify system and user message formatting."""
    adapter = FooAIAdapter(model="foo-large-v1")
    
    system_msg = adapter.make_system_message("Be helpful.")
    assert system_msg["role"] == "system"
    assert system_msg["content"] == "Be helpful."
    
    user_msg = adapter.make_user_message("Hello!")
    assert user_msg["role"] == "user"


def test_foo_adapter_tool_results():
    """Verify tool result message formatting matches provider expectations."""
    adapter = FooAIAdapter(model="foo-large-v1")
    results = [("call_123", "result data"), ("call_456", "more data")]
    
    messages = adapter.make_tool_result_messages(results)
    assert len(messages) == 2
    assert all(m["role"] == "tool" for m in messages)
    assert messages[0]["tool_call_id"] == "call_123"

```

## Using Your Custom LLM Adapter

Once implemented, use your adapter identically to built-in providers:

```python
from harness.adapters.foo import FooAIAdapter
from harness.tools import TOOL_DEFINITIONS

# Initialize adapter

adapter = FooAIAdapter(
    model="foo-large-v1",
    temperature=0.2,
    max_tokens=4096,
)

# Build conversation

messages = [
    adapter.make_system_message("You are a legal research assistant."),
    adapter.make_user_message("Draft a contract clause for IP assignment."),
]

# Execute with tool support

response = adapter.chat(messages, TOOL_DEFINITIONS)

print(f"Response: {response.text}")
print(f"Tokens used: {response.input_tokens} in, {response.output_tokens} out")

if response.tool_calls:
    for call in response.tool_calls:
        print(f"Tool requested: {call.name}({call.arguments})")

```

## Key Source Files for Reference

| File | Purpose |
|------|---------|
| [`harness/adapters/base.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/base.py) | Abstract `ModelAdapter`, `ModelResponse`, and `ToolCall` dataclasses |
| [`harness/adapters/openai.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/openai.py) | OpenAI Responses API implementation with streaming support |
| [`harness/adapters/anthropic.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/anthropic.py) | Anthropic Messages API with adaptive-thinking and batch tool results |
| [`harness/agent_loop.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/agent_loop.py) | Adapter instantiation and conversation orchestration |
| [`tests/test_adapters.py`](https://github.com/harveyai/harvey-labs/blob/main/tests/test_adapters.py) | Contract tests for all adapter implementations |

## Summary

Adding custom LLM model adapters to Harvey AI requires implementing a four-method contract defined in [`harness/adapters/base.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/base.py):

- **`chat()`** handles inference and response parsing.
- **`make_tool_result_messages()`** formats tool execution feedback.
- **`make_system_message()`** and **`make_user_message()`** create role-formatted messages.

The adapter pattern isolates provider-specific code, maintains testability, and requires no changes to Harvey's core agent logic. Study [`harness/adapters/openai.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/openai.py) and [`harness/adapters/anthropic.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/anthropic.py) for production reference implementations.

## Frequently Asked Questions

### What happens if my LLM provider doesn't support tool calling?

Implement `chat()` to return an empty `tool_calls` list and raise an appropriate error if tools are passed. Your adapter can still function for non-tool use cases, or you can implement tool-calling via prompt-based techniques. The [`harness/adapters/base.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/base.py) interface does not require functional tool support.

### Can I add streaming support to my custom adapter?

Yes. The base `ModelAdapter` can be extended with streaming methods. Study event-streaming patterns in [`harness/adapters/openai.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/openai.py) for the `stream=True` parameter handling. Ensure your `ModelResponse` accumulation logic handles partial chunks correctly.

### How do I handle provider-specific authentication in my adapter?

Initialize credentials in your adapter's `__init__` method or via environment variables. Follow the pattern in [`harness/adapters/anthropic.py`](https://github.com/harveyai/harvey-labs/blob/main/harness/adapters/anthropic.py), which retrieves API keys through `os.environ.get("ANTHROPIC_API_KEY")` with clear error messages on missing configuration.

### Does Harvey AI support multiple custom adapters simultaneously?

Yes. The configuration system selects adapters per-conversation or per-deployment. Register multiple custom providers following the naming conventions, then specify the desired `model_provider` in your runtime configuration or environment.