# How to Add New AI Providers to Calliope's Inference Engine: A Step-by-Step Guide

> Add new AI providers to Calliope's inference engine. Extend the InferenceModelProvider enum and implement custom engine modules with this step-by-step guide.

- Repository: [chrisimmel/calliope](https://github.com/chrisimmel/calliope)
- Tags: how-to-guide
- Published: 2026-02-27

---

**New AI providers are added to Calliope by extending the `InferenceModelProvider` enum, implementing provider-specific engine modules under `calliope/inference/engines/`, and routing requests through the dispatch logic in files like [`calliope/inference/text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_text.py).**

Calliope is an open-source inference orchestration framework that abstracts multiple AI vendors behind a unified, strategy-driven API. When integrating a new LLM, image generator, or multimodal service, the repository's provider-centric architecture standardizes how external APIs are consumed and authenticated. This guide demonstrates exactly how to add new AI providers to Calliope's inference engine using the actual source code patterns found in the `chrisimmel/calliope` repository.

## Understanding Calliope's Provider-Centric Architecture

Before implementing a new integration, it is essential to understand the four-layer architecture that routes inference requests:

### 1. Provider Registry

The `InferenceModelProvider` enum in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) serves as the central registry of supported vendors. Every supported AI service—from OpenAI to HuggingFace—is declared here as a string constant, enabling type-safe provider selection throughout the codebase.

### 2. Model Configuration

The `InferenceModel` table (defined in [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py)) stores concrete model metadata including the `provider_model_name` and default parameters. This separation allows the same provider to host multiple models with distinct configurations.

### 3. Dispatch Logic

High-level inference entry points in `calliope/inference/` handle request routing:
- [`text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/text_to_text.py) for language models
- [`text_to_image.py`](https://github.com/chrisimmel/calliope/blob/main/text_to_image.py) for image generation
- [`image_analysis.py`](https://github.com/chrisimmel/calliope/blob/main/image_analysis.py) for vision tasks
- [`text_to_video.py`](https://github.com/chrisimmel/calliope/blob/main/text_to_video.py) for video generation
- [`messages_to_object.py`](https://github.com/chrisimmel/calliope/blob/main/messages_to_object.py) for structured output

These dispatchers select the appropriate engine implementation based on the provider stored in the `ModelConfig`.

### 4. Engine Modules

Concrete API implementations live in `calliope/inference/engines/`. Each file—such as [`openai_text.py`](https://github.com/chrisimmel/calliope/blob/main/openai_text.py), [`hugging_face.py`](https://github.com/chrisimmel/calliope/blob/main/hugging_face.py), or [`replicate.py`](https://github.com/chrisimmel/calliope/blob/main/replicate.py)—encapsulates provider-specific authentication, request formatting, and response parsing.

## Step-by-Step Implementation Guide

Follow these six steps to integrate a new AI service (referred to here as "MyAI") into Calliope's inference framework.

### Step 1: Extend the Provider Enum

Add a new constant to the `InferenceModelProvider` enum to register the vendor within Calliope's type system.

**File:** [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py)

```python
from enum import Enum

class InferenceModelProvider(str, Enum):
    HUGGINGFACE = "huggingface"
    STABILITY = "stability"
    OPENAI = "openai"
    AZURE = "azure"
    REPLICATE = "replicate"
    RUNWAY = "runway"
    MYAI = "myai"  # New provider

```

This allows the database layer and dispatch logic to recognize the vendor as a valid option.

### Step 2: Create the Engine Module

Implement a new Python module containing async functions for each inference type you plan to support. The function signatures must match existing engines: they accept `httpx.AsyncClient`, input data (text or image), `ModelConfig`, and `KeysModel`.

**File:** [`calliope/inference/engines/myai.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/engines/myai.py) (new file)

```python
import httpx
from typing import Any, Dict

from calliope.models import KeysModel
from calliope.tables import ModelConfig

async def myai_text_to_text_inference(
    httpx_client: httpx.AsyncClient,
    text: str,
    model_config: ModelConfig,
    keys: KeysModel,
) -> str:
    """
    Example MyAI implementation for text completion.
    """
    model = model_config.model
    url = f"https://api.myai.com/v1/models/{model.provider_model_name}/completions"
    payload: Dict[str, Any] = {
        "prompt": text,
        **(model_config.model_parameters or {}),
    }
    headers = {"Authorization": f"Bearer {keys.myai_api_key}"}
    resp = await httpx_client.post(url, json=payload, headers=headers)
    resp.raise_for_status()
    return resp.json()["completion"]

```

Keep the public API minimal and consistent with existing patterns to simplify future refactors.

### Step 3: Wire Into Dispatch Logic

Update every high-level dispatcher that should route to your new provider by adding an `elif` branch that imports and calls your engine functions.

**File:** [`calliope/inference/text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_text.py)

```python
from calliope.inference.engines.myai import myai_text_to_text_inference

# Inside the main dispatch function:

elif model.provider == InferenceModelProvider.MYAI:
    print(f"text_to_text_inference.myai {model.provider_model_name}")
    extended_text = await myai_text_to_text_inference(
        httpx_client, text, model_config, keys
    )

```

Repeat this step for [`text_to_image.py`](https://github.com/chrisimmel/calliope/blob/main/text_to_image.py), [`image_analysis.py`](https://github.com/chrisimmel/calliope/blob/main/image_analysis.py), [`text_to_video.py`](https://github.com/chrisimmel/calliope/blob/main/text_to_video.py), and [`messages_to_object.py`](https://github.com/chrisimmel/calliope/blob/main/messages_to_object.py) if your provider supports those modalities.

### Step 4: Configure API Keys

If the new service requires authentication, add the secret field to the `KeysModel` table.

**File:** [`calliope/models/keys.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/keys.py)

```python
class KeysModel(Table, tablename="keys"):
    # ... existing fields ...

    myai_api_key = Varchar(length=255, null=True)

```

This makes the API key available to your engine module via the `keys` parameter without hardcoding secrets in source files.

### Step 5: Update Database Migrations (Optional)

If you modified the database schema—such as adding the new enum value or key column—generate a Piccolo migration to ensure schema consistency across environments.

**Directory:** `calliope/piccolo_migrations/`

Run the Piccolo migration commands to autogenerate the migration files based on your table changes.

### Step 6: Register the Model Configuration

Insert records into the `inference_model` and `model_config` tables to make the new provider selectable. This can be done via the admin UI, CLI, or seed scripts.

**Example SQL:**

```sql
INSERT INTO inference_model (slug, provider, provider_model_name, description)
VALUES ('myai-creative', 'myai', 'creative-v2', 'MyAI creative text model');

INSERT INTO model_config (slug, description, model_id, model_parameters)
VALUES (
    'myai-creative-config', 
    'Config for MyAI creative', 
    (SELECT id FROM inference_model WHERE slug='myai-creative'), 
    '{"max_tokens": 256}'
);

```

Any strategy referencing `myai-creative-config` will now route text-to-text calls through your new engine.

## Key Implementation Files

When adding new AI providers to Calliope, you will primarily work with these files:

- **[`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py)** – Defines the `InferenceModelProvider` enum
- **`calliope/inference/engines/`** – Directory containing provider-specific implementations like [`openai_text.py`](https://github.com/chrisimmel/calliope/blob/main/openai_text.py) and [`hugging_face.py`](https://github.com/chrisimmel/calliope/blob/main/hugging_face.py)
- **[`calliope/inference/text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_text.py)** – Dispatcher for language model requests
- **[`calliope/inference/text_to_image.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_image.py)** – Dispatcher for image generation
- **[`calliope/inference/image_analysis.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/image_analysis.py)** – Dispatcher for vision analysis
- **[`calliope/models/keys.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/keys.py)** – Secrets storage via `KeysModel`
- **[`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py)** – Database schema for model configurations

## Summary

Adding new AI providers to Calliope involves extending the type-safe provider registry, implementing modular engine code, and updating dispatch routing:

- Extend `InferenceModelProvider` in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) to register the vendor
- Create engine modules under `calliope/inference/engines/` with async functions matching the established signatures
- Wire providers into dispatchers like [`calliope/inference/text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_text.py) using `elif model.provider == InferenceModelProvider.NEW_PROVIDER`
- Add API key fields to [`calliope/models/keys.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/keys.py) for secure authentication handling
- Configure model records in the database to make the provider selectable by strategies

This architecture ensures that new providers integrate seamlessly with Calliope's existing strategy-driven workflow while maintaining clean separation between API-specific logic and high-level inference orchestration.

## Frequently Asked Questions

### What is the minimum code required to add a new provider to Calliope?

At minimum, you must extend the `InferenceModelProvider` enum in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) with a new constant, create at least one engine function in a new file under `calliope/inference/engines/`, and add a routing condition in the relevant dispatcher (e.g., [`calliope/inference/text_to_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_text.py)). If the provider requires authentication, you must also add the corresponding field to [`calliope/models/keys.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/keys.py).

### How does Calliope route inference requests to different providers?

Calliope uses a dispatcher pattern where high-level functions like `text_to_text` inspect the `provider` attribute of the `ModelConfig` object. Based on the provider value, the code executes an `elif` branch that calls the specific engine implementation for that vendor, such as functions in [`calliope/inference/engines/openai_text.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/engines/openai_text.py) or your new custom engine module.

### Where should authentication headers and API keys be handled?

Authentication logic belongs exclusively within the engine modules under `calliope/inference/engines/`. Retrieve the API key from the `KeysModel` object passed to your engine function (defined in [`calliope/models/keys.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/keys.py)), then construct headers or client instances locally. Never hardcode credentials in source files; always use the `KeysModel` abstraction.

### Do I need to create database migrations when adding a new provider?

Database migrations are required only if you modify the schema, such as adding a new column to the `KeysModel` table for API keys or altering enum constraints. If you are simply adding a new provider constant to `InferenceModelProvider` and the database uses string storage for enums, you may not need a migration. However, for production deployments, generating a Piccolo migration in `calliope/piccolo_migrations/` ensures schema consistency across environments.