How to Add New AI Providers to Calliope's Inference Engine: A Step-by-Step Guide

New AI providers are added to Calliope by extending the InferenceModelProvider enum, implementing provider-specific engine modules under calliope/inference/engines/, and routing requests through the dispatch logic in files like calliope/inference/text_to_text.py.

Calliope is an open-source inference orchestration framework that abstracts multiple AI vendors behind a unified, strategy-driven API. When integrating a new LLM, image generator, or multimodal service, the repository's provider-centric architecture standardizes how external APIs are consumed and authenticated. This guide demonstrates exactly how to add new AI providers to Calliope's inference engine using the actual source code patterns found in the chrisimmel/calliope repository.

Understanding Calliope's Provider-Centric Architecture

Before implementing a new integration, it is essential to understand the four-layer architecture that routes inference requests:

1. Provider Registry

The InferenceModelProvider enum in calliope/models/inference_model_config.py serves as the central registry of supported vendors. Every supported AI service—from OpenAI to HuggingFace—is declared here as a string constant, enabling type-safe provider selection throughout the codebase.

2. Model Configuration

The InferenceModel table (defined in calliope/tables/model_config.py) stores concrete model metadata including the provider_model_name and default parameters. This separation allows the same provider to host multiple models with distinct configurations.

3. Dispatch Logic

High-level inference entry points in calliope/inference/ handle request routing:

• text_to_text.py for language models
• text_to_image.py for image generation
• image_analysis.py for vision tasks
• text_to_video.py for video generation
• messages_to_object.py for structured output

These dispatchers select the appropriate engine implementation based on the provider stored in the ModelConfig.

4. Engine Modules

Concrete API implementations live in calliope/inference/engines/. Each file—such as openai_text.py, hugging_face.py, or replicate.py—encapsulates provider-specific authentication, request formatting, and response parsing.

Step-by-Step Implementation Guide

Follow these six steps to integrate a new AI service (referred to here as "MyAI") into Calliope's inference framework.

Step 1: Extend the Provider Enum

Add a new constant to the InferenceModelProvider enum to register the vendor within Calliope's type system.

File: calliope/models/inference_model_config.py

from enum import Enum

class InferenceModelProvider(str, Enum):
    HUGGINGFACE = "huggingface"
    STABILITY = "stability"
    OPENAI = "openai"
    AZURE = "azure"
    REPLICATE = "replicate"
    RUNWAY = "runway"
    MYAI = "myai"  # New provider

This allows the database layer and dispatch logic to recognize the vendor as a valid option.

Step 2: Create the Engine Module

Implement a new Python module containing async functions for each inference type you plan to support. The function signatures must match existing engines: they accept httpx.AsyncClient, input data (text or image), ModelConfig, and KeysModel.

File: calliope/inference/engines/myai.py (new file)

import httpx
from typing import Any, Dict

from calliope.models import KeysModel
from calliope.tables import ModelConfig

async def myai_text_to_text_inference(
    httpx_client: httpx.AsyncClient,
    text: str,
    model_config: ModelConfig,
    keys: KeysModel,
) -> str:
    """
    Example MyAI implementation for text completion.
    """
    model = model_config.model
    url = f"https://api.myai.com/v1/models/{model.provider_model_name}/completions"
    payload: Dict[str, Any] = {
        "prompt": text,
        **(model_config.model_parameters or {}),
    }
    headers = {"Authorization": f"Bearer {keys.myai_api_key}"}
    resp = await httpx_client.post(url, json=payload, headers=headers)
    resp.raise_for_status()
    return resp.json()["completion"]

Keep the public API minimal and consistent with existing patterns to simplify future refactors.

Step 3: Wire Into Dispatch Logic

Update every high-level dispatcher that should route to your new provider by adding an elif branch that imports and calls your engine functions.

File: calliope/inference/text_to_text.py

from calliope.inference.engines.myai import myai_text_to_text_inference

# Inside the main dispatch function:

elif model.provider == InferenceModelProvider.MYAI:
    print(f"text_to_text_inference.myai {model.provider_model_name}")
    extended_text = await myai_text_to_text_inference(
        httpx_client, text, model_config, keys
    )

Repeat this step for text_to_image.py, image_analysis.py, text_to_video.py, and messages_to_object.py if your provider supports those modalities.

Step 4: Configure API Keys

If the new service requires authentication, add the secret field to the KeysModel table.

File: calliope/models/keys.py

class KeysModel(Table, tablename="keys"):
    # ... existing fields ...

    myai_api_key = Varchar(length=255, null=True)

This makes the API key available to your engine module via the keys parameter without hardcoding secrets in source files.

Step 5: Update Database Migrations (Optional)

If you modified the database schema—such as adding the new enum value or key column—generate a Piccolo migration to ensure schema consistency across environments.

Directory: calliope/piccolo_migrations/

Run the Piccolo migration commands to autogenerate the migration files based on your table changes.

Step 6: Register the Model Configuration

Insert records into the inference_model and model_config tables to make the new provider selectable. This can be done via the admin UI, CLI, or seed scripts.

Example SQL:

INSERT INTO inference_model (slug, provider, provider_model_name, description)
VALUES ('myai-creative', 'myai', 'creative-v2', 'MyAI creative text model');

INSERT INTO model_config (slug, description, model_id, model_parameters)
VALUES (
    'myai-creative-config', 
    'Config for MyAI creative', 
    (SELECT id FROM inference_model WHERE slug='myai-creative'), 
    '{"max_tokens": 256}'
);

Any strategy referencing myai-creative-config will now route text-to-text calls through your new engine.

Key Implementation Files

When adding new AI providers to Calliope, you will primarily work with these files:

• calliope/models/inference_model_config.py – Defines the InferenceModelProvider enum
• calliope/inference/engines/ – Directory containing provider-specific implementations like openai_text.py and hugging_face.py
• calliope/inference/text_to_text.py – Dispatcher for language model requests
• calliope/inference/text_to_image.py – Dispatcher for image generation
• calliope/inference/image_analysis.py – Dispatcher for vision analysis
• calliope/models/keys.py – Secrets storage via KeysModel
• calliope/tables/model_config.py – Database schema for model configurations

Summary

Adding new AI providers to Calliope involves extending the type-safe provider registry, implementing modular engine code, and updating dispatch routing:

• Extend InferenceModelProvider in calliope/models/inference_model_config.py to register the vendor
• Create engine modules under calliope/inference/engines/ with async functions matching the established signatures
• Wire providers into dispatchers like calliope/inference/text_to_text.py using elif model.provider == InferenceModelProvider.NEW_PROVIDER
• Add API key fields to calliope/models/keys.py for secure authentication handling
• Configure model records in the database to make the provider selectable by strategies

This architecture ensures that new providers integrate seamlessly with Calliope's existing strategy-driven workflow while maintaining clean separation between API-specific logic and high-level inference orchestration.

Frequently Asked Questions

What is the minimum code required to add a new provider to Calliope?

At minimum, you must extend the InferenceModelProvider enum in calliope/models/inference_model_config.py with a new constant, create at least one engine function in a new file under calliope/inference/engines/, and add a routing condition in the relevant dispatcher (e.g., calliope/inference/text_to_text.py). If the provider requires authentication, you must also add the corresponding field to calliope/models/keys.py.

How does Calliope route inference requests to different providers?

Calliope uses a dispatcher pattern where high-level functions like text_to_text inspect the provider attribute of the ModelConfig object. Based on the provider value, the code executes an elif branch that calls the specific engine implementation for that vendor, such as functions in calliope/inference/engines/openai_text.py or your new custom engine module.

Where should authentication headers and API keys be handled?

Authentication logic belongs exclusively within the engine modules under calliope/inference/engines/. Retrieve the API key from the KeysModel object passed to your engine function (defined in calliope/models/keys.py), then construct headers or client instances locally. Never hardcode credentials in source files; always use the KeysModel abstraction.

Do I need to create database migrations when adding a new provider?

Database migrations are required only if you modify the schema, such as adding a new column to the KeysModel table for API keys or altering enum constraints. If you are simply adding a new provider constant to InferenceModelProvider and the database uses string storage for enums, you may not need a migration. However, for production deployments, generating a Piccolo migration in calliope/piccolo_migrations/ ensures schema consistency across environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}