# How the Calliope Inference Engine Architecture Works: A 4-Layer Provider-Agnostic Pipeline

> Explore the 4-layer provider-agnostic Calliope inference engine architecture. Discover how it unifies multimodal requests through a Python API with a model registry, dynamic dispatch, and provider engines.

- Repository: [chrisimmel/calliope](https://github.com/chrisimmel/calliope)
- Tags: architecture
- Published: 2026-02-27

---

**Calliope implements a provider-agnostic inference pipeline that routes multimodal requests through a unified Python API by combining a runtime model registry, persistent database configuration, strategy-based dispatch, and provider-specific engines.**

The Calliope inference engine architecture powers the `chrisimmel/calliope` repository's multimodal story generation capabilities. This design abstracts away provider-specific implementation details, allowing developers to switch between OpenAI, Stability AI, Runway, and other providers without changing application code.

## The Four Layers of the Calliope Inference Engine Architecture

### 1. Model Registry

The foundation of the architecture is the **model registry** defined in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py). This module declares the `InferenceModelProvider` enum (covering HuggingFace, Stability, OpenAI, Azure, Replicate, and Runway) and the `InferenceModelProviderVariant` enum for API flavors.

The registry itself is the `_model_configs_by_name` dictionary, which maps logical model names to `InferenceModelConfigModel` instances. Each entry specifies:

- **provider** – The hosting service.
- **provider_variant** – Optional API flavor (e.g., OpenAI chat vs. completion).
- **provider_model_name** – The exact identifier the provider expects.
- **parameters** – Default request parameters (temperature, steps, etc.).

```python

# Example entry from inference_model_config.py

"stability_stable_diffusion_1.5": InferenceModelConfigModel(
    provider=InferenceModelProvider.STABILITY,
    provider_model_name="stable-diffusion-v1-5",
    parameters={"steps": 30, "seed": 0, "cfg_scale": 7.0},
)

```

The helper function `load_model_configs()` builds an `InferenceModelConfigsModel` that the application injects at startup.

### 2. Persistent Model Configuration

While the registry defines what *can* be used, [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py) persists what *is* being used. This layer uses Piccolo ORM tables to store per-deployment selections:

- **`InferenceModel`** – The database representation of a registry entry.
- **`ModelConfig`** – Links an `InferenceModel` to a specific prompt template and parameter overrides.
- **`StrategyConfig`** – Selects which `ModelConfig` to use for each inference modality (text-to-text, text-to-image, text-to-video, etc.).

```python
class ModelConfig(Table):
    slug = Varchar(length=80, unique=True, index=True)
    model = ForeignKey(references=InferenceModel)
    prompt_template = ForeignKey(references=PromptTemplate, null=True)
    model_parameters = JSONB(null=True)  # Runtime overrides

```

Administrators can switch models by updating database rows rather than deploying new code.

### 3. Strategy and Dispatch Layer

The public API surface lives in [`calliope/inference/__init__.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/__init__.py), which re-exports concrete functions:

```python
from .text_to_image import text_to_image_file_inference
from .text_to_text import text_to_text_inference
from .text_to_video import image_and_text_to_video_file_inference

```

When a client calls `text_to_image_file_inference`, the dispatcher:

1. Resolves the supplied `ModelConfig`.
2. Retrieves the linked `InferenceModel` to read the `provider`.
3. Forwards the request to the correct provider engine based on `model.provider`.
4. Handles retries, content censoring, and error propagation.

The dispatcher logic in [`calliope/inference/text_to_image.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_image.py) illustrates this routing:

```python
if model.provider == InferenceModelProvider.REPLICATE:
    return await text_to_image_file_inference_replicate(...)
elif model.provider == InferenceModelProvider.STABILITY:
    return await text_to_image_file_inference_stability(...)
elif model.provider == InferenceModelProvider.OPENAI:
    return await text_to_image_file_inference_openai(...)
elif model.provider == InferenceModelProvider.HUGGINGFACE:
    return await text_to_image_file_inference_hugging_face(...)
else:
    raise ValueError(f"Unsupported provider: {model.provider}")

```

### 4. Provider Engines

The final layer consists of concrete implementations in `calliope/inference/engines/*.py`. Each engine translates generic requests into provider-specific HTTP or SDK calls:

| Engine | Provider | Key Function |
|--------|----------|--------------|
| [`runway.py`](https://github.com/chrisimmel/calliope/blob/main/runway.py) | Runway Gen-4 | `runway_image_and_text_to_video_inference` |
| [`stability_image.py`](https://github.com/chrisimmel/calliope/blob/main/stability_image.py) | Stability AI | `stability_image_to_image_inference` |
| [`openai_image.py`](https://github.com/chrisimmel/calliope/blob/main/openai_image.py) | OpenAI DALL-E | `text_to_image_file_inference_openai` |
| [`openai_text.py`](https://github.com/chrisimmel/calliope/blob/main/openai_text.py) | OpenAI GPT | `openai_text_to_text_inference` |
| [`azure_vision.py`](https://github.com/chrisimmel/calliope/blob/main/azure_vision.py) | Azure Computer Vision | `analyze_image`, `ocr_image` |
| [`replicate.py`](https://github.com/chrisimmel/calliope/blob/main/replicate.py) | Replicate | `replicate_text_to_image_inference` |

All engines respect a three-tier **parameter override hierarchy**:

1. **Registry defaults** from `InferenceModel` in [`inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/inference_model_config.py).
2. **Configuration overrides** from the selected `ModelConfig` row.
3. **Runtime arguments** passed directly to the inference function.

The Runway engine in [`calliope/inference/engines/runway.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/engines/runway.py) demonstrates this merging:

```python
parameters = {
    **(load_json_if_necessary(model.model_parameters) or {}),
    **(load_json_if_necessary(model_config.model_parameters) or {}),
}
parameters["prompt_image"] = f"data:image/png;base64,{prompt_image}"
parameters["prompt_text"] = prompt_text

```

## How Request Routing Works in Practice

When a client initiates a generation request, the Calliope inference engine architecture executes the following flow:

1. **Resolution**: The dispatcher loads the `ModelConfig` from the database using the provided slug.
2. **Provider Identification**: It reads the `provider` field from the linked `InferenceModel` registry entry.
3. **Engine Selection**: The `if/elif` chain routes to the specific provider engine (e.g., `text_to_image_file_inference_stability` for Stability AI).
4. **Execution**: The engine constructs the provider-specific payload, handles authentication via `KeysModel`, executes the HTTP request or SDK call, and manages polling for asynchronous providers like Runway or Replicate.
5. **Response Handling**: The engine writes output files (images, videos) or returns text content, while the dispatcher handles retries and error normalization.

## Configuring and Extending the Architecture

### Text-to-Image with Stability AI

```python
import httpx
from calliope.models import KeysModel
from calliope.tables import ModelConfig
from calliope.inference import text_to_image_file_inference

async def generate_image():
    async with httpx.AsyncClient() as client:
        model_cfg = await ModelConfig.objects().where(
            ModelConfig.slug == "stable-diffusion-default"
        ).first()

        keys = KeysModel(stability_api_key="YOUR_STABILITY_API_KEY")

        filename = await text_to_image_file_inference(
            httpx_client=client,
            text="A futuristic city at sunset",
            output_image_filename="city.png",
            model_config=model_cfg,
            keys=keys,
            width=512,
            height=512,
        )
        print(f"Image saved to {filename}")

```

### Text-to-Video with Runway

```python
import httpx
from calliope.models import KeysModel
from calliope.tables import ModelConfig
from calliope.inference import image_and_text_to_video_file_inference

async def generate_video():
    async with httpx.AsyncClient() as client:
        cfg = await ModelConfig.objects().where(
            ModelConfig.slug == "runway-gen4-default"
        ).first()
        
        keys = KeysModel(runway_api_key="YOUR_RUNWAY_API_KEY")

        video_path = await image_and_text_to_video_file_inference(
            httpx_client=client,
            prompt_image_file="scene.png",
            prompt_text="A dragon soaring over a mountain range",
            output_video_filename="dragon.mp4",
            model_config=cfg,
            keys=keys,
        )
        print(f"Video saved at {video_path}")

```

### Text-to-Text with OpenAI

```python
import httpx
from calliope.models import KeysModel
from calliope.tables import ModelConfig
from calliope.inference import text_to_text_inference

async def extend_story():
    async with httpx.AsyncClient() as client:
        cfg = await ModelConfig.objects().where(
            ModelConfig.slug == "gpt-4-chat"
        ).first()
        
        keys = KeysModel(openai_api_key="YOUR_OPENAI_API_KEY")

        response = await text_to_text_inference(
            httpx_client=client,
            text="Continue the adventure of a pirate ship lost in a storm.",
            model_config=cfg,
            keys=keys,
        )
        print("LLM reply:", response)

```

## Summary

- **Four-layer abstraction**: The Calliope inference engine architecture separates concerns into a runtime model registry, persistent database configuration, strategy-based dispatch, and provider-specific engines.
- **Provider-agnostic dispatch**: The dispatcher in [`calliope/inference/text_to_image.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/text_to_image.py) and sibling modules routes requests to the correct engine (OpenAI, Stability, Runway, etc.) based on the `provider` field stored in the database.
- **Hierarchical configuration**: Parameters merge in three tiers—registry defaults from [`inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/inference_model_config.py), database overrides from `ModelConfig`, and runtime arguments passed to inference functions.
- **Extensible design**: Adding a new provider requires only extending the `_model_configs_by_name` registry and implementing a matching engine module under `calliope/inference/engines/`.

## Frequently Asked Questions

### What makes Calliope's inference engine provider-agnostic?

The architecture abstracts provider details behind a unified interface. Client code interacts with high-level functions like `text_to_image_file_inference` in [`calliope/inference/__init__.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/inference/__init__.py), while the dispatcher handles provider-specific routing. The `InferenceModelProvider` enum in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) defines supported providers, and the engine layer encapsulates all SDK and HTTP implementation details.

### How does Calliope handle model parameter overrides?

The system implements a three-tier override hierarchy. First, default parameters from the `InferenceModel` registry entry in [`inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/inference_model_config.py) provide baseline values. Second, the `model_parameters` JSONB column in the `ModelConfig` table applies deployment-specific overrides. Third, runtime arguments passed directly to inference functions (like `width` and `height` in `text_to_image_file_inference`) take final precedence.

### Where are the API keys managed in the Calliope architecture?

API keys flow through the `KeysModel` class, which aggregates credentials for all supported providers (OpenAI, Stability, Runway, etc.). Client code instantiates `KeysModel` with the relevant API keys and passes it to inference functions. The provider engines in `calliope/inference/engines/*.py` extract the specific key they need from this model to authenticate their respective SDK or HTTP requests.

### How do I add a new provider to Calliope?

Extending the architecture requires two steps. First, add a new entry to the `_model_configs_by_name` dictionary in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py), specifying the provider enum value, model name, and default parameters. Second, create a new engine module under `calliope/inference/engines/` (e.g., [`new_provider.py`](https://github.com/chrisimmel/calliope/blob/main/new_provider.py)) implementing the provider's SDK or HTTP contract with the standard function signature accepting `httpx_client`, prompt data, `model`, `model_config`, and `keys`. The dispatcher will automatically route requests to your new engine when the corresponding provider is selected in the database configuration.