# How Inference Model Configuration is Managed in Calliope: Database and Registry Patterns

> Discover how Calliope manages inference model configuration using Piccolo ORM and Pydantic models. Achieve dynamic overrides without code changes. Learn more.

- Repository: [chrisimmel/calliope](https://github.com/chrisimmel/calliope)
- Tags: internals
- Published: 2026-02-27

---

**Calliope manages inference model configuration through a hybrid architecture that separates persistent database definitions in Piccolo ORM tables from runtime Pydantic models, enabling dynamic strategy-specific overrides without code changes.**

Calliope, the open-source storytelling framework by chrisimmel/calliope, implements a sophisticated inference model configuration system that decouples model definitions from runtime execution parameters. This architecture allows developers to register new AI providers, define parameter defaults, and override configurations per storytelling strategy while maintaining type safety through Pydantic validation.

## Database-Driven Configuration Schema

Calliope persists all inference model configuration in a Piccolo-managed database (SQLite or PostgreSQL). The schema is defined in [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py) and centers on three interconnected tables that separate model definitions from their runtime usage.

### The InferenceModel Table

The `InferenceModel` table (lines 59-74 in [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py)) stores concrete provider-specific model definitions. Each row represents a specific AI model available through a provider API, capturing:

- **provider**: The inference provider (e.g., OpenAI, Anthropic, HuggingFace)
- **provider_api_variant**: The specific API endpoint type (e.g., chat completion, text completion)
- **provider_model_name**: The provider's internal model identifier
- **model_parameters**: JSON field containing default parameters like `max_tokens` and `temperature`

### The ModelConfig Table

The `ModelConfig` table (lines 12-30) links to an `InferenceModel` via foreign key and optionally to a `PromptTemplate`. This table holds **parameter overrides** that strategies apply at runtime. When a strategy requests inference, it passes a `ModelConfig` instance that may override the base model's default parameters, enabling per-strategy customization without altering the underlying `InferenceModel` definition.

### The StrategyConfig Table

The `StrategyConfig` table (lines 68-84) references one or more `ModelConfig` objects to specify the default models for a given storytelling strategy. This allows complex strategies to define which inference models handle specific sub-tasks (e.g., text generation vs. image generation) while maintaining clean separation between strategy logic and model configuration.

## Python-Level Configuration Models

While the database stores persistent configuration, Calliope uses Pydantic models for runtime validation and type safety. These models reside in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py).

### Pydantic Models and Provider Enums

The module defines enumerations for supported providers and API variants:

```python
from calliope.models.inference_model_config import (
    InferenceModelProvider,
    InferenceModelProviderVariant,
    InferenceModelConfigModel
)

# Supported providers include OPENAI, ANTHROPIC, HUGGINGFACE, etc.

provider = InferenceModelProvider.OPENAI
variant = InferenceModelProviderVariant.OPENAI_CHAT_COMPLETION

```

The `InferenceModelConfigModel` Pydantic class mirrors the database `InferenceModel` fields, providing runtime validation for parameters like `max_tokens`, `temperature`, and `top_p`.

### The Configuration Registry

Calliope maintains an in-memory registry `_model_configs_by_name` that maps human-readable names to `InferenceModelConfigModel` instances. This registry is populated at startup with built-in configurations:

```python

# From calliope/models/inference_model_config.py lines 45-57

_model_configs_by_name = {
    "openai_gpt_4": InferenceModelConfigModel(
        provider=InferenceModelProvider.OPENAI,
        provider_api_variant=InferenceModelProviderVariant.OPENAI_CHAT_COMPLETION,
        provider_model_name="gpt-4",
        model_parameters={"max_tokens": 512, "temperature": 0.7}
    ),
    # Additional built-in configs...

}

```

This registry allows the system to reference models by simple string keys while maintaining full type safety and validation.

## Loading and Using Inference Model Configurations

The bridge between static registry definitions and dynamic runtime usage is handled by the `load_model_configs()` helper function.

### The load_model_configs Helper

Located in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) (lines 6-24), this function builds an `InferenceModelConfigsModel` populated with the desired configurations:

```python
from calliope.models.inference_model_config import load_model_configs

configs = load_model_configs(
    text_to_text_model_config="huggingface_gpt_neo_2.7B",
    text_to_image_model_config="stability_stable_diffusion_1.5",
    text_to_audio_model_config="bark_large",
)

```

The resulting `configs` object is passed to inference functions to specify which models handle specific media transformations.

### Runtime Configuration Overrides

Strategies achieve flexibility by providing different `ModelConfig` database rows at runtime. While the registry provides defaults, the `ModelConfig` table allows per-strategy overrides:

```python

# Creating a custom configuration that overrides base parameters

await ModelConfig.create(
    slug="gpt4-creative",
    model=await InferenceModel.objects().get(InferenceModel.slug == "openai-gpt-4"),
    model_parameters={"temperature": 1.2, "max_tokens": 1024},
)

```

When a strategy references this `ModelConfig` instead of the base model, all inference calls use the overridden parameters without requiring code changes.

## Practical Implementation Examples

The following examples demonstrate complete workflows for configuring and using inference models in Calliope.

### Registering a New Inference Model

```python
from calliope.tables.model_config import InferenceModel
from calliope.models.inference_model_config import (
    InferenceModelProvider,
    InferenceModelProviderVariant
)

# Create a new model definition in the database

await InferenceModel.create(
    slug="anthropic-claude-3",
    description="Anthropic Claude 3 Opus",
    provider=InferenceModelProvider.ANTHROPIC,
    provider_api_variant=InferenceModelProviderVariant.ANTHROPIC_MESSAGES,
    provider_model_name="claude-3-opus-20240229",
    model_parameters={"max_tokens": 4096, "temperature": 0.5},
)

```

### Executing Inference with Custom Configuration

```python
import httpx
from calliope.inference import text_to_text_inference
from calliope.tables.model_config import ModelConfig

async with httpx.AsyncClient() as client:
    # Retrieve the custom configuration from the database

    model_config = await ModelConfig.objects().get(
        ModelConfig.slug == "gpt4-creative"
    )
    
    # Execute inference using the configured model and parameters

    result = await text_to_text_inference(
        client,
        "Tell me a short tale about a brave rabbit.",
        model_config=model_config,
        keys=keys,  # KeysModel containing API secrets

    )
    print(result)

```

## Summary

Calliope's inference model configuration system combines database persistence with runtime Pydantic validation to create a flexible, maintainable architecture:

- **Database Layer**: Three Piccolo tables (`InferenceModel`, `ModelConfig`, `StrategyConfig`) in [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py) separate base model definitions from runtime parameter overrides and strategy assignments.
- **Runtime Layer**: Pydantic models in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py) provide type-safe configuration objects, with a built-in registry mapping human-readable names to validated configs.
- **Flexibility**: The `load_model_configs()` helper bridges static registry entries with dynamic database configurations, allowing strategies to override parameters per-call without code changes.
- **Extensibility**: New providers and models are registered via standard database CRUD operations or Piccolo admin UI, immediately available to all inference functions.

## Frequently Asked Questions

### How does Calliope store inference model configuration persistently?

Calliope persists inference model configuration in a Piccolo-managed database (SQLite or PostgreSQL) using three core tables defined in [`calliope/tables/model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/tables/model_config.py). The `InferenceModel` table stores base provider definitions, `ModelConfig` holds parameter overrides linked to specific models, and `StrategyConfig` associates model configurations with storytelling strategies. This schema allows operators to modify configurations through the Piccolo admin UI or direct database operations without deploying code changes.

### What is the difference between InferenceModel and ModelConfig in Calliope?

`InferenceModel` represents the immutable base definition of a provider's AI model, including the provider name, API variant, model identifier, and default parameters. `ModelConfig` acts as a mutable layer that references an `InferenceModel` via foreign key and overrides specific parameters like `temperature` or `max_tokens` for particular use cases. This separation allows multiple strategies to use the same base model with different runtime behaviors by referencing different `ModelConfig` rows.

### How do I add a custom inference model to Calliope without modifying source code?

You can register new inference models through the Piccolo admin interface or programmatically using the `InferenceModel` table. Create a new row specifying the `provider` (from `InferenceModelProvider` enum), `provider_api_variant`, `provider_model_name`, and default `model_parameters`. Once persisted, the model becomes available for use in `ModelConfig` rows and can be referenced by strategies immediately. For runtime usage without database persistence, you can also extend the `_model_configs_by_name` registry in [`calliope/models/inference_model_config.py`](https://github.com/chrisimmel/calliope/blob/main/calliope/models/inference_model_config.py), though database registration is preferred for production deployments.

### Where does Calliope handle the selection of which inference model to use during text generation?

The selection logic resides in the inference layer functions defined in the Calliope inference module (documented in [`docs/inference.md`](https://github.com/chrisimmel/calliope/blob/main/docs/inference.md)). Functions like `text_to_text_inference` receive a `ModelConfig` parameter (or the Pydantic wrapper) and a `KeysModel` containing API secrets. The function inspects `model.provider` and `model.provider_api_variant` to dispatch to the correct provider-specific engine. Strategies determine which `ModelConfig` to pass, either loading defaults via `load_model_configs()` or retrieving custom configurations from the database using Piccolo ORM queries.