How Inference Model Configuration is Managed in Calliope: Database and Registry Patterns

Calliope manages inference model configuration through a hybrid architecture that separates persistent database definitions in Piccolo ORM tables from runtime Pydantic models, enabling dynamic strategy-specific overrides without code changes.

Calliope, the open-source storytelling framework by chrisimmel/calliope, implements a sophisticated inference model configuration system that decouples model definitions from runtime execution parameters. This architecture allows developers to register new AI providers, define parameter defaults, and override configurations per storytelling strategy while maintaining type safety through Pydantic validation.

Database-Driven Configuration Schema

Calliope persists all inference model configuration in a Piccolo-managed database (SQLite or PostgreSQL). The schema is defined in calliope/tables/model_config.py and centers on three interconnected tables that separate model definitions from their runtime usage.

The InferenceModel Table

The InferenceModel table (lines 59-74 in calliope/tables/model_config.py) stores concrete provider-specific model definitions. Each row represents a specific AI model available through a provider API, capturing:

• provider: The inference provider (e.g., OpenAI, Anthropic, HuggingFace)
• provider_api_variant: The specific API endpoint type (e.g., chat completion, text completion)
• provider_model_name: The provider's internal model identifier
• model_parameters: JSON field containing default parameters like max_tokens and temperature

The ModelConfig Table

The ModelConfig table (lines 12-30) links to an InferenceModel via foreign key and optionally to a PromptTemplate. This table holds parameter overrides that strategies apply at runtime. When a strategy requests inference, it passes a ModelConfig instance that may override the base model's default parameters, enabling per-strategy customization without altering the underlying InferenceModel definition.

The StrategyConfig Table

The StrategyConfig table (lines 68-84) references one or more ModelConfig objects to specify the default models for a given storytelling strategy. This allows complex strategies to define which inference models handle specific sub-tasks (e.g., text generation vs. image generation) while maintaining clean separation between strategy logic and model configuration.

Python-Level Configuration Models

While the database stores persistent configuration, Calliope uses Pydantic models for runtime validation and type safety. These models reside in calliope/models/inference_model_config.py.

Pydantic Models and Provider Enums

The module defines enumerations for supported providers and API variants:

from calliope.models.inference_model_config import (
    InferenceModelProvider,
    InferenceModelProviderVariant,
    InferenceModelConfigModel
)

# Supported providers include OPENAI, ANTHROPIC, HUGGINGFACE, etc.

provider = InferenceModelProvider.OPENAI
variant = InferenceModelProviderVariant.OPENAI_CHAT_COMPLETION

The InferenceModelConfigModel Pydantic class mirrors the database InferenceModel fields, providing runtime validation for parameters like max_tokens, temperature, and top_p.

The Configuration Registry

Calliope maintains an in-memory registry _model_configs_by_name that maps human-readable names to InferenceModelConfigModel instances. This registry is populated at startup with built-in configurations:

# From calliope/models/inference_model_config.py lines 45-57

_model_configs_by_name = {
    "openai_gpt_4": InferenceModelConfigModel(
        provider=InferenceModelProvider.OPENAI,
        provider_api_variant=InferenceModelProviderVariant.OPENAI_CHAT_COMPLETION,
        provider_model_name="gpt-4",
        model_parameters={"max_tokens": 512, "temperature": 0.7}
    ),
    # Additional built-in configs...

}

This registry allows the system to reference models by simple string keys while maintaining full type safety and validation.

Loading and Using Inference Model Configurations

The bridge between static registry definitions and dynamic runtime usage is handled by the load_model_configs() helper function.

The load_model_configs Helper

Located in calliope/models/inference_model_config.py (lines 6-24), this function builds an InferenceModelConfigsModel populated with the desired configurations:

from calliope.models.inference_model_config import load_model_configs

configs = load_model_configs(
    text_to_text_model_config="huggingface_gpt_neo_2.7B",
    text_to_image_model_config="stability_stable_diffusion_1.5",
    text_to_audio_model_config="bark_large",
)

The resulting configs object is passed to inference functions to specify which models handle specific media transformations.

Runtime Configuration Overrides

Strategies achieve flexibility by providing different ModelConfig database rows at runtime. While the registry provides defaults, the ModelConfig table allows per-strategy overrides:

# Creating a custom configuration that overrides base parameters

await ModelConfig.create(
    slug="gpt4-creative",
    model=await InferenceModel.objects().get(InferenceModel.slug == "openai-gpt-4"),
    model_parameters={"temperature": 1.2, "max_tokens": 1024},
)

When a strategy references this ModelConfig instead of the base model, all inference calls use the overridden parameters without requiring code changes.

Practical Implementation Examples

The following examples demonstrate complete workflows for configuring and using inference models in Calliope.

Registering a New Inference Model

from calliope.tables.model_config import InferenceModel
from calliope.models.inference_model_config import (
    InferenceModelProvider,
    InferenceModelProviderVariant
)

# Create a new model definition in the database

await InferenceModel.create(
    slug="anthropic-claude-3",
    description="Anthropic Claude 3 Opus",
    provider=InferenceModelProvider.ANTHROPIC,
    provider_api_variant=InferenceModelProviderVariant.ANTHROPIC_MESSAGES,
    provider_model_name="claude-3-opus-20240229",
    model_parameters={"max_tokens": 4096, "temperature": 0.5},
)

Executing Inference with Custom Configuration

import httpx
from calliope.inference import text_to_text_inference
from calliope.tables.model_config import ModelConfig

async with httpx.AsyncClient() as client:
    # Retrieve the custom configuration from the database

    model_config = await ModelConfig.objects().get(
        ModelConfig.slug == "gpt4-creative"
    )
    
    # Execute inference using the configured model and parameters

    result = await text_to_text_inference(
        client,
        "Tell me a short tale about a brave rabbit.",
        model_config=model_config,
        keys=keys,  # KeysModel containing API secrets

    )
    print(result)

Summary

Calliope's inference model configuration system combines database persistence with runtime Pydantic validation to create a flexible, maintainable architecture:

• Database Layer: Three Piccolo tables (InferenceModel, ModelConfig, StrategyConfig) in calliope/tables/model_config.py separate base model definitions from runtime parameter overrides and strategy assignments.
• Runtime Layer: Pydantic models in calliope/models/inference_model_config.py provide type-safe configuration objects, with a built-in registry mapping human-readable names to validated configs.
• Flexibility: The load_model_configs() helper bridges static registry entries with dynamic database configurations, allowing strategies to override parameters per-call without code changes.
• Extensibility: New providers and models are registered via standard database CRUD operations or Piccolo admin UI, immediately available to all inference functions.

Frequently Asked Questions

How does Calliope store inference model configuration persistently?

Calliope persists inference model configuration in a Piccolo-managed database (SQLite or PostgreSQL) using three core tables defined in calliope/tables/model_config.py. The InferenceModel table stores base provider definitions, ModelConfig holds parameter overrides linked to specific models, and StrategyConfig associates model configurations with storytelling strategies. This schema allows operators to modify configurations through the Piccolo admin UI or direct database operations without deploying code changes.

What is the difference between InferenceModel and ModelConfig in Calliope?

InferenceModel represents the immutable base definition of a provider's AI model, including the provider name, API variant, model identifier, and default parameters. ModelConfig acts as a mutable layer that references an InferenceModel via foreign key and overrides specific parameters like temperature or max_tokens for particular use cases. This separation allows multiple strategies to use the same base model with different runtime behaviors by referencing different ModelConfig rows.

How do I add a custom inference model to Calliope without modifying source code?

You can register new inference models through the Piccolo admin interface or programmatically using the InferenceModel table. Create a new row specifying the provider (from InferenceModelProvider enum), provider_api_variant, provider_model_name, and default model_parameters. Once persisted, the model becomes available for use in ModelConfig rows and can be referenced by strategies immediately. For runtime usage without database persistence, you can also extend the _model_configs_by_name registry in calliope/models/inference_model_config.py, though database registration is preferred for production deployments.

Where does Calliope handle the selection of which inference model to use during text generation?

The selection logic resides in the inference layer functions defined in the Calliope inference module (documented in docs/inference.md). Functions like text_to_text_inference receive a ModelConfig parameter (or the Pydantic wrapper) and a KeysModel containing API secrets. The function inspects model.provider and model.provider_api_variant to dispatch to the correct provider-specific engine. Strategies determine which ModelConfig to pass, either loading defaults via load_model_configs() or retrieving custom configurations from the database using Piccolo ORM queries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}