How to Configure AI Model Parameters and Manage Costs in Calliope

Yes, Calliope provides a hierarchical configuration system using InferenceModel for global presets and ModelConfig for deployment-specific overrides, while KeysModel enables per-client API credential isolation for granular cost tracking.

Calliope is an open-source inference framework designed for multi-tenant AI deployments. Whether you are running inference for a single application or managing thousands of edge devices, you can configure AI model parameters and manage costs in Calliope through its layered architecture of model registries, configuration tables, and credential scoping.

Understanding Calliope's Configuration Architecture

Global Model Presets in InferenceModel

The foundation of parameter management resides in calliope/tables/model_config.py. The InferenceModel table stores provider details, model identifiers, and default model_parameters as a JSON column. These records act as the canonical reference for available AI services.

Deployment-Specific Overrides with ModelConfig

To customize behavior without modifying global presets, Calliope uses the ModelConfig table defined in the same file. This table contains a foreign key to InferenceModel and its own model_parameters JSON field. By attaching a ModelConfig to a Sparrow, Flock, or StrategyConfig, you achieve per-client parameter overrides.

How to Configure AI Model Parameters

Configuring parameters involves defining presets and linking them to deployments.

First, create a custom inference model preset in calliope/models/inference_model_config.py. The InferenceModelConfigModel Pydantic class defines the schema, while the _model_configs_by_name registry stores available configurations:

from calliope.models.inference_model_config import (
    InferenceModelProvider,
    InferenceModelProviderVariant,
    InferenceModelConfigModel,
    _model_configs_by_name,
)

# Register a cost-optimized GPT-4 configuration

_model_configs_by_name["gpt-4-efficient"] = InferenceModelConfigModel(
    provider=InferenceModelProvider.OPENAI,
    provider_variant=InferenceModelProviderVariant.OPENAI_CHAT_COMPLETION,
    provider_model_name="gpt-4",
    parameters={
        "max_tokens": 256,          # limit token usage → lower cost

        "temperature": 0.7,         # more deterministic output

        "presence_penalty": 0.5,
        "frequency_penalty": 0.5,
    },
)

Next, create a ModelConfig record to apply these parameters to a specific deployment. Using the Piccolo ORM as defined in calliope/tables/model_config.py:

from calliope.tables import ModelConfig, InferenceModel

async def create_deployment_config():
    # Reference the preset

    model = await InferenceModel.objects().where(
        InferenceModel.slug == "openai-gpt-4"
    ).first()
    
    # Create deployment-specific overrides

    config = await ModelConfig(
        slug="deployment-42-config",
        model=model,
        model_parameters={"max_tokens": 128, "temperature": 0.5},   # further limit per-client

    ).save()
    return config

Managing API Costs and Credential Isolation

Per-Sparrow and Per-Flock API Keys

Cost management in Calliope relies on the KeysModel class defined in calliope/models/keys.py. This Pydantic model stores provider credentials and attaches to individual Sparrows or Flocks via the keys JSON column in their respective state tables.

To isolate billing for a specific client:

from calliope.models.keys import KeysModel
from calliope.tables import SparrowState

async def assign_client_keys(sparrow_id: str, api_key: str):
    keys = KeysModel(openai_api_key=api_key)   # <-- keep secret in env / UI

    await SparrowState.objects().where(
        SparrowState.sparrow_id == sparrow_id
    ).update(keys=keys.model_dump())

Usage Attribution and Cost Tracking

Because each inference call in calliope/inference/text_to_text.py receives a KeysModel instance, the underlying engine uses the specific API key associated with that sparrow. This design enables precise usage attribution to individual clients or device groups, supporting cost dashboards and quota enforcement without code modifications.

Runtime Parameter Resolution

When a request arrives at the /v1/frames/ endpoint, the handler in calliope/tasks/handlers.py orchestrates the configuration merge:

1. Load the ModelConfig associated with the target Sparrow or Flock
2. Merge the base InferenceModel parameters with the ModelConfig overrides
3. Apply any runtime request parameters
4. Execute inference using the scoped KeysModel credentials

This hierarchy ensures that global defaults, deployment overrides, and per-request tweaks coexist predictably.

Summary

• Configure AI model parameters using InferenceModel for global presets and ModelConfig for deployment-specific overrides stored in calliope/tables/model_config.py.
• Manage costs by isolating API credentials per Sparrow or Flock using KeysModel in calliope/models/keys.py, enabling precise usage attribution.
• Override parameters at runtime by attaching ModelConfig records to individual Sparrows, Flocks, or StrategyConfigs, with resolution handled in calliope/tasks/handlers.py.
• Define custom presets by registering InferenceModelConfigModel instances in calliope/models/inference_model_config.py for reusable, cost-optimized configurations.

Frequently Asked Questions

Can I set different model parameters for different clients using Calliope?

Yes. By creating separate ModelConfig records in calliope/tables/model_config.py and attaching them to different Sparrows or Flocks, you can assign unique temperature, max_tokens, and other parameters to individual clients or device groups. The handler in calliope/tasks/handlers.py automatically resolves the correct configuration at runtime.

How does Calliope track API usage costs per deployment?

Calliope uses the KeysModel class defined in calliope/models/keys.py to store provider API keys. Each Sparrow or Flock maintains its own keys JSON column, ensuring that every inference call uses a specific credential set. This allows you to attribute usage to specific clients via your provider's dashboard and implement per-client quotas.

What is the difference between InferenceModel and ModelConfig in Calliope?

InferenceModel acts as the global registry of available AI providers and their default parameters, stored in the database schema defined in calliope/tables/model_config.py. ModelConfig provides a layer of overrides that can be attached to specific deployments (Sparrows, Flocks, or strategies), allowing you to customize behavior without modifying the global preset.

Can I override model parameters at request time in Calliope?

While the primary mechanism uses persisted ModelConfig records, the architecture in calliope/tasks/handlers.py supports merging runtime parameters with stored configurations. You can extend the request handler to accept query parameters or body fields that override the stored model_parameters JSON before the inference call executes, though this requires custom implementation beyond the default data-driven configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}