How to Configure OpenRouter Models and Provider Routing in Codebuff

Codebuff centralizes AI model configuration in a single catalogue where friendly constants map to provider-specific identifiers, enabling automatic routing, fallback control, and usage tracking across the SDK.

To configure OpenRouter models and provider routing in Codebuff, you work with a layered architecture defined in the CodebuffAI/codebuff repository. The system separates high-level model names from provider-specific implementation details, allowing you to add new models, define provider preferences, and control fallback behavior through a few key configuration files.

Understanding Codebuff's Model Configuration Architecture

Codebuff uses a model catalogue pattern to decouple user-facing model names from provider API identifiers. This architecture lives primarily in common/src/constants.ts, where every supported model is enumerated as a constant mapping.

When you reference a model in your application, you use these friendly constants (e.g., models.openrouter_claude_sonnet_4). The SDK then resolves this to the exact provider string (e.g., anthropic/claude-sonnet-4) and applies routing rules defined in sdk/src/impl/llm.ts.

Configuring OpenRouter Models in the Catalogue

Adding New Models to constants.ts

To add a new OpenRouter model, extend the models object in common/src/constants.ts. Each entry maps a TypeScript constant to the exact identifier OpenRouter expects:

// common/src/constants.ts
export const models = {
  // Existing entries...
  openrouter_claude_sonnet_4: 'anthropic/claude-sonnet-4',
  
  // New model addition
  openrouter_claude_opus_5: 'anthropic/claude-opus-5',
  openrouter_gemini_pro: 'google/gemini-pro-1.5',
} as const;

Defining Provider Routing Order

After adding the model constant, specify the preferred provider order in sdk/src/impl/llm.ts. The providerOrder object tells OpenRouter which providers to try first, enabling automatic failover if a provider is unavailable:

// sdk/src/impl/llm.ts
const providerOrder = {
  // Existing mappings...
  [models.openrouter_claude_sonnet_4]: ['Google', 'Anthropic'],
  
  // New model provider order
  [models.openrouter_claude_opus_5]: ['Anthropic', 'Google'],
  [models.openrouter_gemini_pro]: ['Google', 'Anthropic'],
} as const;

How Provider Routing and Fallbacks Work

Codebuff determines whether to allow provider fallbacks using the isExplicitlyDefinedModel() function in common/src/util/model-utils.ts. This function checks if a model exists in the catalogue:

// common/src/util/model-utils.ts
export function isExplicitlyDefinedModel(model: string): boolean {
  // Lazily loads models into a Set for O(1) lookups
  return getExplicitModelsSet().has(model);
}

In backend/src/llm-apis/openrouter.ts, the openRouterLanguageModel function uses this check to set the allow_fallbacks flag:

• Explicit models (in catalogue): allow_fallbacks: false — OpenRouter must use the specified provider order without substitution
• Dynamic models (not in catalogue): allow_fallbacks: true — OpenRouter can fall back to alternative providers if the primary fails

Enabling Cache Control for OpenRouter Models

Cache control eligibility is determined by the supportsCacheControl() function in common/src/constants/model-config.ts. This function reuses isExplicitlyDefinedModel() to verify catalogue membership:

// common/src/constants/model-config.ts
export function supportsCacheControl(model: string): boolean {
  return isExplicitlyDefinedModel(model);
}

Only explicitly defined models can honor HTTP cache headers. If you add a custom model string dynamically (not in constants.ts), Codebuff treats it as non-cacheable to prevent stale data issues.

Tracking Usage and Billing

Codebuff automatically extracts usage metadata from OpenRouter responses. In sdk/src/impl/llm.ts, the SDK reads providerMetadata from the response and stores cost data under codebuff.usage:

// sdk/src/impl/llm.ts (simplified)
const result = await model.doGenerate({...});
const cost = result.providerMetadata?.openrouter?.cost;
// Stored in codebuff.usage for billing aggregation

This enables per-model cost tracking across your application without manual instrumentation.

Complete Implementation Example

Here is a complete workflow for configuring and using a custom OpenRouter model:

// Step 1: Add to common/src/constants.ts
export const models = {
  openrouter_custom_llama: 'meta-llama/llama-3.1-70b-instruct',
} as const;

// Step 2: Define provider order in sdk/src/impl/llm.ts
const providerOrder = {
  [models.openrouter_custom_llama]: ['Together', 'Fireworks', 'OctoAI'],
} as const;

// Step 3: Use in your application
import { models } from '@codebuff/common/constants';
import { openRouterLanguageModel } from '@codebuff/backend/llm-apis/openrouter';

const model = models.openrouter_custom_llama;
const llm = openRouterLanguageModel(model);

const response = await llm({
  body: {
    model,
    messages: [{ role: 'user', content: 'Explain quantum computing' }],
  },
});

Summary

• Model catalogue: All OpenRouter models are defined in common/src/constants.ts as friendly constants mapping to provider-specific identifiers.
• Provider routing: The providerOrder map in sdk/src/impl/llm.ts defines failover preferences sent to OpenRouter.
• Fallback control: isExplicitlyDefinedModel() in common/src/util/model-utils.ts determines whether OpenRouter may use provider fallbacks (dynamic models) or must stick to the defined order (catalogue models).
• Cache control: Only explicitly defined models support HTTP cache headers via supportsCacheControl().
• Billing: Usage costs are automatically extracted from OpenRouter metadata and stored for aggregation.

Frequently Asked Questions

What is the difference between explicit and dynamic models in Codebuff?

Explicit models are those defined in the models object within common/src/constants.ts. These models have strict provider routing, disable fallbacks, and support cache control. Dynamic models are custom strings passed at runtime that are not in the catalogue; they allow provider fallbacks for resilience but cannot use cache control features.

How do I disable provider fallbacks for a specific OpenRouter model?

Provider fallbacks are automatically disabled for any model defined in common/src/constants.ts. When isExplicitlyDefinedModel() returns true for a model string, the openRouterLanguageModel function in backend/src/llm-apis/openrouter.ts sets allow_fallbacks: false in the OpenRouter request body. To disable fallbacks for a custom model, you must add it to the constants catalogue.

Where does Codebuff store usage data for OpenRouter API calls?

Usage data is extracted from the providerMetadata field in OpenRouter responses within sdk/src/impl/llm.ts. The SDK reads the cost information and stores it under codebuff.usage, enabling the billing subsystem to aggregate per-model costs across all API calls.

Can I use custom model strings not defined in the catalogue?

Yes, Codebuff supports dynamic model strings that are not explicitly defined in common/src/constants.ts. However, these dynamic models automatically enable provider fallbacks (allow_fallbacks: true) and disable cache control support. They are useful for testing new OpenRouter models immediately without waiting for a catalogue update, but lack the strict routing guarantees of explicit models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}