How OpenMAIC Handles Model Routing: Per-Stage LLM Configuration Explained

OpenMAIC routes language model calls through a per-stage mapping system defined by the MODEL_ROUTES environment variable, allowing different conversation stages—such as SYSTEM, USER, and TEACHER—to bind to distinct LLM providers via helper functions in lib/server/model-routes.ts.

The THU-MAIC/OpenMAIC framework separates model selection from generation logic by introducing a dedicated routing layer. This architecture enables developers to assign specialized models to specific phases of a classroom session without modifying application code. Understanding how OpenMAIC handles model routing is essential for optimizing cost and performance across multi-provider deployments.

The MODEL_ROUTES Configuration System

OpenMAIC externalizes routing decisions into a JSON configuration stored in the MODEL_ROUTES environment variable. At startup, the framework validates this map and uses it to resolve which provider and model handle each stage of interaction.

Environment Variable Structure

The MODEL_ROUTES variable expects a JSON object where keys correspond to LLM stages (system, user, teacher, assistant, end) and values specify the target model identifier. As shown in .env.example (lines 411–426), a typical configuration maps each stage to a different model:

MODEL_ROUTES='{
  "system": "glm-5.2",
  "user": "kimi-k2.7-code",
  "teacher": "qwen3.7-max",
  "assistant": "gpt-4o-mini"
}'

This structure allows the SYSTEM stage to use lightweight models for prompt engineering while reserving larger models for TEACHER or USER stages requiring complex reasoning.

Startup Validation

Before accepting traffic, the server validates the MODEL_ROUTES syntax in src/lib/server/config-validation.ts (line 27). The validation routine checks for malformed JSON and missing stage definitions, emitting warnings for invalid entries to prevent runtime resolution failures.

Core Routing Functions in model-routes.ts

The central routing logic resides in src/lib/server/model-routes.ts, which exports two primary functions for resolving stage-to-model mappings at runtime.

getStageModel() for Model Name Resolution

The getStageModel(stage: LlmStage) function returns the concrete model identifier (e.g., "gpt-4o-mini") assigned to a specific stage. This utility is imported throughout the backend to decouple business logic from provider-specific naming conventions.

import { getStageModel, type LlmStage } from '@/lib/server/model-routes';

async function generateForStage(stage: LlmStage, prompt: string) {
  const model = getStageModel(stage);  // Resolves to configured model name
  return await generateWithModel(model, prompt);
}

getStageRoute() for Full Configuration

For scenarios requiring provider-specific options, getStageRoute(stage: LlmStage) returns the complete route configuration object, including the model name, provider alias, and inference parameters. This enables the server to instantiate the correct client implementation based on the stage.

import { getStageRoute } from '@/lib/server/model-routes';

export async function resolveModel(stage: string) {
  const route = getStageRoute(stage as LlmStage);  // Returns { model, provider, options }
  return await route.provider.call(route.model, route.options);
}

Stage-Aware Resolution in Practice

OpenMAIC integrates these routing utilities into its generation pipeline, ensuring that each phase of a classroom session uses its designated model.

Classroom Generation Flow (classroom-generation.ts)

In src/lib/server/classroom-generation.ts (line 22), the system invokes getStageModel() to select the appropriate LLM before processing classroom content. This ensures that content generation for the TEACHER stage might use a high-capacity model while USER stage queries route to a faster, cost-effective alternative.

Server-Side Model Resolution (resolve-model.ts)

The src/lib/server/resolve-model.ts module (line 19) utilizes getStageRoute() to translate stage names into executable provider calls. This file acts as the bridge between OpenMAIC's abstract stage concepts and concrete SDK implementations (OpenAI, Anthropic, or local inference servers).

// Example integration pattern from the codebase
import { getStageRoute } from './model-routes';

export async function callLLMForStage(stage: LlmStage, messages: Message[]) {
  const route = getStageRoute(stage);
  
  // Route.provider determines which client SDK to invoke
  const response = await route.provider.complete({
    model: route.model,
    messages,
    ...route.options
  });
  
  return response;
}

Summary

• Environment-driven configuration: OpenMAIC reads per-stage model assignments from the MODEL_ROUTES environment variable, allowing runtime reconfiguration without code changes.
• Type-safe resolution: The LlmStage type and helper functions getStageModel() and getStageRoute() provide compile-time safety for stage-to-model mappings.
• Centralized validation: Startup checks in config-validation.ts ensure malformed routing configurations are caught before serving traffic.
• Pipeline integration: Both classroom-generation.ts and resolve-model.ts consume the routing layer to maintain clean separation between business logic and provider selection.

Frequently Asked Questions

What format does the MODEL_ROUTES environment variable use?

MODEL_ROUTES accepts a JSON object string where keys are stage identifiers (system, user, teacher, assistant, end) and values are model name strings such as "gpt-4o-mini" or "qwen3.7-max". The framework parses this variable at startup to build the internal routing map.

How does OpenMAIC validate routing configuration at startup?

The validation logic in src/lib/server/config-validation.ts (line 27) parses the MODEL_ROUTES JSON and verifies that required stages are present and that model identifiers conform to expected patterns. Warnings are logged for missing or malformed entries, though the server may continue running with fallback defaults depending on the severity.

Can different stages route to completely different providers?

Yes. While getStageModel() returns only the model identifier, getStageRoute() returns a configuration object containing both the model name and the provider implementation. This allows the SYSTEM stage to use a local Ollama instance while the TEACHER stage uses OpenAI's API, managed through the same unified interface.

Where is the core routing logic implemented?

The primary routing implementation resides in src/lib/server/model-routes.ts, which exports the getStageModel() and getStageRoute() functions. This file is imported by src/lib/server/classroom-generation.ts and src/lib/server/resolve-model.ts to resolve models during request processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}