How to Configure Model Routing for Different Generation Stages in OpenMAIC

To configure model routing in OpenMAIC, modify the provider configuration registry, implement custom logic in the middleware.ts resolver, and specify stage-specific models using the client-side settings store or server-side environment variables like FORCE_MODEL.

OpenMAIC separates its generation pipeline into distinct stages—such as chat-completion, text-to-speech (TTS), and image generation—each potentially requiring different underlying models. This open-source framework from THU-MAIC orchestrates model selection through a sophisticated routing system that balances client preferences with server-side enforcement. Understanding how to configure model routing for different generation stages in OpenMAIC enables developers to customize AI workflows, enforce security policies, and optimize inference across heterogeneous provider configurations.

Core Routing Architecture

OpenMAIC implements model routing through three interconnected components that work together at runtime:

Provider configuration acts as a static registry mapping providers (OpenAI, Anthropic, etc.) to their available models. This configuration lives implicitly within the store system and is validated against server-side allowances.

Middleware resolver handles request-level resolution. Located in middleware.ts, this component inspects incoming requests and determines which client model (UI-selected) or server model (backend-permitted) should handle the current generation stage.

Stage-specific helpers manage post-processing integration. After a tool finishes execution—such as generate_tts in lib/workbench/tts-stage-sync.ts—these helpers merge results back into the live scene using the exact model identifier that produced the output.

The Model Resolution Pipeline

The routing logic follows a precise sequence from user selection to final output integration:

1. Client Selection. When a user selects a model in the Model/Provider dialog, the choice is persisted in the client-side settings store (store) via methods like setModelForStage.

2. Middleware Resolution. Upon receiving a generation request, middleware.ts invokes the resolveModelFromRequest function. This resolver:

   • Inspects the x-generation-stage header to identify the current stage (chat, TTS, image, etc.).
   • Retrieves the client-chosen model for that stage.
   • Validates against the server-side whitelist of permitted models for the authenticated user (as implemented in store/settings-server-sync.test.ts).
   • Falls back to the provider-default model if the client selection is unavailable or unsupported.

3. Stage Integration. Stage helpers such as applyGenerateTtsResultToScenes in tts-stage-sync.ts receive the completed tool output along with the model identifier, ensuring the UI accurately reflects which model generated the content.

Customizing Model Routing

You can adapt the routing behavior to meet specific deployment requirements through several mechanisms:

Add a new model for a specific stage by updating the provider's models array in the static configuration or via the admin API. Restart the development server so the middleware reads the updated registry.

Force a stage to use a particular model by overriding client settings in the store using setState({ selectedModelId: "gpt-4o-mini-tts" }) before sending the request. Alternatively, set the FORCE_MODEL environment variable on the server, which the resolver prioritizes over client selections.

Limit models per user by configuring the server-side synchronization logic in store/settings-server-sync.test.ts. This filters the providersConfig based on whitelisted model IDs, automatically pruning any unauthorized models from the client's available options.

Implementation Examples

Selecting a Model for the Chat Stage (Client-Side)

Use the settings store to persist user selections for specific generation stages:

import { useSettingsStore } from '@/lib/store';

// When the user picks a model from the dropdown:
function onModelSelect(modelId: string) {
  const settings = useSettingsStore.getState();
  settings.setModelForStage('chat', modelId);
}

Middleware Model Resolution (Server-Side)

The middleware.ts file implements the core routing logic through the resolveModelFromRequest function:

// middleware.ts – request interceptor
export async function resolveModelFromRequest(req) {
  const stage = req.headers['x-generation-stage'];   // e.g. "chat", "tts"
  const clientModel = req.headers['x-model-id'];    // optional
  const allowedModels = await getAllowedModelsForUser(req.user);

  // 1️⃣ Prefer the explicit client model if it exists and is allowed
  if (clientModel && allowedModels.includes(clientModel)) {
    return clientModel;
  }

  // 2️⃣ Fall back to the provider's default for the stage
  return getDefaultModelForStage(stage);
}

Applying TTS Results to Scenes

After text-to-speech generation completes, merge the results using the stage-specific helper:

import { applyGenerateTtsResultToScenes } from '@/lib/workbench/tts-stage-sync';

function handleToolResult(event) {
  // `event` is of type WorkbenchEvent from the session store
  const updatedScenes = applyGenerateTtsResultToScenes(currentScenes, event);
  setScenes(updatedScenes);
}

Enforcing Server-Side Model Whitelists

Implement access control by filtering available models based on user permissions:

// store/settings-server-sync.test.ts (illustrative logic)
const serverAllowed = {
  openai: { models: ['gpt-4o-mini', 'gpt-4o'] },
  anthropic: { models: ['claude-sonnet-5'] },
};

function filterModels(userConfig) {
  Object.entries(userConfig.providersConfig).forEach(([provider, cfg]) => {
    const whitelist = serverAllowed[provider]?.models ?? [];
    cfg.models = cfg.models.filter(m => whitelist.includes(m.id));
  });
}

Forcing Models via Environment Variables

Override all routing logic using server-side configuration:

# .env

FORCE_MODEL=gpt-4o-mini-tts

When FORCE_MODEL is set, the resolver in middleware.ts bypasses normal client-side selection and uses the forced value exclusively.

Summary

• Request interception occurs in middleware.ts, where resolveModelFromRequest determines the appropriate model based on stage headers and user permissions.
• Client preferences are stored using useSettingsStore and methods like setModelForStage, allowing per-stage model selection.
• Server enforcement happens through whitelist filtering in store/settings-server-sync.test.ts, validating models against user-specific allowances before execution.
• Stage integration relies on specialized helpers such as applyGenerateTtsResultToScenes in lib/workbench/tts-stage-sync.ts to attribute outputs to their generating models.
• Emergency overrides are available through the FORCE_MODEL environment variable, which takes precedence over all other routing logic.

Frequently Asked Questions

Where is the model routing logic implemented in OpenMAIC?

The primary routing logic resides in middleware.ts, specifically within the resolveModelFromRequest function. This middleware intercepts all generation requests, inspects headers like x-generation-stage and x-model-id, and resolves the final model by checking client preferences against server-side whitelists. Stage-specific handling is then delegated to helpers in the lib/workbench/ directory.

How do I force a specific model for a generation stage?

You have two options: client-side or server-side forcing. On the client, use useSettingsStore.getState().setModelForStage('stageName', 'model-id') before dispatching the request. On the server, set the FORCE_MODEL environment variable to the desired model identifier, which the middleware resolver checks with highest priority, bypassing client selections entirely.

Can I restrict which models a user can access?

Yes. Implement user-specific restrictions by modifying the synchronization logic in store/settings-server-sync.test.ts. This file demonstrates how to filter the providersConfig object against a whitelist of allowed model IDs for each authenticated user. The middleware then validates incoming requests against this filtered configuration, rejecting unauthorized model selections.

What happens if the selected model doesn't support the current stage?

If resolveModelFromRequest determines that the client-selected model is unavailable, unauthorized, or incompatible with the current stage (indicated by the x-generation-stage header), it automatically falls back to the provider's default model for that stage. This ensures the generation pipeline remains functional even when specific models are unavailable or misconfigured.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}