# How to Configure Model Routing for Different Generation Stages in OpenMAIC

> Configure model routing for different generation stages in OpenMAIC. Learn to modify provider configs, implement custom middleware, and specify models via client or server settings.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-08-30

---

**To configure model routing in OpenMAIC, modify the provider configuration registry, implement custom logic in the [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts) resolver, and specify stage-specific models using the client-side settings store or server-side environment variables like `FORCE_MODEL`.**

OpenMAIC separates its generation pipeline into distinct stages—such as chat-completion, text-to-speech (TTS), and image generation—each potentially requiring different underlying models. This open-source framework from THU-MAIC orchestrates model selection through a sophisticated routing system that balances client preferences with server-side enforcement. Understanding how to configure model routing for different generation stages in OpenMAIC enables developers to customize AI workflows, enforce security policies, and optimize inference across heterogeneous provider configurations.

## Core Routing Architecture

OpenMAIC implements model routing through three interconnected components that work together at runtime:

**Provider configuration** acts as a static registry mapping providers (OpenAI, Anthropic, etc.) to their available models. This configuration lives implicitly within the store system and is validated against server-side allowances.

**Middleware resolver** handles request-level resolution. Located in [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts), this component inspects incoming requests and determines which client model (UI-selected) or server model (backend-permitted) should handle the current generation stage.

**Stage-specific helpers** manage post-processing integration. After a tool finishes execution—such as `generate_tts` in [`lib/workbench/tts-stage-sync.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/tts-stage-sync.ts)—these helpers merge results back into the live scene using the exact model identifier that produced the output.

## The Model Resolution Pipeline

The routing logic follows a precise sequence from user selection to final output integration:

1.  **Client Selection.** When a user selects a model in the Model/Provider dialog, the choice is persisted in the client-side settings store (`store`) via methods like `setModelForStage`.

2.  **Middleware Resolution.** Upon receiving a generation request, [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts) invokes the `resolveModelFromRequest` function. This resolver:
    *   Inspects the `x-generation-stage` header to identify the current stage (chat, TTS, image, etc.).
    *   Retrieves the client-chosen model for that stage.
    *   Validates against the server-side whitelist of permitted models for the authenticated user (as implemented in [`store/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/store/settings-server-sync.test.ts)).
    *   Falls back to the provider-default model if the client selection is unavailable or unsupported.

3.  **Stage Integration.** Stage helpers such as `applyGenerateTtsResultToScenes` in [`tts-stage-sync.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tts-stage-sync.ts) receive the completed tool output along with the model identifier, ensuring the UI accurately reflects which model generated the content.

## Customizing Model Routing

You can adapt the routing behavior to meet specific deployment requirements through several mechanisms:

**Add a new model for a specific stage** by updating the provider's `models` array in the static configuration or via the admin API. Restart the development server so the middleware reads the updated registry.

**Force a stage to use a particular model** by overriding client settings in the store using `setState({ selectedModelId: "gpt-4o-mini-tts" })` before sending the request. Alternatively, set the `FORCE_MODEL` environment variable on the server, which the resolver prioritizes over client selections.

**Limit models per user** by configuring the server-side synchronization logic in [`store/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/store/settings-server-sync.test.ts). This filters the `providersConfig` based on whitelisted model IDs, automatically pruning any unauthorized models from the client's available options.

## Implementation Examples

### Selecting a Model for the Chat Stage (Client-Side)

Use the settings store to persist user selections for specific generation stages:

```typescript
import { useSettingsStore } from '@/lib/store';

// When the user picks a model from the dropdown:
function onModelSelect(modelId: string) {
  const settings = useSettingsStore.getState();
  settings.setModelForStage('chat', modelId);
}

```

### Middleware Model Resolution (Server-Side)

The [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts) file implements the core routing logic through the `resolveModelFromRequest` function:

```typescript
// middleware.ts – request interceptor
export async function resolveModelFromRequest(req) {
  const stage = req.headers['x-generation-stage'];   // e.g. "chat", "tts"
  const clientModel = req.headers['x-model-id'];    // optional
  const allowedModels = await getAllowedModelsForUser(req.user);

  // 1️⃣ Prefer the explicit client model if it exists and is allowed
  if (clientModel && allowedModels.includes(clientModel)) {
    return clientModel;
  }

  // 2️⃣ Fall back to the provider's default for the stage
  return getDefaultModelForStage(stage);
}

```

### Applying TTS Results to Scenes

After text-to-speech generation completes, merge the results using the stage-specific helper:

```typescript
import { applyGenerateTtsResultToScenes } from '@/lib/workbench/tts-stage-sync';

function handleToolResult(event) {
  // `event` is of type WorkbenchEvent from the session store
  const updatedScenes = applyGenerateTtsResultToScenes(currentScenes, event);
  setScenes(updatedScenes);
}

```

### Enforcing Server-Side Model Whitelists

Implement access control by filtering available models based on user permissions:

```typescript
// store/settings-server-sync.test.ts (illustrative logic)
const serverAllowed = {
  openai: { models: ['gpt-4o-mini', 'gpt-4o'] },
  anthropic: { models: ['claude-sonnet-5'] },
};

function filterModels(userConfig) {
  Object.entries(userConfig.providersConfig).forEach(([provider, cfg]) => {
    const whitelist = serverAllowed[provider]?.models ?? [];
    cfg.models = cfg.models.filter(m => whitelist.includes(m.id));
  });
}

```

### Forcing Models via Environment Variables

Override all routing logic using server-side configuration:

```bash

# .env

FORCE_MODEL=gpt-4o-mini-tts

```

When `FORCE_MODEL` is set, the resolver in [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts) bypasses normal client-side selection and uses the forced value exclusively.

## Summary

- **Request interception** occurs in [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts), where `resolveModelFromRequest` determines the appropriate model based on stage headers and user permissions.
- **Client preferences** are stored using `useSettingsStore` and methods like `setModelForStage`, allowing per-stage model selection.
- **Server enforcement** happens through whitelist filtering in [`store/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/store/settings-server-sync.test.ts), validating models against user-specific allowances before execution.
- **Stage integration** relies on specialized helpers such as `applyGenerateTtsResultToScenes` in [`lib/workbench/tts-stage-sync.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/tts-stage-sync.ts) to attribute outputs to their generating models.
- **Emergency overrides** are available through the `FORCE_MODEL` environment variable, which takes precedence over all other routing logic.

## Frequently Asked Questions

### Where is the model routing logic implemented in OpenMAIC?

The primary routing logic resides in [`middleware.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/middleware.ts), specifically within the `resolveModelFromRequest` function. This middleware intercepts all generation requests, inspects headers like `x-generation-stage` and `x-model-id`, and resolves the final model by checking client preferences against server-side whitelists. Stage-specific handling is then delegated to helpers in the `lib/workbench/` directory.

### How do I force a specific model for a generation stage?

You have two options: client-side or server-side forcing. On the client, use `useSettingsStore.getState().setModelForStage('stageName', 'model-id')` before dispatching the request. On the server, set the `FORCE_MODEL` environment variable to the desired model identifier, which the middleware resolver checks with highest priority, bypassing client selections entirely.

### Can I restrict which models a user can access?

Yes. Implement user-specific restrictions by modifying the synchronization logic in [`store/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/store/settings-server-sync.test.ts). This file demonstrates how to filter the `providersConfig` object against a whitelist of allowed model IDs for each authenticated user. The middleware then validates incoming requests against this filtered configuration, rejecting unauthorized model selections.

### What happens if the selected model doesn't support the current stage?

If `resolveModelFromRequest` determines that the client-selected model is unavailable, unauthorized, or incompatible with the current stage (indicated by the `x-generation-stage` header), it automatically falls back to the provider's default model for that stage. This ensures the generation pipeline remains functional even when specific models are unavailable or misconfigured.