How to Override Default Routing Behavior Per Request in FreeLLMAPI

Send an X-FreeLLMAPI-Routing-Override HTTP header containing a JSON payload with parameters like strategy, keySelection, or explore to temporarily change routing logic for a single request without modifying global settings.

FreeLLMAPI uses a sophisticated router with bandit-style scoring, rate-limit caps, and head-room thresholds to determine which model and API key to use for each request. While the global configuration handles most traffic patterns, you often need to deviate from these defaults for specific API calls—such as forcing the fastest model for latency-sensitive operations or exploring new models during beta testing. By sending a per-request routing override in a dedicated HTTP header, you can modify routing behavior on the fly while leaving the persisted database and environment configuration untouched.

Understanding the X-FreeLLMAPI-Routing-Override Header

The X-FreeLLMAPI-Routing-Override header accepts a UTF-8 encoded JSON object containing one or more override keys. The router merges these temporary settings with the global configuration for the duration of the request, then discards them after completion.

Supported Override Parameters

All override keys are optional—omitted values fall back to global settings:

• strategy – Switches the routing strategy to priority, balanced, smartest, fastest, reliable, or custom for the current request only.
• keySelection – Changes the key-selection policy to auto or least-remaining to control API key consumption.
• customWeights – Supplies a temporary weight vector with reliability, speed, and intelligence values to customize scoring.
• explore – Forces the router to explore unmeasured models by setting true or false, overriding the global explore_enabled setting.
• headroom – Provides ad-hoc head-room thresholds with rampStart and floor values to adjust capacity limits for high-budget requests.

The header name is case-insensitive, but the JSON must not contain secrets or sensitive credentials. Unknown keys are silently ignored by the router.

Implementation Examples

cURL Example

Use this approach for testing or shell-based automation where you need lowest-latency routing:

curl https://api.freellmapi.com/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" \
  -H "Content-Type: application/json" \
  -H "X-FreeLLMAPI-Routing-Override: {\"strategy\":\"fastest\",\"keySelection\":\"least-remaining\"}" \
  -d '{
        "model":"gpt-4o-mini",
        "messages":[{"role":"user","content":"What is the weather in Paris?"}]
      }'

JavaScript Fetch Example

For Node.js applications requiring dynamic exploration of new models:

await fetch("https://api.freellmapi.com/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.FREELLMAPI_KEY}`,
    "Content-Type": "application/json",
    "X-FreeLLMAPI-Routing-Override":
      JSON.stringify({ explore: true, headroom: { rampStart: 0.3, floor: 0.1 } })
  },
  body: JSON.stringify({
    model: "auto",
    messages: [{ role: "user", content: "Explain quantum tunnelling." }]
  })
});

How the Server Processes Routing Overrides

The request lifecycle handles per-request overrides through specific pipeline stages in the FreeLLMAPI source code:

1. Header Extraction – In server/src/routes/proxy.ts (approximately lines 450–460), the router middleware extracts the header via parseRoutingOverride(req.headers.get('x-freellmapi-routing-override')).

2. Validation – The server/src/services/router.ts module validates the payload against the internal RoutingOverrideSchema (defined around line 1905) using Zod. Invalid fields are discarded without error, allowing the request to proceed with global defaults.

3. Temporary Context Creation – The applyRoutingOverrides function (approximately line 2152 in router.ts) merges the parsed override with persisted settings, creating a temporary RoutingContext object.

4. Routing Decision – The core routeRequest pipeline uses the merged RoutingContext for all scoring, head-room calculations, and key-selection logic. The normal flow continues with lease acquisition, model selection, and provider calls.

5. Cleanup – After the request completes, the temporary override context is discarded. No changes are written to the database or .env files.

Common Use Cases

• Latency-sensitive operations – Force the fastest available model with { "strategy":"fastest" } for real-time applications.
• Quota management – Prefer less-used API keys using { "keySelection":"least-remaining" } to avoid hitting rate limits.
• Model exploration – Enable { "explore":true } during beta testing to route traffic to newly added models with insufficient historical data.
• Custom scoring experiments – Apply temporary weight vectors like { "customWeights":{ "reliability":0.5, "speed":0.3, "intelligence":0.2 } } for A/B testing routing strategies.
• High-budget requests – Adjust capacity thresholds with { "headroom":{ "rampStart":0.4, "floor":0.15 } } for premium workloads.

Key Source Files for Reference

• server/src/routes/proxy.ts – Entry point for the /v1/chat/completions endpoint; extracts and initial-parses the override header.
• server/src/services/router.ts – Core routing engine containing RoutingOverrideSchema, applyRoutingOverrides, and the routeRequest implementation.
• server/src/services/model-weight-overrides.ts – Handles global per-model weight overrides (distinct from per-request overrides).
• shared/types.ts – TypeScript definitions for RoutingContext and override object shapes.

Summary

• Send the X-FreeLLMAPI-Routing-Override header as a JSON object to modify routing for single requests.
• Supported keys include strategy, keySelection, customWeights, explore, and headroom.
• The server validates overrides against RoutingOverrideSchema in server/src/services/router.ts and merges them via applyRoutingOverrides.
• Overrides affect only the current request lifecycle and leave global configurations untouched.
• Reference server/src/routes/proxy.ts for header extraction logic and router.ts for validation and merging implementations.

Frequently Asked Questions

What is the exact header name for per-request routing overrides?

The header name is X-FreeLLMAPI-Routing-Override. According to the source code in server/src/routes/proxy.ts, the header is case-insensitive during extraction, but conventionally uses this PascalCase format with hyphens.

Can I combine multiple routing parameters in a single override?

Yes. The JSON payload accepts multiple keys simultaneously. For example, you can combine {"strategy":"fastest","explore":true,"keySelection":"least-remaining"} to apply all three overrides to the same request. The router merges these with global defaults, applying only the specified overrides while maintaining other settings.

Do per-request routing overrides persist after the request completes?

No. The override context exists only for the duration of the request lifecycle. The applyRoutingOverrides function in server/src/services/router.ts creates a temporary merge that is discarded after routeRequest finishes. No data is written to the database or environment files.

What happens if I send an invalid JSON payload in the override header?

The router validates the payload against RoutingOverrideSchema using Zod. Invalid fields or malformed JSON are silently ignored, and the request proceeds using only the global configuration. The server does not return an error for invalid override syntax to ensure high availability, though invalid overrides are logged for debugging purposes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}