FreeLLMAPI Routing Strategies: A Complete Guide to Model Selection Weights

FreeLLMAPI provides six distinct routing strategies—priority, balanced, smartest, fastest, reliable, and custom—that use weighted scoring vectors to determine which LLM provider handles each request.

The open-source FreeLLMAPI repository (tashfeenahmed/freellmapi) implements a pluggable router architecture that dynamically selects model-provider keys based on configurable optimization criteria. Understanding these routing strategies allows developers to fine-tune whether the system prioritizes latency, accuracy, uptime, or explicit administrative ordering.

What Are Routing Strategies in FreeLLMAPI?

At the core of FreeLLMAPI's request handling is a routing strategy—a preset weight vector defined in server/src/services/scoring.ts that instructs the engine how to value competing attributes like speed, reliability, and intelligence. When a request arrives, the router evaluates candidate models using these weights, computes a composite score for each option, and selects the highest-scoring provider that passes health and quota checks.

The strategy system abstracts complex multi-armed bandit logic into simple configuration presets. Operators can switch between strategies at runtime via the MCP API or programmatically within the server code without redeploying the application.

The Six Built-In Routing Strategies

FreeLLMAPI ships with six predefined routing strategies, each optimized for specific operational requirements.

Priority Strategy

The priority strategy implements deterministic, admin-controlled routing where the router follows the explicit chain order defined in the configuration. In server/src/services/router.ts at line 395, this strategy sets weights: null, effectively disabling the scoring algorithm and honoring the manual provider sequence.

Use this strategy when you require predictable, compliance-friendly ordering or when testing specific provider chains in development environments.

Balanced Strategy

The balanced strategy provides a neutral blend that values speed, reliability, and intelligence equally. It applies the preset BANDIT_PRESETS.balanced with approximately 0.33 weights distributed across all three dimensions.

This strategy serves as the general-purpose default, delivering a well-rounded experience without biasing toward any single metric.

Smartest Strategy

When generation quality takes precedence over latency, the smartest strategy prioritizes model intelligence. Using BANDIT_PRESETS.smartest, it assigns roughly 0.55 weight to intelligence while reducing speed to approximately 0.10.

Deploy this strategy for creative writing, complex problem-solving, or research tasks where output quality matters more than response time.

Fastest Strategy

The fastest strategy optimizes for low latency by boosting the speed weight to approximately 0.55 while reducing intelligence weight to around 0.10 via BANDIT_PRESETS.fastest.

This configuration suits real-time applications such as chat assistants, UI-responsive bots, or streaming interfaces where user-perceived delay must be minimized.

Reliable Strategy

For mission-critical workloads where failures are costly, the reliable strategy applies BANDIT_PRESETS.reliable with heavy weighting (≈ 0.55) toward reliability metrics such as success rate and error reduction.

Use this strategy for production pipelines, automated workflows, or any scenario where provider uptime consistency outweighs raw performance or capability.

Custom Strategy

Advanced users can supply bespoke weight vectors using the custom strategy. When selected, the router reads a persisted RoutingWeights record (as implemented in server/src/services/router.ts) and applies those user-defined values instead of presets.

This strategy enables fine-tuned configurations such as "fast-but-reliable" or "smart-with-head-room" hybrids that match specific business requirements.

How Routing Strategies Work Under the Hood

The routing engine in server/src/services/router.ts implements the setRoutingStrategy and getRoutingStrategy functions that manage the active strategy state. When processing a request, the router:

1. Retrieves the current strategy configuration from the scoring module
2. Evaluates each candidate model/key against the strategy's weight vector
3. Computes a composite score based on real-time performance metrics
4. Selects the highest-scoring provider that satisfies quota and health checks

The weight vector definitions reside in server/src/services/scoring.ts, which exports both the RoutingStrategy type union and the BANDIT_PRESETS constant containing the numerical weight mappings.

Implementing Routing Strategies in Your Application

Changing Strategies via the MCP API

FreeLLMAPI exposes a runtime configuration endpoint that allows operators to switch strategies without restarting the server. The implementation in server/src/routes/mcp.ts handles POST requests to /v1/mcp/routing/strategy, which internally calls setRoutingStrategy.

// Example: POST /v1/mcp/routing/strategy
import fetch from 'node-fetch';

const resp = await fetch('https://api.freellmapi.com/v1/mcp/routing/strategy', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ strategy: 'smartest' }),
});

const data = await resp.json();
console.log('Strategy set to:', data.strategy);

Programmatic Server-Side Configuration

For internal services or initialization scripts, import the router service directly to manipulate策略 programmatically:

import { setRoutingStrategy, getRoutingStrategy } from '../services/router.js';

// Switch to the fastest preset
setRoutingStrategy('fastest');

console.log('Current strategy →', getRoutingStrategy()); // → 'fastest'

Client-Side Type Safety

The client library ships with matching type definitions in client/src/lib/routing.ts, ensuring type-safe strategy selection in browser or Node.js environments:

import { setRoutingStrategy, type RoutingStrategy } from '@freellmapi/client';

const chosen: RoutingStrategy = 'balanced';
setRoutingStrategy(chosen);

Custom weight vectors configured through the custom strategy are persisted via server/src/services/declarative-config.ts, ensuring that bespoke routing preferences survive server restarts.

Summary

• Six routing strategies (priority, balanced, smartest, fastest, reliable, custom) control how FreeLLMAPI selects LLM providers for each request.
• Weight vectors defined in server/src/services/scoring.ts quantitatively balance speed, reliability, and intelligence metrics.
• Priority strategy bypasses scoring entirely for deterministic admin-controlled ordering, while custom strategy enables user-defined weight configurations.
• Runtime strategy switching is supported via the MCP API endpoint /v1/mcp/routing/strategy or programmatically through setRoutingStrategy.
• Type definitions are shared between server (server/src/services/router.ts) and client (client/src/lib/routing.ts) for consistent implementation across the stack.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

The balanced strategy typically serves as the default configuration, applying equal weights (approximately 0.33) to speed, reliability, and intelligence metrics. However, the active strategy depends on the runtime configuration set via the MCP API or initialization parameters in server/src/services/router.ts.

Can I create custom weight vectors for model routing?

Yes, the custom routing strategy allows you to define bespoke weight vectors through a RoutingWeights record. When selected, the router reads these persisted values from server/src/services/declarative-config.ts instead of using the preset constants, enabling tailored optimization for specific latency, accuracy, or uptime requirements.

How does the priority strategy differ from balanced routing?

The priority strategy disables the scoring algorithm entirely by setting weights: null in the router implementation, forcing the system to follow the explicit provider chain order defined by administrators. In contrast, balanced routing applies the BANDIT_PRESETS.balanced weight vector to dynamically score and select providers based on real-time performance metrics.

Where are custom routing weights persisted?

Custom routing weights configured through the custom strategy are stored and retrieved via server/src/services/declarative-config.ts. This persistence layer ensures that user-defined weight vectors survive server restarts and remain active until explicitly modified through the router API or programmatic interfaces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}