How to Configure the Router Service in FreeLLM API

The router service selects the optimal model and API key for each request by applying a configurable routing strategy—such as balanced, fastest, or custom—to a prioritized fallback chain, storing your configuration in SQLite and applying changes instantly without a server restart.

The router service is the core traffic controller of the open-source FreeLLM API project (tashfeenahmed/freellmapi). It determines which model handles incoming requests based on dynamic scoring, rate-limit status, and your preferred routing strategy. All configuration persists to the local SQLite database and takes effect immediately at runtime.

Understanding the Router Service Architecture

The central logic resides in server/src/services/router.ts. This service coordinates with scoring.ts for model analytics, ratelimit.ts for quota enforcement, and model-groups.ts for fusion panel logic. When a request arrives, the router resolves the fallback chain, orders it according to your active routing strategy, and selects a usable API key via round-robin selection. If a model is rate-limited or exhausted, the router automatically attempts the next candidate in the chain.

Configuring the Routing Strategy

The routing strategy determines how the router orders the fallback chain. You can switch strategies at runtime by calling setRoutingStrategy() in server/src/services/router.ts (lines 21–28), which persists the value to the SQLite settings table under the key routing_strategy.

The following strategies are available:

• priority – Uses manual ordering from the fallback configuration.
• balanced – Equal weighting across all performance axes.
• smartest – Prioritizes model capability (intelligence score).
• fastest – Prioritizes low latency and high tokens-per-second.
• reliable – Prioritizes uptime and successful response rates.
• custom – Applies user-defined weights for reliability, speed, and intelligence.

To change the strategy programmatically:

import { setRoutingStrategy, getRoutingStrategy } from '../services/router.js';

// Switch to the "smartest" strategy
setRoutingStrategy('smartest');

// Verify the current setting
console.log('Current routing strategy:', getRoutingStrategy());
// → Current routing strategy: smartest

Defining Custom Weights for Advanced Routing

When you select the custom strategy, you must supply a weight vector via setCustomWeights() (defined in server/src/services/router.ts lines 40–70). The router validates that the components are non-negative and normalizes them so they sum to 1.0.

The three configurable axes are:

• Reliability – Historical success rate and uptime.
• Speed – Latency and throughput metrics.
• Intelligence – Model capability benchmarks.

Example configuration:

import { setRoutingStrategy, setCustomWeights } from '../services/router.js';

// Activate custom strategy first
setRoutingStrategy('custom');

// Define weights (must sum to 1.0)
setCustomWeights({
  reliability: 0.5,
  speed: 0.3,
  intelligence: 0.2
});

These weights are stored in the SQLite settings table under the key routing_custom_weights and influence the scoring calculation performed by scoreChainEntry().

Monitoring Router Configuration and Model Scores

To inspect the current router state and verify how models are being ranked, use getRoutingScores() along with getRoutingStrategy() and getCustomWeights(). This returns the computed score for each model in the fallback chain, including the live 429 penalty (decaying in-memory) and headroom factors.

import {
  getRoutingStrategy,
  getCustomWeights,
  getRoutingScores
} from '../services/router.js';

const strategy = getRoutingStrategy();
const weights = getCustomWeights();   // null unless strategy is custom
const scores = getRoutingScores();    // Live computed rankings

console.log({ strategy, weights, scores });

The getRoutingScores() function returns an array of objects containing each model’s reliability, speed, intelligence, rateLimitFactor, and final score, which powers the Dashboard → Router UI view.

How Configuration Flows Through the System

When a request enters through server/src/routes/proxy.ts, the router executes the following steps using your configuration:

1. Resolve Chain – getActiveChain() retrieves the fallback profile from the database.
2. Order Chain – orderChain() applies the active strategy’s weight vector (or null for legacy priority mode) and computes a final score via scoring.ts.
3. Select Key – selectKeyForModel() iterates through API keys for the selected model, applying the in-memory roundRobinIndex, cooldown checks from ratelimit.ts, and decryption validation.
4. Fallback Loop – If the selected model cannot serve the request (quota exceeded, rate limit, or no valid keys), the router moves to the next model in the ordered chain. If all models fail, it throws a RouteError.

Summary

• The router service is implemented in server/src/services/router.ts and manages model selection via a fallback chain and dynamic scoring.
• Use setRoutingStrategy() to switch between priority, balanced, smartest, fastest, reliable, or custom modes; values persist in the SQLite settings table under routing_strategy.
• When using the custom strategy, define a normalized weight vector via setCustomWeights() to balance reliability, speed, and intelligence; stored under routing_custom_weights.
• Call getRoutingScores() to inspect real-time model rankings, including live penalties and rate-limit factors.
• Changes take effect immediately without restarting the server, as the router reads configuration from the database on each request cycle.

Frequently Asked Questions

Where is the router configuration stored in FreeLLM API?

The configuration is stored in the SQLite database’s settings table. The active routing strategy is saved under the key routing_strategy, and custom weights are stored under routing_custom_weights when using the custom strategy. These values are read on each request cycle, allowing dynamic updates.

What routing strategies are available in FreeLLM API?

The router supports six strategies defined in server/src/services/router.ts: priority (manual ordering), balanced (equal weighting), smartest (favors intelligence), fastest (favors latency), reliable (favors uptime), and custom (user-defined weights). Set these via setRoutingStrategy() to immediately change routing behavior.

How do custom weights work in the router service?

When you set the strategy to custom, you can supply a weight object via setCustomWeights() containing values for reliability, speed, and intelligence. The router normalizes these weights so they sum to 1.0 and applies them to the scoring algorithm in scoring.ts, effectively reordering the fallback chain based on your priorities.

Does changing the router strategy require a server restart?

No. Changes made via setRoutingStrategy() or setCustomWeights() are written immediately to the SQLite database and applied to the next incoming request. The router queries the settings table fresh for each route decision, enabling zero-downtime configuration updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}