How to Run and Configure the Router Service for FreeLLM API

The router service is an integral component of the FreeLLM API server that you configure at runtime using setRoutingStrategy() and setCustomWeights(), with all settings persisted to SQLite and applied instantly without restarting.

The router service in the tashfeenahmed/freellmapi repository handles intelligent model selection and API key management. Located in server/src/services/router.ts, it operates as a core service within the main Express server, automatically activating when you start the FreeLLM API application.

Starting the Router Service

The router service runs automatically when you start the FreeLLM API server. Unlike standalone microservices, it is an internal module that initializes with the main application process. Once the server is running, the router immediately begins processing requests using either default settings or previously persisted configurations from the SQLite settings table.

// The router initializes automatically when the server starts
// No separate command is required to "run" the service
npm run start  # or your configured start command

Configuring Routing Strategies

The router supports six distinct strategies that determine how it selects models from the fallback chain. You can switch strategies at runtime using setRoutingStrategy() defined in router.ts (lines 21-28).

Available strategies:

• priority – Uses the manual priority column for deterministic ordering
• balanced – Default strategy when custom weights are not selected
• smartest – Prioritizes model capability (intelligence score)
• fastest – Optimizes for lowest latency
• reliable – Prioritizes models with highest success rates
• custom – Uses user-defined weights for reliability, speed, and intelligence

import { setRoutingStrategy, getRoutingStrategy } from '../services/router.js';

// Switch to the "smartest" strategy
setRoutingStrategy('smartest');

// Verify the current setting
console.log('Current routing strategy →', getRoutingStrategy());
// → Current routing strategy → smartest

Strategy changes persist to the settings table under the key routing_strategy and take effect on the next request.

Setting Custom Weights

When using the custom strategy, you must define a normalized weight vector that determines how the router balances the three scoring axes. Use setCustomWeights() (lines 40-70 in router.ts) to configure these values.

import { setRoutingStrategy, setCustomWeights } from '../services/router.js';

// First activate custom strategy
setRoutingStrategy('custom');

// Provide normalized weights (must sum to 1)
setCustomWeights({
  reliability: 0.5,   // Prioritize successful responses
  speed:       0.3,   // Value tokens-per-second
  intelligence:0.2    // Prefer more capable models
});

The function validates that all components are non-negative and that the sum is greater than zero before persisting the normalized vector to the routing_custom_weights key in the settings table.

API Key Selection and Rate Limiting

For models with multiple API keys, the router uses round-robin selection via selectKeyForModel() (lines 38-46 in router.ts). The service maintains an in-memory roundRobinIndex per platform that persists only for the process lifetime.

The router also tracks 429 rate-limit penalties dynamically. When a model returns a 429 error, recordRateLimitHit() applies a penalty that decays over time, influencing the model's position in the ordered chain. Successful requests via recordSuccess() reduce this penalty.

Monitoring Router Performance

Inspect current configuration and model scores using getRoutingScores() to verify the router is operating according to your specifications.

import {
  getRoutingStrategy,
  getCustomWeights,
  getRoutingScores
} from '../services/router.js';

const strategy = getRoutingStrategy();
const weights  = getCustomWeights();   // Returns balanced preset if unset
const scores   = getRoutingScores();   // Returns computed scores per model

console.log({ strategy, weights, scores });

getRoutingScores() returns each model's reliability, speed, intelligence, headroom, rate-limit factor, and final weighted score—identical to the data displayed in the web UI's Dashboard → Router page.

How the Router Processes Requests

The routing logic follows a four-step pipeline defined in server/src/services/router.ts:

1. Resolve Chain – getActiveChain() reads the fallback profile from the database
2. Order Chain – orderChain(chain, strategy) applies the active strategy's weights and scores from scoreChainEntry(), incorporating live analytics and 429 penalties
3. Select Key – selectKeyForModel() validates API keys against cooldowns, quotas, and rate limits stored in ratelimit.ts
4. Fallback Loop – If a model cannot serve the request, the router proceeds to the next model in the ordered chain, throwing a RouteError only when all models are exhausted

Key Source Files

File	Role
server/src/services/router.ts	Central routing logic, strategy handling, and persistence
server/src/services/scoring.ts	Score calculation for reliability, speed, intelligence, and headroom
server/src/services/ratelimit.ts	Per-model cooldowns and RPM/RPD/TPM/TPD enforcement
server/src/routes/fallback.ts	HTTP endpoints for managing fallback chains and router settings
server/src/routes/proxy.ts	Main request entry point calling routeRequest()

Summary

• The router service runs automatically as part of the main FreeLLM API server process
• Configure strategies using setRoutingStrategy() with options: priority, balanced, smartest, fastest, reliable, or custom
• Set custom weights via setCustomWeights() when using the custom strategy; values must normalize to 1.0
• All configurations persist to the SQLite settings table and apply instantly without restart
• The router uses round-robin key selection and dynamic penalties for rate-limited models (429s)
• Monitor performance through getRoutingScores() which exposes the same data as the Dashboard UI

Frequently Asked Questions

What happens if all models in the fallback chain fail?

The router throws a RouteError with an "all models exhausted" message. This occurs when every model in the ordered chain either lacks usable API keys, is in cooldown, has exceeded token budgets, or fails rate-limit checks. The error propagates through server/src/routes/proxy.ts to the client.

How do custom weights interact with the penalty system?

Penalties from recordRateLimitHit() modify the final score calculation in orderChain() after the strategy weights are applied. Even with high reliability weights, a model receiving repeated 429 errors will drop in the chain ranking due to the dynamic penalty factor, ensuring the router avoids recently rate-limited providers.

Can I change the routing strategy via HTTP API?

Yes. The server/src/routes/fallback.ts file exposes endpoints that wrap setRoutingStrategy() and setCustomWeights(). POST requests to /v1/admin/router/strategy update the active strategy, while POST requests to /v1/admin/router/weights persist custom weight vectors, both writing to the SQLite settings table via declarative-config.ts.

Why does the round-robin index reset when I restart the server?

The roundRobinIndex used by selectKeyForModel() is stored in-process memory (a JavaScript Map) rather than the database. This state persists only for the life of the Node.js process and resets to zero on server restart, ensuring fair key distribution across uptime sessions but not surviving restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}