# How to Run and Configure the Router Service for FreeLLM API

> Learn to run the FreeLLM API router service with `setRoutingStrategy` and `setCustomWeights`. Configure settings instantly without restarts with this guide.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-07-02

---

**The router service is an integral component of the FreeLLM API server that you configure at runtime using `setRoutingStrategy()` and `setCustomWeights()`, with all settings persisted to SQLite and applied instantly without restarting.**

The **router service** in the `tashfeenahmed/freellmapi` repository handles intelligent model selection and API key management. Located in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), it operates as a core service within the main Express server, automatically activating when you start the FreeLLM API application.

## Starting the Router Service

The router service runs automatically when you start the FreeLLM API server. Unlike standalone microservices, it is an internal module that initializes with the main application process. Once the server is running, the router immediately begins processing requests using either default settings or previously persisted configurations from the SQLite `settings` table.

```typescript
// The router initializes automatically when the server starts
// No separate command is required to "run" the service
npm run start  # or your configured start command

```

## Configuring Routing Strategies

The router supports six distinct strategies that determine how it selects models from the fallback chain. You can switch strategies at runtime using `setRoutingStrategy()` defined in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) (lines 21-28).

Available strategies:
- **priority** – Uses the manual priority column for deterministic ordering
- **balanced** – Default strategy when custom weights are not selected
- **smartest** – Prioritizes model capability (intelligence score)
- **fastest** – Optimizes for lowest latency
- **reliable** – Prioritizes models with highest success rates
- **custom** – Uses user-defined weights for reliability, speed, and intelligence

```typescript
import { setRoutingStrategy, getRoutingStrategy } from '../services/router.js';

// Switch to the "smartest" strategy
setRoutingStrategy('smartest');

// Verify the current setting
console.log('Current routing strategy →', getRoutingStrategy());
// → Current routing strategy → smartest

```

Strategy changes persist to the `settings` table under the key `routing_strategy` and take effect on the next request.

## Setting Custom Weights

When using the `custom` strategy, you must define a normalized weight vector that determines how the router balances the three scoring axes. Use `setCustomWeights()` (lines 40-70 in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)) to configure these values.

```typescript
import { setRoutingStrategy, setCustomWeights } from '../services/router.js';

// First activate custom strategy
setRoutingStrategy('custom');

// Provide normalized weights (must sum to 1)
setCustomWeights({
  reliability: 0.5,   // Prioritize successful responses
  speed:       0.3,   // Value tokens-per-second
  intelligence:0.2    // Prefer more capable models
});

```

The function validates that all components are non-negative and that the sum is greater than zero before persisting the normalized vector to the `routing_custom_weights` key in the `settings` table.

## API Key Selection and Rate Limiting

For models with multiple API keys, the router uses round-robin selection via `selectKeyForModel()` (lines 38-46 in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)). The service maintains an in-memory `roundRobinIndex` per platform that persists only for the process lifetime.

The router also tracks **429 rate-limit penalties** dynamically. When a model returns a 429 error, `recordRateLimitHit()` applies a penalty that decays over time, influencing the model's position in the ordered chain. Successful requests via `recordSuccess()` reduce this penalty.

## Monitoring Router Performance

Inspect current configuration and model scores using `getRoutingScores()` to verify the router is operating according to your specifications.

```typescript
import {
  getRoutingStrategy,
  getCustomWeights,
  getRoutingScores
} from '../services/router.js';

const strategy = getRoutingStrategy();
const weights  = getCustomWeights();   // Returns balanced preset if unset
const scores   = getRoutingScores();   // Returns computed scores per model

console.log({ strategy, weights, scores });

```

`getRoutingScores()` returns each model's reliability, speed, intelligence, headroom, rate-limit factor, and final weighted score—identical to the data displayed in the web UI's Dashboard → Router page.

## How the Router Processes Requests

The routing logic follows a four-step pipeline defined in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts):

1. **Resolve Chain** – `getActiveChain()` reads the fallback profile from the database
2. **Order Chain** – `orderChain(chain, strategy)` applies the active strategy's weights and scores from `scoreChainEntry()`, incorporating live analytics and 429 penalties
3. **Select Key** – `selectKeyForModel()` validates API keys against cooldowns, quotas, and rate limits stored in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts)
4. **Fallback Loop** – If a model cannot serve the request, the router proceeds to the next model in the ordered chain, throwing a `RouteError` only when all models are exhausted

## Key Source Files

| File | Role |
|------|------|
| [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) | Central routing logic, strategy handling, and persistence |
| [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts) | Score calculation for reliability, speed, intelligence, and headroom |
| [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) | Per-model cooldowns and RPM/RPD/TPM/TPD enforcement |
| [`server/src/routes/fallback.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/fallback.ts) | HTTP endpoints for managing fallback chains and router settings |
| [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) | Main request entry point calling `routeRequest()` |

## Summary

- The router service runs automatically as part of the main FreeLLM API server process
- Configure strategies using `setRoutingStrategy()` with options: priority, balanced, smartest, fastest, reliable, or custom
- Set custom weights via `setCustomWeights()` when using the custom strategy; values must normalize to 1.0
- All configurations persist to the SQLite `settings` table and apply instantly without restart
- The router uses round-robin key selection and dynamic penalties for rate-limited models (429s)
- Monitor performance through `getRoutingScores()` which exposes the same data as the Dashboard UI

## Frequently Asked Questions

### What happens if all models in the fallback chain fail?

The router throws a `RouteError` with an "all models exhausted" message. This occurs when every model in the ordered chain either lacks usable API keys, is in cooldown, has exceeded token budgets, or fails rate-limit checks. The error propagates through [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) to the client.

### How do custom weights interact with the penalty system?

Penalties from `recordRateLimitHit()` modify the final score calculation in `orderChain()` after the strategy weights are applied. Even with high reliability weights, a model receiving repeated 429 errors will drop in the chain ranking due to the dynamic penalty factor, ensuring the router avoids recently rate-limited providers.

### Can I change the routing strategy via HTTP API?

Yes. The [`server/src/routes/fallback.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/fallback.ts) file exposes endpoints that wrap `setRoutingStrategy()` and `setCustomWeights()`. POST requests to `/v1/admin/router/strategy` update the active strategy, while POST requests to `/v1/admin/router/weights` persist custom weight vectors, both writing to the SQLite `settings` table via [`declarative-config.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/declarative-config.ts).

### Why does the round-robin index reset when I restart the server?

The `roundRobinIndex` used by `selectKeyForModel()` is stored in-process memory (a JavaScript `Map`) rather than the database. This state persists only for the life of the Node.js process and resets to zero on server restart, ensuring fair key distribution across uptime sessions but not surviving restarts.