How to Configure FreeLLMAPI Routing Strategies: A Complete Developer Guide

FreeLLMAPI routing strategies determine how LLM requests are distributed across available models, using built-in presets like balanced and fastest or custom weight vectors configured via the router.ts service or HTTP API.

The tashfeenahmed/freellmapi repository provides a flexible routing engine that directs incoming LLM requests to the optimal model based on configurable strategies. When you configure FreeLLMAPI routing strategies, you control whether the system prioritizes speed, reliability, intelligence, or your own custom criteria. These settings persist in the SQLite settings table and are consulted by the router on every request.

Understanding FreeLLMAPI Routing Strategies

The routing engine in server/src/services/router.ts implements a multi-armed bandit approach to model selection. Each strategy represents a different weight vector that scores candidate models based on three dimensions: speed, reliability, and intelligence. When a request arrives, the router builds an orderChain of candidate models and applies the active weight vector to calculate a convex score for each option.

The system defaults to the balanced strategy if no configuration exists, as defined by the DEFAULT_STRATEGY constant on line 59 of router.ts. Strategies are validated against a whitelist before persistence, ensuring only supported values enter the database.

Available Routing Strategies

FreeLLMAPI offers six distinct routing strategies defined in server/src/services/scoring.ts (lines 31-55). You can select any of these presets or define your own weight distribution.

• priority: Legacy manual ordering that follows the configured model chain without scoring.
• balanced: Equal weighting across speed, reliability, and intelligence dimensions.
• smartest: Prioritizes the intelligence dimension for complex reasoning tasks.
• fastest: Maximizes throughput by weighting speed highest.
• reliable: Favors consistency and uptime metrics over raw performance.
• custom: Uses user-defined weights stored under the routing_custom_weights setting.

When using the custom strategy, the router normalizes your weight vector on write and applies it directly to the scoring calculation.

Configuring Strategies Programmatically

The router.ts module exports getter and setter functions that allow runtime configuration. All changes persist to the SQLite settings table and take effect immediately on subsequent requests.

Switching to Built-in Presets

To change the active strategy, use setRoutingStrategy() with any valid preset name. The function validates against the whitelist before writing to the database.

import { setRoutingStrategy, getRoutingStrategy } from './server/src/services/router.js';

await setRoutingStrategy('fastest');
console.log('Current strategy:', getRoutingStrategy());
// Current strategy: fastest

Enabling Exploration Mode

The router supports an exploration mechanism controlled by EXPLORE_CHANCE (10% by default). When enabled via routing_explore_enabled, unmeasured models receive a probability boost to ensure new providers get fair evaluation.

import { setExploreEnabled, getExploreEnabled } from './server/src/services/router.js';

await setExploreEnabled(true);
console.log('Exploration enabled:', getExploreEnabled());
// Exploration enabled: true

Setting Custom Weight Vectors

For fine-grained control, switch to the custom strategy and supply your own weight distribution. The setCustomWeights() function normalizes the vector before persisting it to routing_custom_weights.

import { setRoutingStrategy, setCustomWeights, getCustomWeights } from './server/src/services/router.js';

await setRoutingStrategy('custom');
await setCustomWeights({ reliability: 0.8, speed: 0.1, intelligence: 0.1 });

console.log('Custom weights:', getCustomWeights());
// Custom weights: { reliability: 0.8, speed: 0.1, intelligence: 0.1 }

Configuring Peak-Hours Adjustments

Operators can enable time-of-day re-weighting via routing_peak_hours_adjust. During configured windows (e.g., 18:00-22:00 UTC), the router shifts weight from speed to reliability using the peakAdjustedWeights function.

import { setPeakHoursConfig, getPeakHoursConfig } from './server/src/services/router.js';

await setPeakHoursConfig({
  enabled: true,
  startHour: 18,
  endHour: 22,
  timezone: 'UTC',
});

console.log('Peak-hours config:', getPeakHoursConfig());
// Peak-hours config: { enabled: true, startHour: 18, endHour: 22, timezone: 'UTC' }

Changing Key Selection Strategy

After selecting a model, the router determines which API key to use based on the key_selection_strategy setting. Options include 'auto' (per-key bandit) or 'least-remaining' (quota-aware selection).

import { setKeySelectionStrategy, getKeySelectionStrategy } from './server/src/services/router.js';

await setKeySelectionStrategy('least-remaining');
console.log('Key selection:', getKeySelectionStrategy());
// Key selection: least-remaining

Using the HTTP API

The /api/fallback/routing endpoint exposes the same configuration options via REST. Changes made through the dashboard or direct HTTP calls update the same SQLite settings used by the programmatic API.

# Set strategy to "smartest"

curl -X PUT -H "Authorization: Bearer <admin-token>" \
     -d '{"strategy":"smartest"}' \
     https://your-host/api/fallback/routing

# Enable exploration

curl -X PUT -H "Authorization: Bearer <admin-token>" \
     -d '{"exploreEnabled":true}' \
     https://your-host/api/fallback/routing

Note that the declarative configuration parser in declarative-config.ts (lines 474-485) also respects these same keys, allowing infrastructure-as-code deployments to specify routing preferences in JSON or YAML files.

Core Source Files and Architecture

Understanding the following files helps when debugging or extending routing behavior:

• server/src/services/router.ts: Core routing engine containing DEFAULT_STRATEGY, getRoutingStrategy(), setRoutingStrategy(), and the scoring logic that applies weights to the orderChain.
• server/src/services/scoring.ts: Defines the RoutingStrategy enum, weight presets for built-in strategies, and the peakAdjustedWeights calculation.
• server/src/services/declarative-config.ts: Parses configuration files and applies routing settings during startup (lines 474-485).
• server/src/routes/responses.ts: Implements the /api/fallback/routing HTTP handlers that bridge REST calls to the router service.

Summary

• FreeLLMAPI routing strategies determine model selection via weight vectors stored in the SQLite settings table.
• Six built-in strategies are available: priority, balanced, smartest, fastest, reliable, and custom.
• The default strategy is balanced, falling back to this value when no setting exists in the database.
• Use setRoutingStrategy() to switch presets, or configure custom weights via setCustomWeights().
• Enable exploration mode with setExploreEnabled() to give unmeasured models a 10% evaluation chance.
• Configure peak-hours adjustments to shift priority from speed to reliability during high-traffic windows.
• Change key selection via setKeySelectionStrategy() to choose between bandit-based or quota-aware API key usage.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

If no routing strategy exists in the database, FreeLLMAPI falls back to the balanced preset. This default is hardcoded as DEFAULT_STRATEGY on line 59 of server/src/services/router.ts, ensuring equal weighting across speed, reliability, and intelligence until you explicitly configure otherwise.

How do I create a completely custom routing strategy?

Set the strategy to custom using setRoutingStrategy('custom'), then define your weight vector with setCustomWeights(). The system accepts any convex combination of reliability, speed, and intelligence values, normalizing them before storage in the routing_custom_weights setting. The router applies these weights directly to the scoring calculation in server/src/services/router.ts.

Can I configure different routing behaviors for different times of day?

Yes. Enable the routing_peak_hours_adjust setting and configure the window via setPeakHoursConfig(). During the specified hours, the peakAdjustedWeights function in server/src/services/scoring.ts automatically shifts weight from the speed dimension to the reliability dimension, ensuring more stable performance during high-traffic periods.

What is the difference between 'auto' and 'least-remaining' key selection?

The auto strategy uses a per-key bandit algorithm that learns which API keys perform best for specific models. The least-remaining strategy selects the key with the most quota remaining, making it ideal for environments with hard rate limits. Configure this via setKeySelectionStrategy() in server/src/services/router.ts (lines 69-85).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}