FreeLLMAPI Routing Strategies: 6 Methods for Intelligent LLM Selection

FreeLLMAPI provides six distinct routing strategies—including priority, balanced, smartest, fastest, reliable, and custom—that dynamically select LLM models using weighted scoring algorithms based on reliability, speed, and intelligence metrics.

FreeLLMAPI is an open-source routing layer that automatically distributes requests across multiple Large Language Model (LLM) providers using configurable routing strategies. The system stores the active strategy in the routing_strategy runtime setting and applies sophisticated bandit algorithms or manual priority chains to determine which model serves each request. According to the source code in server/src/services/router.ts, you can switch strategies dynamically without restarting the server.

Available Routing Strategies in FreeLLMAPI

The routing system defines six valid strategies in the VALID_STRATEGIES constant within server/src/services/router.ts. Each strategy uses a specific weight vector—except for the legacy priority mode—to calculate model scores across three axes: reliability, speed, and intelligence.

Priority (Legacy Manual Fallback)

The priority strategy implements a legacy manual fallback chain defined by the fallback-config table. Models are attempted strictly in the order of their static priority column values, with a dynamic 429-penalty that pushes frequently rate-limited models down the list. This strategy does not use bandit scoring or weight vectors; it relies entirely on manual configuration and penalty adjustments.

Balanced (Default Bandit Preset)

The balanced strategy serves as the default when the server starts, defined by the DEFAULT_STRATEGY constant in server/src/services/scoring.ts. This preset applies a weight vector of { reliability: 0.5, speed: 0.25, intelligence: 0.25 }, giving reliability half the influence while splitting the remaining weight between speed and intelligence. This provides a well-rounded trade-off for general-purpose workloads.

Smartest (Intelligence-First)

The smartest strategy prioritizes model capability while maintaining reliability safeguards. It uses the weight vector { reliability: 0.35, speed: 0.1, intelligence: 0.55 }, making it ideal when you need the most capable model that remains dependable. Use this strategy for complex reasoning tasks where latency is less critical than output quality.

Fastest (Speed-Optimized)

The fastest strategy optimizes for throughput and low latency using weights { reliability: 0.35, speed: 0.55, intelligence: 0.1 }. This configuration keeps a safety margin with reliability while maximizing speed, making it suitable for high-throughput applications where response time matters more than model capability.

Reliable (Maximum Uptime)

The reliable strategy puts reliability front-and-center with weights { reliability: 0.7, speed: 0.15, intelligence: 0.15 }. This strategy selects models that "just work" even if they are slower or less capable, making it ideal for production environments where uptime is critical.

Custom (User-Defined Weights)

The custom strategy accepts a user-defined weight vector persisted in the routing_custom_weights setting. The system normalizes these weights to sum to 1 if they do not already, with the normalization occurring in setCustomWeights() within server/src/services/router.ts. This allows fine-grained control over the three scoring axes for specialized use cases.

How Routing Strategies Are Applied

The routing logic in server/src/services/router.ts implements a two-phase selection process. First, getRoutingStrategy() reads the current setting from storage and falls back to the default if the value is missing or invalid.

When processing a request, the orderChain() function receives the active list of enabled models and the current RoutingStrategy. If the strategy is priority, the function simply applies the 429-penalty to the static priority column and sorts models accordingly. For all other strategies (bandit modes), the function:

• Builds a weight vector from either a preset or the custom user-defined weights
• Computes per-model reliability, speed, and intelligence scores via scoring.ts
• Applies guard-rails including headroomFactor and rateLimitFactor
• Multiplies the weighted base using combineScore()
• Sorts models by the resulting composite score, using the static priority column only as a tie-breaker

Bandit strategies (all non-priority modes) perform Thompson sampling on reliability using sampleBeta() for live routing, introducing exploration while favoring higher-scoring models.

Configuring Routing Strategies and Custom Weights

You can programmatically adjust routing strategies and custom weights using the functions exported from server/src/services/router.ts. These functions are also exposed through the /api/fallback/routing HTTP API endpoints.

import {
  getRoutingStrategy,
  setRoutingStrategy,
  getCustomWeights,
  setCustomWeights,
} from './services/router.js';

// Read the current strategy
const current = getRoutingStrategy();   // e.g. 'balanced'

// Switch to the fastest preset
setRoutingStrategy('fastest');

// Verify the change
console.log('New strategy:', getRoutingStrategy());

// Define custom weights (60% reliability, 20% speed, 20% intelligence)
setCustomWeights({ reliability: 0.6, speed: 0.2, intelligence: 0.2 });

// Retrieve the stored custom vector (normalised)
console.log('Custom weights:', getCustomWeights());

// Apply the custom strategy at runtime
setRoutingStrategy('custom');

The custom weights are stored as JSON in the routing_custom_weights setting. While the UI normalizes weights to sum to 1 when saving, the setCustomWeights() function in server/src/services/router.ts performs backend normalization to guard against malformed input.

Key Source Files for Routing Implementation

File	Role
server/src/services/router.ts	Core routing orchestration, persistence of strategy and custom weights, ordering logic via orderChain() and getRoutingStrategy().
server/src/services/scoring.ts	Definitions of RoutingStrategy, BANDIT_PRESETS, default strategy constants, and scoring mathematics including Thompson sampling.
server/src/services/declarative-config.ts	Handles bulk configuration updates from the dashboard, including setRoutingStrategy and setCustomWeights.
server/src/routes/fallback.ts	HTTP API exposing GET/PUT endpoints for routing configuration consumed by the dashboard.
server/src/__tests__/services/router.test.ts	Unit tests verifying each strategy's behavior and weight calculations.

Summary

• FreeLLMAPI offers six routing strategies: priority, balanced, smartest, fastest, reliable, and custom.
• The balanced strategy is the default, weighting reliability at 50% and splitting speed and intelligence at 25% each.
• Priority mode uses manual fallback chains with 429-penalty adjustments, while all other strategies use bandit algorithms with Thompson sampling.
• Custom strategies allow user-defined weight vectors via setCustomWeights(), normalized to sum to 1.
• Strategy selection occurs in getRoutingStrategy() and model ordering happens in orderChain() within server/src/services/router.ts.
• All strategies except priority use composite scoring across reliability, speed, and intelligence axes with configurable guard-rails.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

The default routing strategy is balanced, defined by the DEFAULT_STRATEGY constant in server/src/services/scoring.ts. This preset applies a weight vector of { reliability: 0.5, speed: 0.25, intelligence: 0.25 } to provide a well-rounded trade-off between model dependability and performance. You can change this at runtime using setRoutingStrategy() or via the /api/fallback/routing API endpoint.

How does the priority routing strategy differ from bandit strategies?

The priority strategy implements a legacy manual fallback chain where models are tried strictly in the order defined by the fallback-config table, modified only by a dynamic 429-penalty that demotes rate-limited models. Unlike bandit strategies (balanced, smartest, fastest, reliable, and custom), priority mode does not use weighted scoring algorithms or Thompson sampling; it relies entirely on static configuration and penalty adjustments.

Can I use custom weight vectors for model selection?

Yes. The custom routing strategy allows you to define your own weight vector across the three axes: reliability, speed, and intelligence. Store your weights using setCustomWeights() in server/src/services/router.ts, which normalizes the values to sum to 1 if needed. After setting the weights, activate the custom strategy by calling setRoutingStrategy('custom'). The system persists these values in the routing_custom_weights setting.

How does FreeLLMAPI handle exploration vs exploitation when routing requests?

For all bandit strategies (non-priority modes), FreeLLMAPI implements Thompson sampling on the reliability metric using sampleBeta() in server/src/services/scoring.ts. This introduces probabilistic exploration that occasionally routes requests to less-certain models to gather performance data, while still favoring higher-scoring models for exploitation. The weight vectors (preset or custom) then combine these reliability samples with speed and intelligence scores to produce the final model ranking.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}