What Routing Strategies Does FreeLLMAPI Support?

FreeLLMAPI supports six configurable routing strategies—priority, balanced, smartest, fastest, reliable, and custom—that determine how LLM requests are ranked and routed based on weights for reliability, speed, and intelligence.

The FreeLLMAPI routing engine, implemented in server/src/services/router.ts, provides a flexible fallback chain system that automatically selects the optimal model for each request according to user-defined preferences. According to the tashfeenahmed/freellmapi source code, these strategies control whether the system prioritizes manual ordering, raw speed, model intelligence, uptime reliability, or a custom balance of factors.

Overview of Available Routing Strategies

The routing system defines its valid strategies in a constant array called VALID_STRATEGIES. As declared in server/src/services/router.ts, the supported strategies are:

• priority
• balanced
• smartest
• fastest
• reliable
• custom

Each strategy applies a different scoring algorithm to available models, creating an ordered chain that the request router traverses until it finds a healthy provider.

The Six Routing Strategies Explained

Priority Routing

The priority strategy uses legacy manual ordering. Models are ranked according to the priority order stored in the database, with an automatic penalty applied to models that recently returned HTTP 429 rate-limit errors. This strategy is selected when users explicitly choose "Priority" in the dashboard or when the stored configuration specifies it.

Balanced Routing

balanced serves as the DEFAULT_STRATEGY when no explicit strategy is set. This preset creates an equilibrium between reliability, speed, and intelligence. It is the fallback value returned by getRoutingStrategy() when the settings table contains a missing or invalid strategy name.

Smartest Routing

The smartest strategy emphasizes intelligence (model capability) while still accounting for reliability and speed metrics. When selected via the "Smartest" preset, the scoring algorithm assigns higher weight to model benchmarks and capability scores.

Fastest Routing

The fastest strategy prioritizes speed metrics, specifically tokens-per-second throughput and latency measurements, over other factors. This is ideal for applications where response time is critical.

Reliable Routing

The reliable strategy focuses exclusively on reliability, estimated by a Thompson-sampling bandit algorithm. This approach uses Bayesian updating to estimate success rates, selecting models with the highest probability of successful completion based on historical uptime data.

Custom Routing

The custom strategy allows users to define their own normalized weight vector across three dimensions: reliability, speed, and intelligence. These weights are stored in the routing_custom_weights setting and must sum to 1.0.

How Routing Strategy Selection Works

The active routing strategy persists in the database settings table. The system retrieves the current strategy via getRoutingStrategy():

// server/src/services/router.ts
const VALID_STRATEGIES: RoutingStrategy[] = ['priority','balanced','smartest','fastest','reliable','custom'];

If the stored value is missing or not included in VALID_STRATEGIES, the system automatically falls back to 'balanced'. This validation occurs at lines 54-58 of server/src/services/router.ts.

To programmatically change strategies, use setRoutingStrategy():

import { setRoutingStrategy } from '@/services/router';

setRoutingStrategy('fastest');  // Persists to DB and takes immediate effect

Configuring Custom Weights

When using the custom strategy, you define specific weights for each dimension via setCustomWeights(). The function automatically normalizes the input so the three values sum to 1:

import { setCustomWeights } from '@/services/router';

// Define custom weighting: 50% reliability, 30% speed, 20% intelligence
setCustomWeights({ 
  reliability: 0.5, 
  speed: 0.3, 
  intelligence: 0.2 
});

As implemented in server/src/services/router.ts (lines 89-102), the function validates the payload, calculates the sum, and stores the normalized JSON in the settings table under the routing_custom_weights key.

Routing a Request Programmatically

Internally, the routeRequest() function accepts routing parameters including vision and tool requirements:

import { routeRequest } from '@/services/router';

const result = routeRequest(
  /* estimatedTokens */ 1500,
  /* skipKeys */ undefined,
  /* preferredModelDbId */ undefined,
  /* requireVision */ true,   // Only vision-capable models considered
  /* requireTools */ false,
);

console.log('Chosen model:', result.modelId, 'via provider', result.provider.name);

Key Implementation Files

The routing strategy system spans several critical files in the codebase:

File	Purpose
server/src/services/router.ts	Contains VALID_STRATEGIES, DEFAULT_STRATEGY, getRoutingStrategy(), setCustomWeights(), and the core orderChain/routeRequest logic.
server/src/services/scoring.ts	Implements bandit-style scoring functions including reliabilityPosterior() and sampleBeta() that drive the non-priority strategies.
server/src/routes/fallback.ts	Exposes HTTP endpoints (GET /api/fallback/routing, PUT /api/fallback/routing) for UI and API access to strategy configuration.
server/src/db/index.ts	Provides getSetting() and setSetting() helpers for persisting strategy selections and custom weight vectors.

Summary

• FreeLLMAPI supports six routing strategies: priority, balanced, smartest, fastest, reliable, and custom.
• The default strategy is balanced, which is automatically used when no valid strategy is configured.
• The reliable strategy uses Thompson-sampling bandit algorithms to maximize uptime probability.
• Custom strategies allow normalized weighting across reliability, speed, and intelligence axes.
• Strategy persistence and retrieval are handled in server/src/services/router.ts with database backing via server/src/db/index.ts.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

The default routing strategy is balanced. According to the source code in server/src/services/router.ts, when getRoutingStrategy() encounters a missing or invalid strategy value in the settings table, it automatically returns 'balanced' as the fallback. This preset optimizes for a mix of reliability, speed, and intelligence rather than maximizing any single metric.

How does the reliable routing strategy work?

The reliable strategy employs a Thompson-sampling bandit algorithm defined in server/src/services/scoring.ts. It maintains Bayesian posteriors for each model's success rate and samples from these distributions to select the model with the highest estimated reliability. This approach naturally balances exploration of newly added models against exploitation of historically reliable providers.

Can I create my own routing strategy with custom weights?

Yes. The custom strategy allows you to define specific weights for reliability, speed, and intelligence using setCustomWeights(). The system normalizes your inputs so they sum to 1.0 and stores them in the routing_custom_weights setting. When active, the custom strategy applies these weights to score models dynamically for each request.

Where is the routing strategy stored and how is it validated?

The active strategy is stored in the database settings table and retrieved via the getSetting() helper in server/src/db/index.ts. The getRoutingStrategy() function validates the stored value against the VALID_STRATEGIES array in server/src/services/router.ts. If the stored value is not one of the six supported strategies, the system immediately falls back to the balanced default.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}