# How the FreeLLMAPI Router Selects the Best Available LLM Model: Bandit Scoring and Fallback Chains

> Discover how the FreeLLMAPI router picks the best LLM model using bandit scoring and fallback chains for optimal performance and reliability. Maximize your API calls.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-07-01

---

**The FreeLLMAPI router selects the best available LLM model by computing a bandit-style convex score from reliability, speed, and intelligence signals, applying guard-rail multipliers for free-tier headroom and live rate-limit penalties, then choosing the highest-scoring model with a usable API key from an ordered fallback chain.**

The `tashfeenahmed/freellmapi` repository implements an intelligent request router that automatically selects the optimal large language model for each request. Understanding how the FreeLLMAPI router selects the best available LLM model requires examining its three-stage pipeline: chain resolution, bandit scoring, and key validation. This architecture makes the router both **data-driven** (learning from historical successes and failures) and **policy-driven** (allowing operators to tune selection via routing strategies).

## Building the Candidate Chain

The router first constructs an ordered list of candidate models through a fallback chain resolution process. This stage determines which models are eligible for scoring based on user configuration and global defaults.

### Profile-Aware Fallback Resolution

The router inspects the active user profile to determine the candidate pool. If a user has an active profile, the system pulls models linked to that profile via the `profile_models` association; otherwise, it falls back to the global `fallback_config` table. This logic resides in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) within the `getActiveChain` function (lines 26-55).

```typescript
function getActiveChain(db: Database): ChainRow[] { … }

```

The `resolveRoutingChain` function (lines 14-22) handles dynamic shortcuts and global sort aliases, ensuring the router adapts to both personalized and default routing configurations.

### Dynamic Global Sort Aliases

Special call patterns like `auto:fast` are translated into preset strategies (such as `fastest`) via the `GLOBAL_SORT_ALIASES` mapping. When triggered, the router builds a chain of all enabled models and orders them according to the alias-defined strategy. This allows users to request speed-optimized routing without explicitly naming a model.

## Bandit-Style Model Scoring

Once the candidate chain is resolved, the router scores each model using a **bandit algorithm** that balances exploitation of known high performers with exploration of newer options. The `orderChain` function (lines 76-95 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)) orchestrates this process, calling `scoreChainEntry` and `combineScore` to compute final rankings.

### Normalized Signal Axes

Each model is evaluated across three normalized axes, defined in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts):

- **Reliability**: Computed via a Beta posterior of success/failure counts using `reliabilityPosterior` (lines 31-40). For live routing, this uses Thompson sampling (`sampleBeta`) to inject exploration, while dashboard views use `expectedReliability` for stable averages. Returns a value in **[0, 1]**.

- **Speed**: The `speedScore` function blends throughput (`throughputScore`) and first-byte latency (`ttfbScore`), returning **[0, 1]** or the `SPEED_PRIOR` if no historical data exists (lines 1-7).

- **Intelligence**: The `intelligenceComposite` function calculates a tier-first composite score normalized by min/max values across the chain, returning **[0, 1]** (lines 12-15).

### Guard-Rail Multipliers

Before finalizing scores, the router applies two guard-rail multipliers to prevent over-utilization of constrained resources:

- **Headroom Factor**: The `headroomFactor` function (lines 24-30 of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts)) reduces the score as a model approaches its free-tier token budget, ensuring proportional routing away from quota-limited providers.

- **Rate-Limit Penalty**: Every HTTP 429 response increments a per-model penalty via `recordRateLimitHit` (lines 82-90 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)). The penalty decays over time and is converted into a multiplier by `rateLimitFactor` (lines 32-42 of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts)), temporarily suppressing models that are currently throttled.

### The Convex Combination Formula

The `combineScore` function (lines 85-92 of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts)) calculates the final score using a weighted convex combination of the three signal axes, then applies the guard-rail multipliers:

```typescript
export function combineScore(inputs: ScoreInputs, weights: RoutingWeights): number {
  const wSum = weights.reliability + weights.speed + weights.intelligence || 1;
  const base = (weights.reliability * inputs.reliability +
                weights.speed * inputs.speed +
                weights.intelligence * inputs.intelligence) / wSum;
  return base * inputs.headroom * inputs.rateLimit;
}

```

The `weights` parameter is determined by the active **routing strategy** (e.g., `balanced`, `fastest`, `reliable`), allowing operators to bias selection toward specific model characteristics.

## API Key Selection and Validation

After sorting the chain by score descending, the router iterates through the ordered list to find a usable model. For each candidate, `selectKeyForModel` (lines 53-66 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)) performs the following validation steps:

1. Verifies the provider exists via `hasProvider`.
2. Retrieves all enabled and healthy keys for the platform.
3. Validates per-key constraints: cooldown status, provider daily caps, RPM/RPD limits, TPM/TPD limits, and decryption success.
4. Rotates selection across valid keys using a `roundRobinIndex`.

If a key passes all gates, the router returns a `RouteResult` containing the provider instance, decrypted API key, and model identifiers. If no key is available for a given model, the router proceeds to the next entry in the scored chain. When the entire chain is exhausted, the router throws a `RouteError` with status **429**, containing diagnostics explaining why each model was rejected (lines 19-28).

## Practical Implementation Examples

### Retrieve the Ordered Model Chain

To inspect the current model ranking without making a request, use `getOrderedFusionChain`:

```typescript
import { getOrderedFusionChain } from './services/router';

const chain = getOrderedFusionChain();   // Returns FusionCandidate[]
console.log('Top-scoring models:', chain.slice(0, 3));

```

This returns an array of objects including `modelDbId`, `platform`, `modelId`, and `displayName`, already sorted by the active strategy.

### Switch Routing Strategies Programmatically

Force the router to prioritize speed over reliability by changing the strategy:

```typescript
import { setRoutingStrategy, getRoutingStrategy } from './services/router';

setRoutingStrategy('fastest');
console.log('Active strategy:', getRoutingStrategy()); // → 'fastest'

```

The next call to `routeRequest` will use the weight vector defined in `BANDIT_PRESETS` for the `fastest` configuration.

### Pin a Specific Model for Sticky Sessions

To ensure a conversation consistently uses a specific model (e.g., database ID 42), pass the model ID to `routeRequest`:

```typescript
import { routeRequest } from './services/router';

const result = routeRequest(1200, undefined, 42);
console.log(`Pinned ${result.modelId} on ${result.platform}`);

```

The router moves the pinned model to the front of the sorted chain while still executing the standard key-selection validation.

## Summary

- **Three-stage pipeline**: The router resolves the active chain from profile or global config, scores models using a bandit algorithm, and selects the first model with a valid API key.
- **Bandit scoring**: Combines reliability (Beta posterior), speed (throughput + latency), and intelligence (tier composite) into a weighted convex score, with Thompson sampling for exploration.
- **Guard-rails**: Applies `headroomFactor` for free-tier quotas and `rateLimitFactor` for live 429 penalties to prevent routing to constrained models.
- **Strategy-driven**: Uses configurable `RoutingWeights` (balanced, fastest, reliable) to bias selection without hardcoding model preferences.
- **Fallback mechanism**: Automatically proceeds down the scored chain if a model's keys are exhausted, cooldown-limited, or capped, ensuring high availability.

## Frequently Asked Questions

### What routing strategies does FreeLLMAPI support?

FreeLLMAPI supports multiple routing strategies defined in `BANDIT_PRESETS`, including **balanced** (default), **fastest**, and **reliable**. Each strategy defines a specific weight vector for the convex combination of reliability, speed, and intelligence signals. Operators can switch strategies dynamically via `setRoutingStrategy()`, and users can trigger preset aliases like `auto:fast` which map to these configurations.

### How does the router handle rate limits and 429 errors?

When a provider returns a 429 status, the router invokes `recordRateLimitHit` (lines 82-90 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)) to increment a per-model penalty. This penalty decays over time and is applied as a multiplicative factor via `rateLimitFactor` (lines 32-42 of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts)), temporarily reducing the model's selection score. The router also checks hard limits including RPM, RPD, TPM, and TPD via `canMakeRequest` in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) before allowing a key to be used.

### Can I force the router to use a specific model instead of automatic selection?

Yes. Pass the specific `modelDbId` as the third argument to `routeRequest()`. The router will move that model to the front of the sorted chain via the pinning logic, then execute the standard `selectKeyForModel` validation. If the pinned model has no usable keys, the router falls back to the next model in the chain unless configured otherwise.

### What happens when all models in the fallback chain are exhausted?

If `selectKeyForModel` fails to find a valid key for every model in the ordered chain, the `routeRequest` function throws a `RouteError` with HTTP status **429** (lines 19-28 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)). The error includes detailed diagnostics indicating why each model was rejected—such as cooldown status, decryption failure, or quota exhaustion—enabling operators to diagnose systemic provider issues.