How the Fastest Strategy Works in FreeLLMAPI: Bandit Routing for Low-Latency LLM Calls

The Fastest strategy prioritizes speed by assigning 55% of the routing weight to throughput and latency metrics, while reserving 35% for reliability and 10% for intelligence to prevent selecting models that are fast but unstable.

The Fastest strategy is one of four built-in routing presets in FreeLLMAPI that determines how incoming LLM requests are distributed across available models using a multi-armed bandit algorithm. According to the FreeLLMAPI source code, this preset optimizes for raw performance by heavily weighting speed metrics while maintaining minimal guard-rails for reliability. This article examines the exact weight distribution in server/src/services/scoring.ts, the scoring mechanics that rank models, and the peak-hour exemption logic unique to this strategy.

Fastest Strategy Weight Distribution

The Bandit Preset Configuration

In server/src/services/scoring.ts, the Fastest strategy is defined as a weight vector within the BANDIT_PRESETS constant (lines 44–52). Unlike the balanced or smartest presets, Fastest allocates the majority of its weight to speed:

export const BANDIT_PRESETS: Record<Exclude<RoutingStrategy,
    'priority' | 'custom'>, RoutingWeights> = {
  balanced: { reliability: 0.5, speed: 0.25, intelligence: 0.25 },
  smartest: { reliability: 0.35, speed: 0.1,  intelligence: 0.55 },
  fastest:  { reliability: 0.35, speed: 0.55, intelligence: 0.1 },
  reliable: { reliability: 0.7,  speed: 0.15, intelligence: 0.15 },
};

Speed receives 0.55 (55%) of the total weight, making it the dominant factor in model selection. Reliability retains 0.35 (35%), ensuring that ultra-fast models with high failure rates do not receive traffic. Intelligence receives only 0.1 (10%), as capability ranking is deprioritized in favor of latency.

How the Scoring Algorithm Ranks Models

Base Score Calculation

When a request arrives, the router (server/src/services/router.ts) retrieves the weight vector for the active strategy and computes a base score using a linear combination of three normalized dimensions:

base = w_rel·reliability + w_speed·speed + w_intel·intelligence

• Reliability: Posterior probability of success using Beta sampling
• Speed: Composite metric of throughput and first-byte latency
• Intelligence: Tier-adjusted capability rank combining model size labels with compressed rankings

Guard-Rail Factors

Two multiplicative factors scale the base score to protect quota limits and handle errors:

1. Head-room factor: Reduces scores for models approaching their free quota limits
2. Rate-limit factor: Demotes models currently returning HTTP 429 errors

The final effective score is calculated as base × headroomFactor × rateLimitFactor (see lines 14–18 of scoring.ts). The model with the highest effective score wins the request.

Peak-Hour Exemption Behavior

FreeLLMAPI can optionally boost reliability weights during operator-declared peak windows to prioritize stability over speed. However, the Fastest strategy is explicitly excluded from this adjustment.

In server/src/services/scoring.ts (lines 16–17), the PEAK_EXEMPT_STRATEGIES array lists presets that ignore peak-hour re-weighting:

export const PEAK_EXEMPT_STRATEGIES: readonly RoutingStrategy[] = ['fastest', 'reliable'];

This exemption ensures that selecting fastest guarantees the raw speed-focused weight vector remains unchanged, regardless of external peak-hour configurations.

Configuring the Fastest Strategy

Via REST API

Switch to the Fastest strategy by calling the fallback routing endpoint defined in server/src/routes/fallback.ts (lines 41–45):

curl -X PUT https://your-server.com/api/fallback/routing \
  -H "Content-Type: application/json" \
  -d '{"strategy":"fastest"}'

Via Client Library

The client library provides a convenience helper in client/src/lib/routing.ts (lines 59–62):

import { setRoutingStrategy } from '@freellmapi/client';

await setRoutingStrategy('fastest');

Inspecting Active Weights

To verify the current weight distribution programmatically, import the presets map from the server package:

import { getCurrentStrategy, BANDIT_PRESETS } from '@freellmapi/server';

const strategy = getCurrentStrategy();               // e.g. 'fastest'
const weights = BANDIT_PRESETS[strategy];            // { reliability:0.35, speed:0.55, intelligence:0.1 }
console.log('Fastest weights →', weights);

Summary

• Weight allocation: The Fastest strategy assigns 55% weight to speed, 35% to reliability, and 10% to intelligence
• Scoring mechanism: Models are ranked by a linear combination of reliability, speed, and intelligence, scaled by head-room and rate-limit factors
• Peak exemption: The Fastest strategy ignores peak-hour reliability boosts, maintaining consistent speed-first routing
• Configuration: Activate via REST API at /api/fallback/routing or using the client library's setRoutingStrategy('fastest')
• Source location: Defined in server/src/services/scoring.ts with router logic in server/src/services/router.ts

Frequently Asked Questions

What weight does the Fastest strategy assign to speed?

The Fastest strategy allocates 0.55 (55%) of the total routing weight to speed metrics. This is the highest speed weight among all presets, significantly exceeding the 0.25 in balanced mode and 0.1 in smartest mode.

How does FreeLLMAPI prevent fast but unreliable models from being selected?

The algorithm reserves 35% weight for reliability, calculated as the posterior probability of success using Beta sampling. Additionally, a rate-limit factor demotes models returning 429 errors, and a head-room factor protects quota limits, ensuring raw speed does not override stability entirely.

Does the Fastest strategy change behavior during peak hours?

No. The Fastest strategy is listed in PEAK_EXEMPT_STRATEGIES alongside the reliable preset. This means it ignores peak-hour reliability adjustments and maintains its 55/35/10 weight distribution regardless of operator-declared peak windows.

Where is the Fastest strategy defined in the source code?

The weight vector is defined in server/src/services/scoring.ts at lines 44–52 within the BANDIT_PRESETS constant. The routing logic that applies these weights is implemented in server/src/services/router.ts, while the HTTP endpoint for switching strategies is located in server/src/routes/fallback.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}