# How the Fastest Strategy Works in FreeLLMAPI: Bandit Routing for Low-Latency LLM Calls

> Discover how FreeLLMAPI's Fastest strategy achieves low-latency LLM calls using bandit routing. Learn how it prioritizes speed with 55% weight on throughput and latency.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: performance
- Published: 2026-08-26

---

**The Fastest strategy prioritizes speed by assigning 55% of the routing weight to throughput and latency metrics, while reserving 35% for reliability and 10% for intelligence to prevent selecting models that are fast but unstable.**

The Fastest strategy is one of four built-in routing presets in FreeLLMAPI that determines how incoming LLM requests are distributed across available models using a multi-armed bandit algorithm. According to the FreeLLMAPI source code, this preset optimizes for raw performance by heavily weighting speed metrics while maintaining minimal guard-rails for reliability. This article examines the exact weight distribution in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts), the scoring mechanics that rank models, and the peak-hour exemption logic unique to this strategy.

## Fastest Strategy Weight Distribution

### The Bandit Preset Configuration

In [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts), the Fastest strategy is defined as a **weight vector** within the `BANDIT_PRESETS` constant (lines 44–52). Unlike the balanced or smartest presets, Fastest allocates the majority of its weight to speed:

```typescript
export const BANDIT_PRESETS: Record<Exclude<RoutingStrategy,
    'priority' | 'custom'>, RoutingWeights> = {
  balanced: { reliability: 0.5, speed: 0.25, intelligence: 0.25 },
  smartest: { reliability: 0.35, speed: 0.1,  intelligence: 0.55 },
  fastest:  { reliability: 0.35, speed: 0.55, intelligence: 0.1 },
  reliable: { reliability: 0.7,  speed: 0.15, intelligence: 0.15 },
};

```

**Speed receives 0.55 (55%)** of the total weight, making it the dominant factor in model selection. **Reliability retains 0.35 (35%)**, ensuring that ultra-fast models with high failure rates do not receive traffic. **Intelligence receives only 0.1 (10%)**, as capability ranking is deprioritized in favor of latency.

## How the Scoring Algorithm Ranks Models

### Base Score Calculation

When a request arrives, the router ([`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)) retrieves the weight vector for the active strategy and computes a **base score** using a linear combination of three normalized dimensions:

```typescript
base = w_rel·reliability + w_speed·speed + w_intel·intelligence

```

- **Reliability**: Posterior probability of success using Beta sampling
- **Speed**: Composite metric of throughput and first-byte latency
- **Intelligence**: Tier-adjusted capability rank combining model size labels with compressed rankings

### Guard-Rail Factors

Two multiplicative factors scale the base score to protect quota limits and handle errors:

1. **Head-room factor**: Reduces scores for models approaching their free quota limits
2. **Rate-limit factor**: Demotes models currently returning HTTP 429 errors

The final **effective score** is calculated as `base × headroomFactor × rateLimitFactor` (see lines 14–18 of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts)). The model with the highest effective score wins the request.

## Peak-Hour Exemption Behavior

FreeLLMAPI can optionally boost reliability weights during operator-declared peak windows to prioritize stability over speed. However, the Fastest strategy is explicitly excluded from this adjustment.

In [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts) (lines 16–17), the `PEAK_EXEMPT_STRATEGIES` array lists presets that ignore peak-hour re-weighting:

```typescript
export const PEAK_EXEMPT_STRATEGIES: readonly RoutingStrategy[] = ['fastest', 'reliable'];

```

This exemption ensures that selecting `fastest` guarantees the raw speed-focused weight vector remains unchanged, regardless of external peak-hour configurations.

## Configuring the Fastest Strategy

### Via REST API

Switch to the Fastest strategy by calling the fallback routing endpoint defined in [`server/src/routes/fallback.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/fallback.ts) (lines 41–45):

```bash
curl -X PUT https://your-server.com/api/fallback/routing \
  -H "Content-Type: application/json" \
  -d '{"strategy":"fastest"}'

```

### Via Client Library

The client library provides a convenience helper in [`client/src/lib/routing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/client/src/lib/routing.ts) (lines 59–62):

```typescript
import { setRoutingStrategy } from '@freellmapi/client';

await setRoutingStrategy('fastest');

```

### Inspecting Active Weights

To verify the current weight distribution programmatically, import the presets map from the server package:

```typescript
import { getCurrentStrategy, BANDIT_PRESETS } from '@freellmapi/server';

const strategy = getCurrentStrategy();               // e.g. 'fastest'
const weights = BANDIT_PRESETS[strategy];            // { reliability:0.35, speed:0.55, intelligence:0.1 }
console.log('Fastest weights →', weights);

```

## Summary

- **Weight allocation**: The Fastest strategy assigns 55% weight to speed, 35% to reliability, and 10% to intelligence
- **Scoring mechanism**: Models are ranked by a linear combination of reliability, speed, and intelligence, scaled by head-room and rate-limit factors
- **Peak exemption**: The Fastest strategy ignores peak-hour reliability boosts, maintaining consistent speed-first routing
- **Configuration**: Activate via REST API at `/api/fallback/routing` or using the client library's `setRoutingStrategy('fastest')`
- **Source location**: Defined in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts) with router logic in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)

## Frequently Asked Questions

### What weight does the Fastest strategy assign to speed?

The Fastest strategy allocates **0.55 (55%)** of the total routing weight to speed metrics. This is the highest speed weight among all presets, significantly exceeding the 0.25 in balanced mode and 0.1 in smartest mode.

### How does FreeLLMAPI prevent fast but unreliable models from being selected?

The algorithm reserves **35% weight for reliability**, calculated as the posterior probability of success using Beta sampling. Additionally, a **rate-limit factor** demotes models returning 429 errors, and a **head-room factor** protects quota limits, ensuring raw speed does not override stability entirely.

### Does the Fastest strategy change behavior during peak hours?

**No**. The Fastest strategy is listed in `PEAK_EXEMPT_STRATEGIES` alongside the reliable preset. This means it ignores peak-hour reliability adjustments and maintains its 55/35/10 weight distribution regardless of operator-declared peak windows.

### Where is the Fastest strategy defined in the source code?

The weight vector is defined in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts) at lines 44–52 within the `BANDIT_PRESETS` constant. The routing logic that applies these weights is implemented in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), while the HTTP endpoint for switching strategies is located in [`server/src/routes/fallback.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/fallback.ts).