Scoring Algorithms Used by FreeLLMAPI for Model Selection: A Multi-Dimensional Convex System

FreeLLMAPI employs a multi-dimensional convex scoring system that collapses Bayesian reliability sampling, throughput-based speed metrics, normalized intelligence rankings, and multiplicative guardrails into a single interpretable score for routing decisions.

FreeLLMAPI is an open-source routing layer that dynamically selects the optimal large language model for each request. According to the tashfeenahmed/freellmapi source code, the platform implements a bandit-driven scoring pipeline in server/src/services/scoring.ts that evaluates models across five distinct axes to determine routing priority.

Reliability Scoring with Beta Posterior Thompson Sampling

The reliability algorithm treats each model as a Bernoulli bandit with a Beta(1, 1) uniform prior.

The reliabilityPosterior() function updates this prior with observed successes and failures, returning parameters α = 1 + successes and β = 1 + failures. Expected reliability—displayed in the dashboard via expectedReliability()—is calculated as the posterior mean α / (α + β).

For actual routing decisions, FreeLLMAPI implements Thompson sampling by drawing a random sample from the posterior distribution via sampleBeta(). This stochastic approach naturally balances exploration of uncertain models against exploitation of proven high-performers without requiring explicit epsilon-greedy parameters.

Speed Scoring: Throughput and Time-to-First-Byte

Speed evaluation combines two latency signals with weighted aggregation.

Throughput scoring utilizes exponential saturation: 1 - exp(-tokPerSec / SPEED_SCALE_TOK_S). This formula prevents extremely fast models from dominating the linear scale while maintaining sensitivity in the typical performance range.

TTFB scoring applies a linear ramp between TTFB_BEST_MS (full credit) and TTFB_WORST_MS (zero credit), rewarding models with lower time-to-first-byte latency.

The speedScore() function blends these components using weights 0.6 for throughput (THROUGHPUT_WEIGHT) and 0.4 for TTFB (TTFB_WEIGHT). If either metric is unavailable, the system falls back to the available metric or returns SPEED_PRIOR = 0.6 to ensure continuous routing capability.

Intelligence Score Normalization

Raw intelligence rankings—derived from tiered model capabilities—are normalized across the enabled chain to prevent tier domination.

The intelligenceScore() function applies min-max normalization: (composite - min) / (max - min), scaling values to the [0, 1] range. If all models share identical intelligence composites, the function returns 1 to maintain neutral-high scores without penalizing uniform quality.

Multiplicative Guardrail Factors

Two safety factors protect against quota exhaustion and rate limiting through multiplicative dampening.

Headroom factor: The headroomFactor() function linearly reduces scores from 1.0 down to HEADROOM_FLOOR = 0.1 as remaining quota falls below HEADROOM_RAMP_START = 0.2 (20% of budget). This protects models nearing their free-tier limits while maintaining minimal routing probability.

Rate-limit factor: The rateLimitFactor() penalizes models returning 429 errors, scaling to a minimum of RATE_LIMIT_MAX_DAMP = 0.6. Even fully throttled models retain at least 40% of their base score, preventing complete blackouts during temporary provider restrictions.

Convex Score Combination and Weighted Presets

The final score merges five axes through a convex combination that preserves interpretability.

First, combineScore() creates a base score using normalized weights for reliability, speed, and intelligence that sum to 1.0. Then it applies the multiplicative guardrails:

finalScore = baseScore × headroomFactor × rateLimitFactor

FreeLLMAPI provides four built-in bandit presets in BANDIT_PRESETS: balanced, smartest, fastest, and reliable. Each preset encodes distinct convex weight combinations optimizing for different use cases. Users may also define custom weight vectors via the API to override these defaults.

Routing Implementation

The scoreChainEntry() function in server/src/services/router.ts orchestrates the complete pipeline. For each model in the routing chain, it collects raw metrics, computes the five axis scores, applies guardrails, and invokes combineScore() with the active weight preset. Models are ordered by descending final score for request routing, with deterministic expected scores exposed via getRoutingScores() for dashboard visualization.

Code Example: Computing Model Scores

import {
  reliabilityPosterior,
  expectedReliability,
  sampleBeta,
  speedScore,
  intelligenceScore,
  headroomFactor,
  rateLimitFactor,
  combineScore,
  BANDIT_PRESETS,
} from '@/server/src/services/scoring';

// Raw metrics from runtime monitoring
const successes = 120;
const failures = 30;
const tokPerSec = 85;
const ttfbMs = 750;
const intelligenceComposite = 0.42;
const intelMin = 0.30;
const intelMax = 0.60;
const usedTokens = 3_000_000;
const budgetTokens = 5_000_000;
const rateLimitPenalty = 2;

// 1. Reliability: Thompson sample for routing, expected for UI
const { alpha, beta } = reliabilityPosterior(successes, failures);
const sampledReliability = sampleBeta(alpha, beta);
const displayReliability = expectedReliability(successes, failures);

// 2. Speed: Throughput + TTFB blend
const spd = speedScore(tokPerSec, ttfbMs);

// 3. Intelligence: Min-max normalized
const intel = intelligenceScore(intelligenceComposite, intelMin, intelMax);

// 4. Guardrails
const headroom = headroomFactor(usedTokens, budgetTokens);
const rateLimit = rateLimitFactor(rateLimitPenalty);

// 5. Final convex combination using "balanced" preset
const finalScore = combineScore(
  { reliability: sampledReliability, speed: spd, intelligence: intel, headroom, rateLimit },
  BANDIT_PRESETS.balanced
);

console.log('Routing score:', finalScore);

Summary

• FreeLLMAPI implements a convex scoring system in server/src/services/scoring.ts that balances reliability, speed, and intelligence through mathematically rigorous functions.
• Reliability uses Beta posterior Thompson sampling with sampleBeta() for exploration and expectedReliability() for dashboard display.
• Speed combines exponentially saturated throughput and linear TTFB scoring with weights 0.6 and 0.4, falling back to SPEED_PRIOR = 0.6 when data is missing.
• Guardrails apply multiplicative dampening via headroomFactor() (floor 0.1) and rateLimitFactor() (floor 0.6) to prevent quota exhaustion.
• Four presets (balanced, smartest, fastest, reliable) encode common trade-offs, with custom weight vector support via the API.
• The router in server/src/services/router.ts executes scoreChainEntry() to rank models by combined score for each request.

Frequently Asked Questions

How does FreeLLMAPI balance exploration versus exploitation in model selection?

FreeLLMAPI implements Thompson sampling via the sampleBeta() function in server/src/services/scoring.ts. Instead of using the expected reliability value for routing decisions, it draws a random sample from the Beta(α, β) posterior distribution. This Bayesian approach naturally explores uncertain models—those with fewer observations—while exploiting known high-performers, eliminating the need for manual epsilon-greedy tuning.

What happens when a model approaches its free-tier quota limit?

The headroomFactor() function applies a linear penalty when remaining quota drops below 20% (HEADROOM_RAMP_START = 0.2). The score multiplier decreases from 1.0 down to a floor of 0.1 (HEADROOM_FLOOR) as the model nears exhaustion. This guardrail ensures near-limit models are deprioritized while retaining minimal routing probability for emergency fallback scenarios.

Can users customize the scoring weights for different use cases?

Yes. FreeLLMAPI exposes four built-in presets in BANDIT_PRESETS—balanced, smartest, fastest, and reliable—each encoding different convex weight combinations. Users can also pass custom weight vectors via the API to combineScore(), allowing precise control over the trade-off between reliability, throughput, and intelligence rankings.

How does the system handle missing speed metrics for a model?

The speedScore() function implements graceful degradation. If throughput is unavailable, it falls back to the TTFB score alone, and vice versa. If neither metric exists, the function returns SPEED_PRIOR = 0.6, ensuring new or unobserved models receive a neutral baseline score rather than failing the routing decision.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}