FreeLLMAPI Routing Strategies: How Requests Are Distributed Across Free-Tier Models

FreeLLMAPI provides five distinct routing strategies—priority, balanced, smartest, fastest, and custom—that determine how incoming requests are distributed across available free-tier language models based on priority levels, capacity, historical performance, latency metrics, or user-defined weights.

The tashfeenahmed/freellmapi repository implements an intelligent request router that automatically selects the optimal model endpoint for each API call. These FreeLLMAPI routing strategies control whether traffic flows to the highest-priority model, distributes evenly across all available capacity, or optimizes for speed and reliability. All strategies exclusively consider models with free-tier keys (identified by the :free suffix) and automatically bypass endpoints that have exhausted their quota limits.

Available Routing Strategies in FreeLLMAPI

The router service (server/src/services/router.ts) supports five primary strategies that can be set programmatically or via declarative configuration. Each strategy evaluates the pool of enabled free-tier models using different selection criteria.

Priority-Based Routing (priority)

The priority strategy ranks all enabled models according to their configured priority field, attempting the highest-priority endpoint first before falling back to lower-ranked alternatives. This approach ensures preferred models receive traffic until they reach capacity limits, making it ideal when you want specific providers to handle the majority of requests while maintaining automatic failover.

Balanced Distribution (balanced)

The balanced strategy distributes requests evenly across all enabled models, weighting each endpoint by its current headroom (remaining quota capacity). Rather than overwhelming a single high-priority model, this approach spreads load proportionally based on available resources, preventing any single endpoint from becoming a bottleneck during high-traffic periods.

Performance-Optimized Selection (smartest)

The smartest strategy selects the model with the historically highest success rate and lowest penalty score for the specific request pattern. By analyzing past performance data stored in the routing metrics, this strategy optimizes for reliability and response quality, automatically favoring endpoints that consistently return valid completions with minimal errors.

Latency-Driven Routing (fastest)

The fastest strategy routes requests to the model exhibiting the lowest observed latency (smallest latencyMs metric) during recent operations. This approach prioritizes response speed above other factors, making it suitable for real-time applications where time-to-first-token is the critical performance indicator.

Custom Weight Configuration (custom)

The custom strategy applies a user-defined weight vector specified through the declarative configuration interface or API settings. This allows fine-grained control over traffic distribution percentages (for example, allocating 70% to Model A and 30% to Model B) regardless of automatic metrics or priority rankings.

How to Configure Routing Strategies in FreeLLMAPI

Switch between routing strategies by calling setRoutingStrategy() from the router service. This function updates the global routing configuration and immediately affects subsequent request distributions.

import { setRoutingStrategy } from '@/services/router.js';

// Prioritize high-ranked free models with automatic fallback
setRoutingStrategy('priority');

// Distribute load evenly based on available quota headroom
setRoutingStrategy('balanced');

// Select models with the best historical success rates
setRoutingStrategy('smartest');

// Optimize for lowest response latency
setRoutingStrategy('fastest');

// Apply custom percentage weights via configuration
setRoutingStrategy('custom');

You can also define the strategy statically in the declarative configuration file (server/src/services/declarative-config.ts), where it persists across server restarts without requiring programmatic intervention.

Technical Implementation and Source Files

The routing logic is centralized in server/src/services/router.ts, which exports setRoutingStrategy() and implements the selection algorithms for each strategy. The RoutingStrategy type definition located in shared/types.ts enumerates the allowed string literals ('priority', 'balanced', 'smartest', 'fastest', 'custom') and provides TypeScript constraints for the configuration interface.

For testing and validation, the repository includes server/src/scripts/routing-sim.ts, which simulates request loads against different strategies to measure their impact on distribution patterns and latency. The high-level architectural overview in docs/architecture.md describes how these strategies interact with the free-tier filtering mechanism to ensure only valid :free endpoints are considered during selection.

Summary

• Five distinct strategies control request distribution: priority, balanced, smartest, fastest, and custom.
• All strategies are free-tier aware, automatically filtering for models with :free suffixes and skipping exhausted quotas.
• Configuration occurs via setRoutingStrategy() in server/src/services/router.ts or through the declarative config interface.
• The RoutingStrategy type in shared/types.ts provides compile-time safety for strategy selection.
• The balanced strategy weights by headroom, while smartest and fastest optimize for historical performance and latency respectively.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

The repository typically initializes with the priority strategy as the default, ensuring that higher-ranked free-tier models receive traffic first before falling back to alternatives. You can verify the current default by checking the initialization logic in server/src/services/router.ts or by inspecting the default configuration values in server/src/services/declarative-config.ts.

How does FreeLLMAPI handle quota exhaustion when using routing strategies?

The router automatically excludes models that have exhausted their free-tier quota from the selection pool, regardless of which strategy is active. When a model reaches its limit, the system flags it as unavailable and redistributes requests to the next eligible endpoint according to the active strategy's ranking or weighting algorithm.

Can I combine multiple routing strategies in FreeLLMAPI?

No, the router accepts only a single active strategy per configuration context. However, you can simulate hybrid behavior using the custom strategy, which allows you to define specific weight vectors that effectively blend priority and load-balancing concepts. Switch strategies dynamically at runtime by calling setRoutingStrategy() with a new value.

Where is the RoutingStrategy type defined in the codebase?

The RoutingStrategy union type is defined in shared/types.ts, which centralizes type definitions used across both the server and client packages. This file exports the type as a string literal union ('priority' | 'balanced' | 'smartest' | 'fastest' | 'custom') that constrains the parameter accepted by setRoutingStrategy() in the router implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}