What Is the Default Routing Strategy in FreeLLMAPI?

FreeLLMAPI uses an analytics-driven "balanced" routing strategy by default, which automatically distributes AI requests across the available model pool based on real-time performance metrics unless explicitly overridden via configuration.

FreeLLMAPI is an open-source inference gateway designed to route large language model (LLM) requests efficiently across multiple providers. Understanding the default routing strategy in FreeLLMAPI is essential for operators who want to optimize latency, cost, and reliability without manual configuration.

Understanding the Balanced Routing Strategy

The balanced strategy represents an analytics-driven approach to request distribution. When the service initializes, the routing system continuously evaluates model performance metrics to distribute incoming requests optimally across the available pool. This approach ensures no single provider becomes a bottleneck while maintaining consistent response quality.

According to the source code in server/src/services/scoring.ts, analytics-driven routing is explicitly enabled by default with the 'balanced' mode selected at startup.

Where the Default Is Defined in the Source Code

The default routing behavior is established across three key files in the codebase, each handling different aspects of strategy management.

The Scoring Service Configuration

The authoritative definition of the default strategy resides in server/src/services/scoring.ts. This file contains the core comment that establishes the baseline behavior: analytics-driven routing is on by default ('balanced'). The scoring logic implemented here provides the performance metrics that drive routing decisions when operating in balanced mode.

Router Implementation

The server/src/services/router.ts file implements the actual selection logic that consumes the active strategy. When the system initializes, this service queries the current routing configuration and applies the balanced distribution algorithm unless an alternative strategy has been specified. The router dynamically adjusts request distribution based on real-time scoring data provided by the scoring service.

Configuration Override Mechanism

Operator overrides are handled in server/src/services/declarative-config.ts. This configuration parser checks for explicit strategy declarations in the deployment settings, defaulting to 'balanced' when no alternative is specified. The declarative configuration system allows seamless switching between routing modes without requiring code changes.

How to Check and Override the Default Strategy

Developers and operators can programmatically inspect and modify the routing strategy at runtime using the exposed service methods.

To check the current active strategy:

// Query the currently active routing strategy
import { getRoutingStrategy } from '@/services/router';

const currentStrategy = getRoutingStrategy();
console.log(`Active routing strategy: ${currentStrategy}`); 
// Output: 'balanced' (unless explicitly changed)

To override the default strategy:

// Switch to an alternative routing strategy
import { setRoutingStrategy } from '@/services/router';

// Change from 'balanced' to 'explore' mode
setRoutingStrategy('explore');

Summary

• FreeLLMAPI initializes with 'balanced' as the default routing strategy, as explicitly defined in server/src/services/scoring.ts.
• The balanced mode operates as an analytics-driven system that distributes requests based on real-time performance metrics rather than static rules.
• Strategy selection logic is implemented in server/src/services/router.ts, while configuration overrides are processed through server/src/services/declarative-config.ts.
• Operators can inspect the current strategy using getRoutingStrategy() and switch modes programmatically via setRoutingStrategy().

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

FreeLLMAPI uses the 'balanced' routing strategy by default. According to the source code in server/src/services/scoring.ts, analytics-driven routing is enabled automatically when the service starts unless an operator explicitly configures an alternative mode.

How can I change the routing strategy from balanced to something else?

You can override the default by calling the setRoutingStrategy() function imported from @/services/router, passing a string such as 'explore' as the argument. Alternatively, you can specify a different strategy in the declarative configuration file processed by server/src/services/declarative-config.ts.

Where is the default routing strategy configured in the codebase?

The default is primarily defined in server/src/services/scoring.ts through a comment that establishes analytics-driven routing with 'balanced' mode. The actual enforcement occurs in server/src/services/router.ts, while server/src/services/declarative-config.ts handles configuration-based overrides.

Does the balanced strategy affect performance?

Yes, the balanced strategy optimizes performance by distributing requests across multiple models based on real-time scoring metrics. This prevents overload on any single provider and typically improves overall latency and throughput compared to static routing approaches.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}