FreeLLMAPI Routing Strategies: 6 Methods to Optimize AI Model Selection

FreeLLMAPI provides six distinct routing strategies—priority, balanced, smartest, fastest, reliable, and custom—that use weighted scoring algorithms to automatically select the optimal model-provider key based on speed, intelligence, and reliability criteria.

The open-source FreeLLMAPI project (tashfeenahmed/freellmapi) implements a pluggable routing layer that evaluates every incoming request against configurable weight vectors. These routing strategies determine how the engine balances trade-offs between latency, accuracy, and system uptime when selecting which provider key should handle a given prompt.

What Are Routing Strategies in FreeLLMAPI?

At its core, the router acts as a decision engine that scores each candidate model-provider key before issuing a request. The routing strategy defines the weight vector used during this scoring phase, effectively telling the system how much to value specific attributes like response speed, model intelligence, or historical reliability.

The strategy definitions and preset weight configurations reside in server/src/services/scoring.ts, while the actual scoring logic and strategy application occur in server/src/services/router.ts. When a request enters the system, the router computes a score for each available key using the active strategy’s weights, then selects the highest-scoring candidate that passes health and quota checks.

The Six Built-In Routing Strategies

FreeLLMAPI supports six routing strategies, each designed for specific operational requirements. These presets are defined in the BANDIT_PRESETS constant within the scoring module.

priority (Manual Order)

The priority strategy implements legacy behavior where the router follows the explicit chain order defined by the administrator without applying algorithmic weight adjustments.

• Weight Vector: null (no scoring adjustments applied)
• Implementation: See line 395 in server/src/services/router.ts where the router bypasses weight calculations when weights: null
• Use Case: Compliance environments, deterministic testing scenarios, or when administrative control must override algorithmic selection

balanced (Neutral Blend)

The balanced strategy provides a neutral configuration that values speed, reliability, and intelligence roughly equally.

• Weight Vector: BANDIT_PRESETS.balanced assigns approximately 0.33 to each dimension
• Use Case: General-purpose workloads where no single attribute should dominate the selection process

smartest (Intelligence-First)

The smartest strategy prioritizes model capability and reasoning quality over raw speed.

• Weight Vector: BANDIT_PRESETS.smartest assigns approximately 0.55 weight to intelligence and reduces speed priority to roughly 0.10
• Use Case: Creative writing, complex problem-solving, code generation, or research tasks where output quality outweighs latency concerns

fastest (Latency-First)

The fastest strategy optimizes for minimal response time, potentially sacrificing model sophistication.

• Weight Vector: BANDIT_PRESETS.fastest boosts speed to approximately 0.55 while reducing intelligence weight to roughly 0.10
• Use Case: Real-time chatbots, UI-responsive applications, or high-throughput scenarios where user experience depends on rapid responses

reliable (Uptime-First)

The reliable strategy favors providers and models with strong historical success rates and low error frequencies.

• Weight Vector: BANDIT_PRESETS.reliable assigns approximately 0.55 weight to reliability metrics
• Use Case: Mission-critical production workloads where request failures or timeouts carry significant business cost

custom (User-Defined Weights)

The custom strategy allows operators to supply their own weight vectors through persisted configuration records.

• Weight Vector: Reads from a RoutingWeights record stored via server/src/services/declarative-config.ts
• Use Case: Advanced tuning scenarios requiring bespoke balances (e.g., "fast but reliable" or "smart with headroom")

How to Configure Routing Strategies

FreeLLMAPI exposes strategy configuration through multiple interfaces, supporting both runtime adjustments and programmatic control.

Runtime Configuration via the MCP API

Operators can change the active strategy without restarting the server by calling the Model Control Plane (MCP) endpoint:

// POST /v1/mcp/routing/strategy
// Implementation: server/src/routes/mcp.ts
import fetch from 'node-fetch';

const resp = await fetch('https://api.freellmapi.com/v1/mcp/routing/strategy', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ strategy: 'smartest' }), // Valid: priority | balanced | smartest | fastest | reliable | custom
});

const data = await resp.json();
console.log('Strategy set to:', data.strategy);

This endpoint invokes setRoutingStrategy at line 404 in server/src/services/router.ts, immediately updating the routing behavior for subsequent requests.

Server-Side Programmatic Control

For internal server logic or initialization scripts, import the router service directly:

import { setRoutingStrategy, getRoutingStrategy } from '../services/router.js';

// Switch to lowest-latency routing
setRoutingStrategy('fastest');

// Verify current configuration
console.log('Current strategy:', getRoutingStrategy()); // -> 'fastest'

Client-Side Type Safety

The client library provides type definitions to ensure compile-time checking of strategy values:

import { setRoutingStrategy, type RoutingStrategy } from '@freellmapi/client';

// Type-safe strategy selection
const chosen: RoutingStrategy = 'balanced';
setRoutingStrategy(chosen);

Type definitions are exported from client/src/lib/routing.ts (line 59), ensuring consistency between server implementations and client applications.

Summary

• Six strategies available: priority, balanced, smartest, fastest, reliable, and custom provide deterministic manual ordering or algorithmic selection based on weighted scoring.
• Configuration locations: Preset weight vectors defined in server/src/services/scoring.ts, runtime logic in server/src/services/router.ts, and persistence handled by server/src/services/declarative-config.ts for custom weights.
• Runtime flexibility: Change strategies dynamically via the MCP API endpoint or programmatically using setRoutingStrategy without requiring server restarts.
• Type safety: Client-side code can leverage the RoutingStrategy type exported from client/src/lib/routing.ts to prevent invalid strategy selections.

Frequently Asked Questions

What is the default routing strategy in FreeLLMAPI?

The system typically initializes with the priority strategy, which follows the explicit manual ordering defined in the configuration chain. This legacy behavior ensures deterministic routing until an administrator explicitly switches to an algorithmic strategy like balanced or fastest via the API or configuration.

How do custom routing weights persist across restarts?

When using the custom strategy, FreeLLMAPI persists user-defined weight vectors through the declarative-config.ts service. The system stores these values as RoutingWeights records in the configured datastore, reloading them automatically during server initialization to maintain consistent routing behavior across deployments.

Can I change routing strategies without restarting the server?

Yes. FreeLLMAPI supports hot-swapping routing strategies at runtime through the /v1/mcp/routing/strategy endpoint or by calling setRoutingStrategy() programmatically. The change takes effect immediately for all subsequent requests without requiring process restarts or downtime.

Which strategy should I use for real-time chatbot applications?

For real-time chatbots requiring responsive user interfaces, deploy the fastest strategy. This configuration assigns approximately 0.55 weight to speed metrics (see BANDIT_PRESETS.fastest in scoring.ts), ensuring the router selects the lowest-latency available model even if it sacrifices some reasoning capability for response time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}