# How the FreeLLMAPI Router Selects Providers for LLM Requests

> Discover how the FreeLLMAPI router intelligently selects LLM providers. Learn about its deterministic pipeline filtering, multi-axis scoring, and fallback chain for optimal performance and resilience.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-30

---

**The FreeLLMAPI router implements a deterministic, analytics-driven pipeline that filters providers by quota and rate limits, scores remaining candidates using multi-axis metrics including latency and success rates, and automatically selects the highest-ranked provider while maintaining a fallback chain for resilience.**

FreeLLMAPI is an open-source API aggregation layer that intelligently routes requests to multiple large language model (LLM) providers. Understanding exactly how the FreeLLMAPI router selects providers for requests enables developers to optimize their integration and predict routing behavior. The system implements a sophisticated scoring engine that weighs real-time performance data against static configuration rules to minimize costs and maximize reliability.

## The Seven-Stage Provider Selection Pipeline

The routing decision follows a strict pipeline defined in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts). Each stage eliminates unsuitable candidates or adjusts their priority scores before the final selection.

### Model-Group Resolution and Fallback Chains

When a request arrives with `model: "auto"` or a specific group identifier, the router first expands the requested model identifier into candidate providers. According to comments around line 724 in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts), the system walks a **fallback chain** defined in the UI configuration, creating an ordered list of potential handlers. If the caller specifies an explicit provider such as `openrouter/gpt-4o-mini`, the router treats that provider as the sole primary candidate but still maintains the fallback chain for error recovery.

### Provider-Quota Filtering

For each candidate model, the router checks the provider-quota table to verify whether the current API key possesses the necessary permissions. The mapping of platforms to quota keys resides in [`server/src/services/provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/provider-quota.ts) (lines 122-165). This step enforces **free-only** or **paid-only** restrictions, immediately removing providers that the key cannot access.

### Rate-Limit and Penalty Inspection

The router consults the **ratelimit** and **penalty-inspector** services to identify blocked or throttled providers. If a key has exceeded a provider's Requests Per Day (RPD) limit, the system benchmarks that provider for 24 hours and skips it during selection. The rate-limit logic in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (line 185) tracks these limits, while [`server/src/services/penalty-inspector.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/penalty-inspector.ts) retrieves active penalties via the `getAllPenalties` function to downgrade providers with recent failures.

### Multi-Axis Scoring System

Each remaining candidate receives a **score vector** calculated in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts). The system evaluates multiple axes:

- **Latency**: Historical response times
- **Success-rate**: Percentage of successful requests
- **Token-budget**: Cost efficiency metrics
- **Custom weights**: Runtime overrides set via administrative endpoints

The scoring engine supports multiple **routing strategies**, including `tier-first` (prioritizing provider tiers) and `rank-as-tiebreaker` (using rank to resolve score ties), as documented in the top comments of [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts).

### Custom Weights and Runtime Strategy

Administrators can modify router behavior at runtime through the **declarative configuration** API. The `setCustomWeights` and `setRoutingStrategy` functions in [`server/src/services/declarative-config.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/declarative-config.ts) mutate in-memory weighting tables that the scorer reads immediately. This allows dynamic adjustment of provider priorities without restarting the service.

### Final Selection and Error Handling

After scoring, the router selects the highest-ranked candidate. If the chosen provider returns a 429 error or other hard failure, the router transparently retries the request with the next-best candidate. The fallback walk logic appears around line 735 in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts), ensuring that requests eventually succeed even if primary providers fail.

### Response Metric Recording

The HTTP entry point in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) wraps the selected provider's response. It imports `routeRequest` to execute the routing decision and `recordSuccess` to feed performance data back into the scoring system, creating a self-healing feedback loop.

## Practical Implementation Examples

### Automatic Provider Selection

Trigger the auto-router to automatically select the best provider based on your API key's quotas and current performance metrics:

```typescript
import axios from 'axios';

const response = await axios.post(
  'https://api.freellmapi.com/v1/chat/completions',
  {
    model: 'auto',
    messages: [{ role: 'user', content: 'Explain quantum tunneling.' }]
  },
  { headers: { Authorization: `Bearer ${YOUR_API_KEY}` } }
);

```

### Explicit Provider Selection with Fallback

Force a specific provider while retaining the router's fallback capabilities:

```typescript
const response = await axios.post(
  'https://api.freellmapi.com/v1/chat/completions',
  {
    model: 'openrouter/gpt-4o-mini',
    messages: [{ role: 'user', content: 'Generate a poem.' }]
  },
  { headers: { Authorization: `Bearer ${YOUR_API_KEY}` } }
);

```

### Administrative Routing Configuration

Adjust the routing strategy at runtime using administrative privileges:

```typescript
await axios.post(
  'https://api.freellmapi.com/v1/admin/routing/strategy',
  { strategy: 'rank' },
  { headers: { Authorization: `Bearer ${ADMIN_KEY}` } }
);

```

## Summary

- The FreeLLMAPI router uses a **deterministic seven-stage pipeline** defined in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) to select providers.
- **Quota filtering** in [`provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/provider-quota.ts) and **rate-limit checks** in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) eliminate ineligible candidates before scoring.
- The **scoring engine** in [`scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/scoring.ts) ranks providers using latency, success rates, and custom weights according to configurable strategies.
- **Automatic fallback** occurs when the primary provider returns errors, with the retry logic implemented around line 735 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts).
- Runtime configuration via [`declarative-config.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/declarative-config.ts) allows dynamic adjustment of weights and strategies without service interruption.

## Frequently Asked Questions

### What happens when I set the model to "auto" in FreeLLMAPI?

When you specify `model: "auto"`, the router initiates the **fallback chain walk** described around line 724 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts). It expands the automatic selection into all available providers permitted by your API key's quota, then scores and ranks them to determine the optimal handler for your specific request.

### How does FreeLLMAPI prevent selecting providers that have hit rate limits?

The router consults [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (specifically line 185) to check RPD limits before scoring. If your key has exhausted a provider's quota, the system **benchmarks that provider for 24 hours**, effectively removing it from the candidate pool until the limit resets.

### Can I customize which providers FreeLLMAPI prioritizes for my account?

Yes, administrators can adjust routing behavior by calling the declarative configuration endpoints. The `setCustomWeights` function in [`server/src/services/declarative-config.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/declarative-config.ts) allows you to apply custom weight multipliers to specific providers, while `setRoutingStrategy` switches between `tier-first` and `rank-as-tiebreaker` modes.

### What occurs if the selected provider returns a 429 error?

The router implements transparent fallback logic. If the primary provider returns a 429 or other hard error, the system automatically retries the request with the next highest-scoring candidate from the fallback chain. This logic is implemented around line 735 in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), ensuring high availability without client-side intervention.