What Is the FreeLLMAPI Router Service? Core Responsibilities and Architecture

The FreeLLMAPI Router Service acts as the intelligent traffic controller of the platform, automatically selecting optimal language models, enforcing provider quotas, managing fallback chains, and exposing a unified OpenAI-compatible API endpoint.

The FreeLLMAPI Router Service is the central nervous system of the tashfeenahmed/freellmapi repository. It abstracts away the complexity of managing multiple LLM providers—including OpenRouter, HuggingFace, and Groq—by dynamically routing requests to the best available model while respecting rate limits and quota constraints. This service ensures that client applications receive consistent, high-performance responses without needing to manage provider-specific integrations.

Core Responsibilities of the FreeLLMAPI Router Service

Intelligent Model Selection and Routing

When a client sends a request with model: "auto" or omits the model field entirely, the router evaluates all enabled providers using a sophisticated scoring algorithm. The selection logic resides in server/src/services/router.ts, specifically within the routeRequest and resolveRoutingChain functions. These functions consult server/src/services/scoring.ts to weight factors such as latency, cost, and custom provider strategies before selecting the optimal model.

Quota Enforcement and Rate Limiting

Before finalizing a routing decision, the service checks provider-specific quotas stored in the SQLite database via server/src/services/provider-quota.ts. Additionally, global rate-limit counters managed in server/src/services/ratelimit.ts ensure that API keys do not exceed their daily request budgets. If a provider has exhausted its quota, the router automatically excludes it from the candidate pool and selects the next best alternative.

Fallback Profiles and Quirk Handling

Administrators can define fallback profiles that map failing models to alternatives, implemented in server/src/routes/fallback.ts. The router also consults server/src/services/quirks.ts to handle model-specific behaviors such as "force-vision" or "disable-tools" flags. These quirks are applied during the routing phase to ensure compatibility with specific model capabilities.

Analytics and Request Logging

Every routing decision is recorded for performance analysis. The recordSuccess and recordRateLimitHit functions in server/src/services/router.ts log outcomes to the analytics dashboard, informing future scoring decisions and providing visibility into provider reliability.

How Requests Flow Through the Router Service

Incoming client requests hit the Express route defined in server/src/routes/proxy.ts, which handles all /v1/* endpoints. This proxy forwards requests to the router service, translates the internal RouteResult into a standard OpenAI-compatible JSON response, and returns it to the client. This architecture provides a single, stable API surface that masks the underlying complexity of multiple provider integrations.

Key Implementation Files in the FreeLLMAPI Router Service

Understanding the router's architecture requires familiarity with these specific files:

• server/src/services/router.ts – Contains the main routeRequest function and orchestrates the entire routing pipeline including scoring and analytics recording.
• server/src/services/scoring.ts – Implements the scoring algorithm that weights latency, cost, and availability to rank provider options.
• server/src/services/ratelimit.ts – Tracks global rate-limit counters per API key and provider to prevent quota overruns.
• server/src/services/provider-quota.ts – Manages per-provider quota handling and normalization against SQLite storage.
• server/src/services/quirks.ts – Handles model-specific feature flags such as vision capabilities or tool usage restrictions.
• server/src/routes/proxy.ts – The Express entry point that forwards OpenAI-compatible requests to the router service.
• server/src/routes/fallback.ts – Exposes CRUD endpoints for managing fallback profile mappings.
• server/src/services/model-groups.ts – Defines model groupings such as "auto", "fast", and "cheap" used in selection logic.
• server/src/services/media.ts – Separates handling for generative-media models, excluding them from the standard chat routing pool.

Practical Usage Examples for the FreeLLMAPI Router Service

Automatic Model Selection with "auto"

To leverage the router's intelligence, send requests with model: "auto":

import fetch from "node-fetch";

await fetch("https://api.freellmapi.com/v1/chat/completions", {
  method: "POST",
  headers: { 
    "Content-Type": "application/json", 
    "Authorization": "Bearer YOUR_KEY" 
  },
  body: JSON.stringify({
    model: "auto",
    messages: [{ 
      role: "user", 
      content: "Explain quantum tunneling in plain language." 
    }]
  })
}).then(r => r.json())
  .then(console.log);

When model: "auto" is supplied, the request triggers routeRequest inside router.ts, which executes the scoring pipeline and returns the selected provider's response.

Bypassing the Router for Direct Provider Access

To skip automatic selection and target a specific provider:

await fetch("https://api.freellmapi.com/v1/chat/completions", {
  method: "POST",
  headers: { 
    "Content-Type": "application/json", 
    "Authorization": "Bearer YOUR_KEY" 
  },
  body: JSON.stringify({
    model: "openrouter/google/gemma-1.5-7b",
    messages: [{ 
      role: "user", 
      content: "Write a haiku about sunrise." 
    }]
  })
});

Specifying a concrete platform/model identifier skips the auto-selection step and routes the request directly to the chosen provider.

Managing Fallback Profiles via API

The router supports dynamic fallback configuration through the REST API:

// List all fallback profiles
await fetch("https://api.freellmapi.com/v1/fallbacks", {
  headers: { Authorization: "Bearer YOUR_KEY" }
}).then(r => r.json()).then(console.log);

// Update a profile to map a free model to a paid alternative
await fetch("https://api.freellmapi.com/v1/fallbacks/42", {
  method: "PUT",
  headers: { 
    "Content-Type": "application/json", 
    "Authorization": "Bearer YOUR_KEY" 
  },
  body: JSON.stringify({ 
    primary: "openrouter/google/gemma-1.5-7b", 
    fallback: "openrouter/google/gemma-1.5-pro" 
  })
});

These endpoints are implemented in routes/fallback.ts, which manipulates the router's internal fallback tables used during resolveRoutingChain execution.

Summary

• The FreeLLMAPI Router Service dynamically selects optimal language models from multiple providers when clients specify model: "auto".
• It enforces strict quota and rate-limit checks via provider-quota.ts and ratelimit.ts before allowing requests to reach providers.
• Fallback profiles and model quirks are managed through dedicated services that inform the routing decision in router.ts.
• All routing decisions are logged via recordSuccess and recordRateLimitHit to maintain analytics and improve future scoring.
• The service exposes a unified OpenAI-compatible API through server/src/routes/proxy.ts, abstracting provider-specific implementations.

Frequently Asked Questions

What happens when I send model: "auto" to the FreeLLMAPI Router Service?

The router evaluates all enabled providers using the scoring algorithm in server/src/services/scoring.ts, checks quotas in server/src/services/provider-quota.ts, and selects the optimal model based on latency, cost, and availability. This selection happens inside the routeRequest function before forwarding your request to the chosen provider.

How does the router handle rate limiting and quotas?

Before routing any request, the service checks SQLite-backed quota counters in server/src/services/provider-quota.ts and global rate-limit trackers in server/src/services/ratelimit.ts. If a provider has exceeded its daily budget or an API key has hit its limit, that provider is excluded from the candidate pool and the router falls back to the next best option.

Can I force the router to use a specific model instead of auto-selecting?

Yes. By specifying a concrete provider identifier such as openrouter/google/gemma-1.5-7b in the model field, you bypass the automatic selection logic entirely. The request skips the scoring pipeline in router.ts and routes directly to the specified provider.

What are fallback profiles and how do I configure them?

Fallback profiles are routing rules that map primary models to alternatives when failures occur. Administrators can manage these via the /v1/fallbacks endpoints implemented in server/src/routes/fallback.ts, or the router consults these profiles during resolveRoutingChain execution to automatically reroute failed requests to designated backup models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}