FreeLLMAPI Rate Limit Scopes: Account, Model, and Endpoint Limits Explained

FreeLLMAPI enforces rate limits at three distinct hierarchical scopes—account (provider), model, and endpoint—to granularly control API usage and prevent quota exhaustion across multiple LLM providers.

FreeLLMAPI is an open-source unified API gateway that aggregates access to various language model providers while maintaining strict quota management. The tashfeenahmed/freellmapi repository implements a multi-layered rate limiting system that operates at three specific levels to respect upstream provider constraints and enable fine-grained access control.

The Three Rate Limit Scopes

FreeLLMAPI categorizes rate restrictions into three granular scopes, each tracked independently in the routing layer.

Account (Provider) Scope

The account scope represents the broadest restriction level, limiting all requests that share the same provider account. According to the source code in server/src/services/provider-quota.ts, the system registers special account-wide identifiers such as modelscope::account to enforce global daily or monthly quotas imposed by free-tier providers. This scope aggregates usage across every model and endpoint associated with a specific provider credential set.

Model Scope

At the intermediate level, the model scope restricts requests targeting a specific model ID—such as gpt-4o or gemma-2b—regardless of which API key initiates the request. In server/src/lib/model-scope.js, the parseModelScope function reads the JSON configuration stored in the api_keys.model_scope_json column, while scopeAllows validates that the requesting key's permitted scope includes the target model identifier. This enables providers to enforce per-model caps, for example limiting a specific model to 200 requests per day.

Endpoint (Custom) Scope

The most specific restriction, the endpoint scope, applies to requests routed through custom relay endpoints identified by their base URL. As implemented in server/src/lib/endpoint-scope.js, the endpointScopeForBaseUrl function maps a custom base URL to an internal string identifier stored as endpoint_scope on the model row. This mechanism allows self-hosted relay services to impose their own limits, such as 100 requests per minute per relay endpoint.

How the Router Applies Rate Limit Scopes

When processing requests in server/src/services/router.ts, FreeLLMAPI applies these scopes through a two-phase filtering system.

First, the router filters out any API keys whose model scope does not admit the requested model using the scopeAllows helper. After this filtering step, the remaining keys are ranked by health metrics, recent rate-limit penalties, and the configured routing strategy.

The rate-limit factor (rateLimitFactor), implemented in server/src/services/ratelimit.ts, then demotes keys that have recently triggered HTTP 429 responses or that reside in throttled scopes. This ensures the routing algorithm selects keys with available quota capacity, preventing unnecessary rate limit errors.

Code Examples for Working With Rate Limits

Checking Current Rate Limit Status

The following example demonstrates how to retrieve rate limit information for a specific model via the public API:

// Fetch rate limit status for gpt-4o
const resp = await fetch('https://api.freellmapi.com/v1/models?model=gpt-4o');
const data = await resp.json();

console.log('Rate limit info:', data.rate_limit);
// Output: { limit: 2000, remaining: 1587, reset: 1693425600 }

The response includes the limit (total allowed), remaining (calls left), and reset (epoch timestamp when the quota refreshes), sourced from the provider-specific quota registration in provider-quota.ts.

Using Custom Endpoints With Scope Headers

To route requests through a custom relay endpoint with its own rate limit scope:

const resp = await fetch(
  'https://api.freellmapi.com/v1/chat/completions',
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <YOUR_KEY>',
      'X-Endpoint-Url': 'https://my-relay.example.com'
    },
    body: JSON.stringify({ 
      model: 'my-custom-model', 
      messages: [{ role: 'user', content: 'Hello' }]
    })
  }
);

const result = await resp.json();
console.log('Endpoint-scoped remaining:', result.rate_limit.remaining);

The X-Endpoint-Url header is transformed by endpointScopeForBaseUrl into an internal endpoint_scope string, allowing the router to apply the appropriate per-endpoint quota.

Programmatically Validating Model Scopes

To check which models a specific API key can access based on its scope configuration:

import { parseModelScope, scopeAllows } from 'server/lib/model-scope.js';

const keyInfo = await db.query(
  'SELECT model_scope_json FROM api_keys WHERE id = ?', 
  [keyId]
);

const scopes = parseModelScope(keyInfo.model_scope_json);
console.log('Key can serve models:', [...scopes]);

// Check if specific model is allowed
const canServeGpt4o = scopeAllows(scopes, 'gpt-4o');

This logic mirrors the internal validation used by the router to enforce model scope restrictions before forwarding requests.

Summary

FreeLLMAPI implements a hierarchical rate limiting system across three distinct scopes:

• Account (Provider) Scope: Enforced via server/src/services/provider-quota.ts to manage global provider quotas
• Model Scope: Validated through parseModelScope and scopeAllows in server/src/lib/model-scope.js to restrict access per model ID
• Endpoint Scope: Mapped using endpointScopeForBaseUrl in server/src/lib/endpoint-scope.js for custom relay limits

The routing layer in server/src/services/router.ts filters keys by model scope first, then applies the rateLimitFactor from server/src/services/ratelimit.ts to select viable keys and prevent 429 errors.

Frequently Asked Questions

What happens when a rate limit is exceeded in FreeLLMAPI?

When a request exceeds any of the three scope limits, the system returns an HTTP 429 (Too Many Requests) response. The rateLimitFactor mechanism in server/src/services/ratelimit.ts tracks these failures and temporarily demotes the affected keys in the routing pool, preventing subsequent requests from hitting the same exhausted quota until the reset period elapses.

How do I check which models my API key can access?

You can inspect the model_scope_json field associated with your API key in the database, then parse it using the parseModelScope function from server/src/lib/model-scope.js. This returns a Set of model identifiers that the key is authorized to request, reflecting the model scope constraints configured for your key.

Can I set custom rate limits for my own relay endpoint?

Yes, by specifying a custom endpoint URL via the X-Endpoint-Url header, FreeLLMAPI generates an endpoint scope identifier using endpointScopeForBaseUrl. You can then configure specific rate limits for this scope in the system configuration, allowing your self-hosted relay to enforce independent quotas such as 100 requests per minute.

Where are provider-specific rate limit quotas defined?

Provider-specific quotas, such as ModelScope's 2000 requests per day, are registered in server/src/services/provider-quota.ts. This service maintains the account-wide scope identifiers (e.g., modelscope::account) that track aggregate usage across all models for a given provider account.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}