How OmniRoute Calculates the Free Tier Budget for Requests

OmniRoute calculates the free tier budget as the minimum of two constraints: the provider's declared rate limits (stored in freeNote fields) and a token-effort bucket derived from request thinking or budget_tokens fields using the XAI translator's budgetToEffort routine.

OmniRoute implements a sophisticated dual-source budget calculation for free-tier requests. The system treats every free-tier model as a budget-constrained endpoint, deriving the final per-request allowance from both provider-declared limits and client-specified token budgets. This approach prevents abuse while maintaining predictable performance for cost-conscious users.

Provider-Declared Rate Limits in freeNote

Each provider that offers a free tier publishes a rate-limit description in their registry entry's freeNote field. Common formats include:

• "~40 RPM"
• "2 req·s⁻¹"
• "100 req·hr⁻¹"

The open-sse/services/providerDefaultRateLimit.ts module parses these strings and computes a per-second request budget for the model. For example, "~40 RPM" (requests per minute) translates to approximately 0.66 req/s.

This provider-level budget forms the hard ceiling for all free-tier requests regardless of client configuration.

Token Budget to Effort Conversion

When a request contains a thinking or budget_tokens field, OmniRoute invokes the XAI translator's budgetToEffort routine. This function maps the supplied token count to a reasoning-effort bucket as implemented in tests/unit/xai-translators.test.ts:

Token range	Effort bucket
0, negative, or NaN	undefined (no extra budget)
1 – 3,999	low
4,000 – 15,999	medium
16,000+	high

Free-tier models are restricted to the low bucket unless the provider explicitly advertises a higher allowance. Gemini Flash's "~40 RPM" is one such exception that permits elevated effort levels.

Default Effort Fallback

If the client omits a budget entirely, the translator falls back to the default effort for the model:

• low for pure free-tier models
• medium for mixed free/paid models

Final Budget Determination

The open-sse/services/accountFallback.ts service computes the final per-request budget using:

const finalBudget = Math.min(effortToTokens(effort), providerFreeTokenCap);

This takes the minimum of:

• The token-budget-derived effort (converted to tokens)
• The provider's free-tier token cap (from freeNote processing)

If the calculation yields a budget exceeding the provider's free quota, the request is rejected with the error: "free tier of the model has been exhausted". The accountFallback.ts module classifies this as a quota-exhausted condition.

Practical Code Examples

Example 1: Client Request with Token Budget

// Client asks a free-tier Gemini model for 10,000 tokens
const request = {
  model: "gemini-2.5-flash-free",
  thinking: { type: "enabled", budget_tokens: 10_000 },
};

// XAI translator runs budgetToEffort → "medium"
const effort = budgetToEffort(request.thinking.budget_tokens); // "medium"

// Provider's freeNote: "~40 RPM" → ~0.66 req/s
// Free-tier manager caps to the lower of the two budgets
const finalBudget = Math.min(effortToTokens(effort), providerFreeTokenCap);

Example 2: budgetToEffort Implementation

// Pseudo-implementation based on tests/unit/xai-translators.test.ts
function budgetToEffort(tokens: number): "low" | "medium" | "high" | undefined {
  if (!Number.isFinite(tokens) || tokens <= 0) return undefined;
  if (tokens < 4000) return "low";
  if (tokens < 16000) return "medium";
  return "high";
}

Key Files in the Budget Calculation Pipeline

File	Purpose
open-sse/services/providerDefaultRateLimit.ts	Parses freeNote strings into per-second request budgets
tests/unit/xai-translators.test.ts	Validates budgetToEffort token-to-effort mappings
open-sse/services/accountFallback.ts	Enforces final budget limits and quota-exhausted classification

Summary

• Dual-source calculation: OmniRoute combines provider freeNote limits with client budget_tokens to determine free tier allowances
• budgetToEffort mapping: Token ranges translate to low/medium/high effort buckets, with free tiers defaulting to low
• Minimum enforcement: The final budget is always the lesser of provider capacity and client-requested effort
• Quota protection: Excessive requests trigger quota-exhausted errors via accountFallback.ts

Frequently Asked Questions

What happens if I don't specify budget_tokens in my request?

The XAI translator falls back to the model's default effort level—typically low for pure free-tier models and medium for mixed free/paid models. This ensures predictable behavior without client-side configuration.

Can I request high effort on any free-tier model?

No. Free-tier models are restricted to low effort unless the provider explicitly advertises a higher allowance in their freeNote field. The gemini-2.5-flash-free model with its "~40 RPM" designation is one documented exception.

Where does OmniRoute store provider rate limit information?

Provider rate limits are stored in the freeNote field of each provider's registry entry. The open-sse/services/providerDefaultRateLimit.ts module parses these human-readable strings (like "~40 RPM") into programmatic per-second budgets.

How does OmniRoute handle quota exhaustion?

When a request exceeds the calculated free tier budget, open-sse/services/accountFallback.ts rejects it with a quota-exhausted error and classifies the condition accordingly. This prevents cascading failures and provides clear feedback for retry logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}