# How to Optimize Token Usage with Prompt Compression Budgets in OmniRoute

> Discover how OmniRoute optimizes token usage with prompt compression budgets using configurable limits and automatic fallback modes to reduce LLM costs.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: optimization-guide
- Published: 2026-07-26

---

**OmniRoute reduces upstream LLM token costs by enforcing configurable compression budgets that automatically fall back to less aggressive modes when token limits are exceeded.**

OmniRoute implements a sophisticated prompt-compression pipeline that helps organizations optimize token usage with prompt compression budgets. The system automatically selects the most efficient compression strategy while respecting hard token limits configured per combo. According to the OmniRoute source code, this budget-aware architecture ensures optimal cost control without requiring manual prompt engineering for every request.

## How the Compression Pipeline Works

The compression service lives in `open-sse/services/compression` and processes every request through a series of budget-aware stages. When a request arrives, the engine first resolves a **compression plan** via [`resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resolveCompressionPlan.ts). This plan evaluates combo-level overrides and checks the **budget gate** ([`budgetGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/budgetGate.ts) and its helper [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts)) to determine whether the projected token count exceeds the configured allowance.

If the request would exceed the budget, the pipeline automatically falls back through a progression of less aggressive modes: `lite` → `standard` → `off`. This guarantees that no request exceeds your configured token ceiling while still attempting to provide some level of compression.

### Strategy Selection Based on Budget

The [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) file contains the logic for choosing between available compression engines—including `lite`, `caveman`, `RTK`, and `stacked`—based on the request parameters, combo overrides, and current budget state. The selector consults the budget gate before finalizing the engine choice, ensuring the selected strategy fits within the remaining token allowance.

## Configuring Compression Budgets

Compression settings are validated by Zod schemas defined in [`src/shared/validation/compressionConfigSchemas.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/compressionConfigSchemas.ts). Per-combo budgets persist in SQLite via [`src/lib/db/compressionContextBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compressionContextBudget.ts) (re-exported through [`src/lib/localDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/localDb.ts)), allowing fine-grained control over token allocation per use case.

To set a hard budget for a specific combo, send a PUT request to the settings endpoint:

```bash
curl -X PUT https://omniroute.example.com/api/settings/compression \
  -H "Authorization: Bearer $OMNIRoute_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "compressionComboId": "combo-42",
        "budget": {
          "maxTokens": 8192,
          "hard": true
        }
      }'

```

Setting `"hard": true` invokes the enforcement logic in [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts), which prevents the pipeline from exceeding the specified `maxTokens` and triggers the fallback mechanism if necessary.

## Request-Level Compression Control

While budgets constrain the system globally, individual requests can request specific compression modes via the `X-OmniRoute-Compression` header. The header handling logic in [`open-sse/utils/compressionHeaderEcho.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/compressionHeaderEcho.ts) processes these overrides, though the budget gate may still downgrade the mode if the request exceeds the combo's token limit.

Force a specific compression mode for a single request:

```bash
curl https://omniroute.example.com/v1/chat/completions \
  -H "Authorization: Bearer $OMNIRoute_API_KEY" \
  -H "X-OmniRoute-Compression: caveman" \
  -d '{
        "model": "gpt-4o",
        "messages": [{"role":"user","content":"<very long prompt>"}]
      }'

```

For programmatic usage, the Node.js client supports passing a compression preference that respects the budget constraints:

```ts
import { Omniglot } from "@omniroute/client";

const client = new Omniglot({ apiKey: process.env.OMNIRoute_API_KEY });

await client.chat.completions.create({
  model: "anthropic/claude-3.5-sonnet",
  messages: [{ role: "user", content: longPrompt }],
  compression: "lite",
});

```

## Monitoring Token Usage and Budget Hits

After execution, [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts) records telemetry showing tokens saved, the final compression mode applied, and whether the budget was exceeded. This data feeds into analytics dashboards and can be accessed programmatically.

Inspect the latest compression analytics to verify budget compliance:

```ts
import { getLatestCompressionAnalyticsRun } from "@/lib/db/compressionAnalytics";

async function showLastRun() {
  const run = await getLatestCompressionAnalyticsRun();
  console.log(`Combo ${run.comboId} used ${run.tokensBefore} → ${run.tokensAfter} tokens`);
  console.log(`Budget hit: ${run.budgetExceeded}`);
}
showLastRun();

```

Real-time budget telemetry streams through the WebSocket endpoint defined in [`src/server/ws/liveServer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server/ws/liveServer.ts) (lines 363-409), enabling live monitoring of token savings and budget enforcement events.

## Summary

- **Budget gates** ([`budgetGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/budgetGate.ts) and [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts)) enforce token limits by automatically falling back to less aggressive compression modes when thresholds approach.
- **Per-combo budgets** stored in [`src/lib/db/compressionContextBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compressionContextBudget.ts) provide fine-grained control over token allocation via validated JSON configurations.
- **Request headers** (`X-OmniRoute-Compression`) allow client-side hints, but the system may override them to respect hard budget constraints.
- **Telemetry collection** via [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts) and live WebSocket feeds enables real-time monitoring of token savings and budget hits.

## Frequently Asked Questions

### What happens when a request exceeds the compression budget?

When the projected token count exceeds the configured budget, the **budget gate** ([`budgetGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/budgetGate.ts)) intercepts the request and triggers the fallback logic in [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts). The system automatically downgrades the compression mode from `lite` to `standard` or ultimately to `off`, ensuring the request stays within the `maxTokens` limit while still completing successfully.

### Can I set different budget limits for different API endpoints?

Yes. OmniRoute stores budgets per **compression combo** in [`src/lib/db/compressionContextBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compressionContextBudget.ts). Each combo can have its own token ceiling and hard/soft enforcement settings, allowing you to allocate larger budgets for critical production endpoints while keeping development or testing environments strictly limited.

### How do I know which compression mode was actually applied?

The [`open-sse/utils/compressionHeaderEcho.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/compressionHeaderEcho.ts) utility echoes the final compression mode back in the response headers. Additionally, the [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts) service records the final mode used, the token count before and after compression, and whether the budget was hit—accessible via the analytics API or the live WebSocket stream in [`src/server/ws/liveServer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server/ws/liveServer.ts).

### What is the difference between hard and soft budget enforcement?

A **hard budget** (`"hard": true`) activates the enforcement logic in [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts), which prevents any compression strategy that would exceed the token limit and forces a fallback to a cheaper mode or disables compression entirely. A soft budget allows the system to exceed the limit slightly if no viable fallback exists, though OmniRoute primarily recommends hard limits for cost-sensitive production environments.