# OmniRoute Token Compression: How the Multi-Engine Pipeline Reduces LLM Costs

> Discover OmniRoute token compression which slashes LLM costs by 15-95% using a multi-engine pipeline to rewrite prompts. Control costs globally or per request.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-27

---

**OmniRoute's token compression feature rewrites prompts through a configurable pipeline of specialized engines** (Caveman, RTK, Stacked, etc.) **to reduce token consumption by 15–95%, controlled globally or per-request via the** `x-omniroute-compression` **header.**

The **OmniRoute token compression** system in the `diegosouzapw/OmniRoute` repository is a modular, plug-in pipeline that shrinks prompts before they reach language models. By implementing multiple compression engines—from lightweight URL stripping to aggressive summarization—it allows the same logical requests to fit within smaller token windows while consuming fewer provider-quota tokens.

## Compression Modes and Configuration Options

OmniRoute offers seven distinct compression modes that trade token savings against content preservation. The mode determines which engine or stack of engines processes the request.

| Mode | Description | Typical Token Savings |
|------|-------------|----------------------|
| **off** | No compression applied; raw prompt forwarded. | 0% |
| **lite** | Strips obvious image URLs and removes empty blocks. | Low |
| **standard** | Uses the Caveman engine for aggressive trimming, deduplication, and summarization. | 15–60% |
| **aggressive** | Enhanced Caveman execution with stricter truncation. | 40–80% |
| **ultra** | Maximum aggressive trimming for largest possible reduction. | 60–95% |
| **rtk** | Retrieval-augmented compression using the RTK engine and stored snippets. | Variable |
| **stacked** | Runs a sequenced stack (Caveman → RTK → …) to maximize savings. | Highest |

Global activation is controlled via the `OMNI_COMPRESSION_WORKERS` environment variable and database configuration, while per-request overrides use the `x-omniroute-compression` header documented in [`src/app/api/reference/API_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/reference/API_REFERENCE.md).

## The Compression Pipeline Architecture

The system processes every chat completion request through a structured flow from entry to compressed payload.

### Request Entry and Routing

The handler for chat completions in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) inspects both the global compression switch and the incoming request headers. If the `x-omniroute-compression` header is present, it takes precedence over the global setting. The validation layer defined in [`open-sse/services/compression/validation.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/validation.ts) ensures the requested mode is supported before execution begins.

### Plan Resolution and Engine Selection

Once a mode is determined, [`open-sse/services/compression/resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/resolveCompressionPlan.ts) builds a concrete **compression plan** enriched with the runtime **engine catalog** from [`open-sse/services/compression/engineCatalog.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engineCatalog.ts). This catalog registers all available implementations including Caveman, RTK, and Stacked variants. The [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) module then maps the plan to a concrete engine instance.

### Execution and Worker Pool

Heavy CPU work is delegated to a **worker pool** managed by [`open-sse/services/compression/compressionWorkerPool.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/compressionWorkerPool.ts) to keep the main event loop responsive. The public entry point `applyCompressionAsync`—exposed at [`src/app/api/compression/preview/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/compression/preview/route.ts) and consumed internally by [`src/app/api/internal/codex-responses-ws/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/internal/codex-responses-ws/compression.ts)—hands the request body to the selected engine for processing.

## Built-in Compression Engines

Each engine implements a specific strategy for reducing token count while preserving semantic intent.

### Caveman Engine (Standard and Aggressive Modes)

The [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) file implements the primary compression logic used in **standard** and **aggressive** modes. It performs aggressive trimming, deduplication of repeated phrases, and summarization of verbose content sections. This engine yields the bulk of token savings for general-purpose prompts.

### RTK Engine (Retrieval-Augmented Compression)

For **rtk** mode, the system leverages [`open-sse/services/compression/engines/rtk/smartTruncate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/smartTruncate.ts) and related files in the RTK directory. This engine performs smart truncation based on stored knowledge snippets, deduplication against retrieved context, and line-filtering to remove redundant technical details while preserving domain-specific terminology.

### Stacked Pipeline Orchestration

When **stacked** mode is requested, [`open-sse/services/compression/stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stackedStepCore.ts) coordinates the execution of multiple engines in sequence—typically running Caveman followed by RTK—to achieve maximum token reduction. The output of each stage feeds into the next, compounding savings while allowing each engine to handle the content it optimizes best.

### Safety Mechanisms

The pipeline includes protective steps to prevent critical information loss. [`open-sse/services/compression/riskGate/riskGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/riskGate/riskGate.ts) aborts compression entirely if risky patterns (such as specific code signatures or legal clauses) are detected. Additionally, [`open-sse/services/compression/prefixFreeze.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/prefixFreeze.ts) protects frequently-observed system-prompt prefixes from being trimmed, ensuring behavioral instructions remain intact.

## Configuring and Using Token Compression

Developers can interact with the compression system through global settings, HTTP headers, or preview endpoints.

### Global Configuration

Enable compression across all requests by setting the environment variable `OMNI_COMPRESSION_WORKERS` to a positive integer indicating the number of worker threads. Database-level configuration can further tune default modes and engine parameters.

### Per-Request Header Overrides

Force a specific compression mode for individual requests using the `x-omniroute-compression` header:

```http
POST /api/chat/completions HTTP/1.1
Content-Type: application/json
x-omniroute-compression: ultra

{
  "model": "gpt-4o-mini",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Explain quantum computing in detail..." }
  ]
}

```

From a TypeScript client:

```typescript
import { fetchChat } from "omniroute-client";

await fetchChat({
  model: "gpt-4o-mini",
  messages,
  headers: { "x-omniroute-compression": "rtk" }
});

```

### Previewing Compression Results

Inspect how a specific prompt will be transformed before sending it to the model by calling the preview endpoint implemented in [`src/app/api/compression/preview/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/compression/preview/route.ts):

```http
POST /api/compression/preview HTTP/1.1
Content-Type: application/json
x-omniroute-compression: stacked

{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Explain token compression in detail..." }
  ],
  "max_output_tokens": 1024
}

```

The endpoint returns the compressed payload and token statistics without forwarding the request to a language model provider.

## Analytics and Observability

OmniRoute tracks compression performance through a dedicated analytics subsystem.

### Database Persistence

Token counts before and after compression are persisted to the `compression_analytics` table, created by the migration runner in [`src/lib/db/migrationRunner.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/migrationRunner.ts). This enables historical analysis of savings per mode and engine.

### Analytics Endpoint

Query aggregated statistics via the endpoint served by [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts):

```http
GET /api/analytics/compression HTTP/1.1
Authorization: Bearer <admin-token>

```

This returns JSON showing total tokens saved, compression ratios, and mode distribution:

```json
{
  "totalTokensBefore": 125000,
  "totalTokensAfter": 72000,
  "tokensSaved": 53000,
  "modeDistribution": { "lite": 12, "standard": 34, "rtk": 7 }
}

```

## Summary

- **OmniRoute token compression** reduces prompt size through a modular pipeline of specialized engines including Caveman, RTK, and Stacked modes, achieving **15–95% token savings**.
- The system is controlled globally via `OMNI_COMPRESSION_WORKERS` or per-request using the **`x-omniroute-compression`** header, with validation handled in [`open-sse/services/compression/validation.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/validation.ts).
- **Plan resolution** occurs in [`open-sse/services/compression/resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/resolveCompressionPlan.ts), while **engine selection** is managed by [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts).
- CPU-intensive compression runs in a **worker pool** ([`open-sse/services/compression/compressionWorkerPool.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/compressionWorkerPool.ts)) to maintain API responsiveness.
- **Safety mechanisms** including risk gates ([`open-sse/services/compression/riskGate/riskGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/riskGate/riskGate.ts)) and prefix freezing ([`open-sse/services/compression/prefixFreeze.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/prefixFreeze.ts)) prevent loss of critical context.
- Operators can **preview compression** via [`src/app/api/compression/preview/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/compression/preview/route.ts) and monitor savings through the `compression_analytics` table and `/api/analytics/compression` endpoint.

## Frequently Asked Questions

### What compression mode should I use for production?

Start with **standard** mode (`x-omniroute-compression: standard`), which uses the Caveman engine in [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) to achieve 15–60% savings with minimal semantic loss. If you operate with large context windows or repetitive technical documentation, experiment with **rtk** mode for retrieval-augmented compression, or **stacked** mode for maximum efficiency when latency is acceptable.

### Can I disable compression for specific requests?

Yes. Send the header `x-omniroute-compression: off` to bypass the pipeline for individual requests, or set the global environment variable to disable workers entirely. The request handler in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) checks this header before invoking `applyCompressionAsync`, allowing fine-grained control over which prompts get processed.

### How does OmniRoute ensure important system instructions aren't lost during compression?

The pipeline implements **prefix freezing** via [`open-sse/services/compression/prefixFreeze.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/prefixFreeze.ts), which protects frequently-observed system-prompt prefixes from trimming. Additionally, the **risk gate** in [`open-sse/services/compression/riskGate/riskGate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/riskGate/riskGate.ts) scans for risky patterns before compression begins and aborts the process if sensitive content structures are detected, ensuring critical instructions remain intact.

### Where does OmniRoute store compression statistics?

Token counts before and after compression are written to the **`compression_analytics`** table, created by [`src/lib/db/migrationRunner.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/migrationRunner.ts). Administrators can query aggregated metrics through the `/api/analytics/compression` endpoint implemented in [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts), which returns JSON containing total savings and mode distribution across the fleet.