# How OmniRoute's Token Compression Reduces Token Usage: The Stacked Pipeline Explained

> Discover how OmniRoute's token compression slashes token usage by 80-95%. Learn about the stacked pipeline that efficiently removes redundant content from LLM requests.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-15

---

**OmniRoute reduces token usage by 80-95% through a stacked compression pipeline that chains multiple compression engines—typically RTK (rapid token-killer) followed by Caveman—to eliminate different classes of redundant content from LLM request payloads.**

The **OmniRoute** routing layer implements a deterministic, multi-stage compression system designed specifically for scenarios where tool results or logs balloon prompt sizes. Rather than applying a single generic algorithm, the system orchestrates specialized engines that each target distinct patterns of waste. This article examines how the stacked pipeline works, where it lives in the codebase, and how developers can leverage it.

## The Stacked Compression Architecture

OmniRoute's token compression is built around the principle that **no single engine catches every type of redundancy**. The solution is a **stacked pipeline**: requests flow through multiple compression engines sequentially, with each stage operating on the output of the previous one.

### Core Components

| Component | File Path | Responsibility |
|-----------|-----------|--------------|
| **Strategy selector** | [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) | Decides which engines to apply and orchestrates execution order |
| **RTK engine** | [`open-sse/services/compression/rtk.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/rtk.ts) | Fast O(N) line-level deduplication |
| **Caveman engine** | [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) | Deep heuristic analysis for structural patterns |
| **Schema validation** | [`src/shared/validation/compressionConfigSchemas.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/compressionConfigSchemas.ts) | Guards against over-aggressive compression |
| **Telemetry layer** | [`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts) | Records savings metrics and budget enforcement |

The **stacked** nature is essential. RTK alone would only drop repetitive lines, while Caveman alone would miss large blocks of duplicated logs. When both are applied, the test suite in [`tests/unit/compression/stacked-compression-tool-result-savings.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/stacked-compression-tool-result-savings.test.ts) confirms **>90% token savings** on deliberately noisy Anthropic `tool_result` blocks.

## How the Pipeline Works

The `applyStackedCompression` function in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) implements a five-step process:

1. **Input inspection** — Examines the request body for `tool_result` blocks or other compressible sections
2. **RTK pass** — Removes obvious line-level duplication using a small hash set of seen lines
3. **Caveman pass** — Applies richer heuristics (code-fence stripping, URL deduplication, version-string pruning, CONSTANT_CASE identifier removal)
4. **Validation fallback** — Runs `validateCompression()`; if compression would break downstream guardrails, the original payload is preserved
5. **Telemetry capture** — Persists results to `compressionRunTelemetry` for budget-gate enforcement

Because RTK shrinks the payload first, Caveman can afford more expensive analysis without impacting latency.

## The Two Compression Engines

### RTK: Rapid Token-Killer

The **RTK engine** ([`rtk.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk.ts)) provides **fast, deterministic line-level deduplication**. It operates in O(N) time and is designed for obvious noise: repeated error messages, stack trace lines, and log spam.

**What RTK removes:**
- Identical consecutive lines
- Repeated error patterns (e.g., "ERROR: connection refused at line 42" × 300)
- Whitespace-normalized duplicates

**Example transformation:**

```

Before:  300 lines of identical "ERROR: connection refused"
After:   1 representative line                    (≈95% reduction)

```

### Caveman: Deep Structural Analysis

The **Caveman engine** ([`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts)) performs **heavyweight semantic compression** on the already-reduced payload. It understands code and log structure, removing patterns that preserve meaning but consume tokens.

**What Caveman removes:**
- Code fences (```) and their redundant language tags
- URLs replaced with placeholder hashes
- Version strings (e.g., `v2.3.1-beta.4` → `vX.X.X`)
- CONSTANT_CASE identifiers in logs
- Timestamp normalization

Running Caveman after RTK is critical: the expensive analysis operates on a 95% smaller input, keeping total pipeline latency acceptable.

## Using Token Compression in Practice

### Programmatic Usage

Import `applyStackedCompression` and specify your engine stack via the configuration array:

```typescript
import { applyStackedCompression } from "open-sse/services/compression/strategySelector";

const requestBody = {
  model: "claude-sonnet-5",
  messages: [
    { role: "user", content: "Run the build and show me the log." },
    {
      role: "assistant",
      content: [
        { type: "text", text: "Running the build now." },
        {
          type: "tool_use",
          id: "toolu_01X",
          name: "Bash",
          input: { command: "npm run build" }
        }
      ]
    },
    {
      role: "user",
      content: [
        {
          type: "tool_result",
          tool_use_id: "toolu_01X",
          content: [{ type: "text", text: generateHugeLog() }] // 300+ duplicate lines
        }
      ]
    }
  ]
};

// Apply the standard RTK + Caveman stack
const compressed = applyStackedCompression(requestBody, [
  { engine: "rtk", intensity: "standard" },
  { engine: "caveman", intensity: "full" }
]);

if (compressed.compressed) {
  console.log(`✅ ${compressed.stats?.savingsPercent?.toFixed(1)}% token savings`);
} else {
  console.log("⚠️ Fallback: compression rejected by validation");
}

```

### CLI Preview and Application

The OmniRoute CLI (`bin/cli/commands/compression.mjs`) supports manual inspection and batch processing:

```bash

# Preview compression plan without modifying data

omniroute compression preview --file ./request.json

# Apply default stack and write compressed output

omniroute compression apply --file ./request.json --out ./request.compressed.json

```

### React Component Integration

For dashboard visibility, use the `CompressionPanel` component:

```tsx
import { CompressionPanel } from "@/components/compression/CompressionPanel";

function ChatMessage({ message }) {
  return (
    <div>
      <MessageContent message={message} />
      <CompressionPanel payload={message} />
    </div>
  );
}

```

## Safety Mechanisms

### Validation Guardrails

The Zod schemas in [`compressionConfigSchemas.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/compressionConfigSchemas.ts) define acceptable compression boundaries. The `validateCompression()` function checks that:

- Required code fences remain intact for markdown parsing
- Tool result identifiers are preserved
- No structural JSON is corrupted

If validation fails, the system **falls back to the original payload** rather than risk downstream errors.

### Budget-Gate Enforcement

The telemetry system prevents runaway compression. Each run records:
- `stats.savingsPercent`
- `stats.fallbackApplied`

This data feeds a **compression-budget gate** that blocks aggressive configurations proven to strip useful context in production traffic.

## Token Savings in Practice

Consider a typical CI/CD log scenario:

| Stage | Token Count | Reduction |
|-------|-------------|-----------|
| Original payload | 12,000 tokens | — |
| After RTK | 600 tokens | 95% |
| After Caveman | 400 tokens | 97% cumulative |

The **stacked pipeline achieves >90% reduction** where single-engine approaches plateau at 50-70%.

## Summary

- **Stacked compression** chains RTK then Caveman to eliminate both repetitive lines and structural redundancy
- The pipeline is **opt-in** via the request-level `compression` field; omitted requests pass through unchanged
- **Validation fallbacks** guarantee payload integrity—corrupted compressions abort to original content
- **Telemetry and budget gates** prevent over-compression that would hide critical context
- All orchestration happens in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts), with engines isolated in [`rtk.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk.ts) and [`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts)

## Frequently Asked Questions

### What compression ratio can OmniRoute achieve?

According to the source code test suite in [`stacked-compression-tool-result-savings.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stacked-compression-tool-result-savings.test.ts), the RTK + Caveman stack consistently achieves **>90% token savings** on typical `tool_result` payloads. Individual results vary: highly repetitive logs may reach 95-97%, while diverse content might see 60-80%. The telemetry system records actual savings per request.

### Is token compression enabled by default?

No. OmniRoute's token compression is **opt-in** per request. Developers must explicitly include a `compression` configuration array when calling `applyStackedCompression`. If the field is omitted, the request transmits unchanged. This design prevents accidental information loss in production systems.

### How does OmniRoute prevent compression from breaking my prompts?

After the engine pipeline completes, `validateCompression()` runs against Zod schemas defined in [`compressionConfigSchemas.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/compressionConfigSchemas.ts). This validation checks structural integrity: required code fences, tool identifiers, and JSON validity. If validation fails, the system **discards the compressed version** and sends the original payload, logging the fallback decision to `compressionRunTelemetry`.

### Can I use only RTK or only Caveman?

Yes. The `applyStackedCompression` function accepts any array of engine configurations. Single-engine stacks are valid: `[{ engine: "rtk", intensity: "standard" }]` or `[{ engine: "caveman", intensity: "full" }]`. However, the documentation emphasizes that **stacked configurations** capture complementary redundancy classes that neither engine handles alone.