How OmniRoute's Token Compression Reduces Token Usage: The Stacked Pipeline Explained

OmniRoute reduces token usage by 80-95% through a stacked compression pipeline that chains multiple compression engines—typically RTK (rapid token-killer) followed by Caveman—to eliminate different classes of redundant content from LLM request payloads.

The OmniRoute routing layer implements a deterministic, multi-stage compression system designed specifically for scenarios where tool results or logs balloon prompt sizes. Rather than applying a single generic algorithm, the system orchestrates specialized engines that each target distinct patterns of waste. This article examines how the stacked pipeline works, where it lives in the codebase, and how developers can leverage it.

The Stacked Compression Architecture

OmniRoute's token compression is built around the principle that no single engine catches every type of redundancy. The solution is a stacked pipeline: requests flow through multiple compression engines sequentially, with each stage operating on the output of the previous one.

Core Components

Component	File Path	Responsibility
Strategy selector	open-sse/services/compression/strategySelector.ts	Decides which engines to apply and orchestrates execution order
RTK engine	open-sse/services/compression/rtk.ts	Fast O(N) line-level deduplication
Caveman engine	open-sse/services/compression/caveman.ts	Deep heuristic analysis for structural patterns
Schema validation	src/shared/validation/compressionConfigSchemas.ts	Guards against over-aggressive compression
Telemetry layer	src/lib/db/compression.ts	Records savings metrics and budget enforcement

The stacked nature is essential. RTK alone would only drop repetitive lines, while Caveman alone would miss large blocks of duplicated logs. When both are applied, the test suite in tests/unit/compression/stacked-compression-tool-result-savings.test.ts confirms >90% token savings on deliberately noisy Anthropic tool_result blocks.

How the Pipeline Works

The applyStackedCompression function in strategySelector.ts implements a five-step process:

1. Input inspection — Examines the request body for tool_result blocks or other compressible sections
2. RTK pass — Removes obvious line-level duplication using a small hash set of seen lines
3. Caveman pass — Applies richer heuristics (code-fence stripping, URL deduplication, version-string pruning, CONSTANT_CASE identifier removal)
4. Validation fallback — Runs validateCompression(); if compression would break downstream guardrails, the original payload is preserved
5. Telemetry capture — Persists results to compressionRunTelemetry for budget-gate enforcement

Because RTK shrinks the payload first, Caveman can afford more expensive analysis without impacting latency.

The Two Compression Engines

RTK: Rapid Token-Killer

The RTK engine (rtk.ts) provides fast, deterministic line-level deduplication. It operates in O(N) time and is designed for obvious noise: repeated error messages, stack trace lines, and log spam.

What RTK removes:

• Identical consecutive lines
• Repeated error patterns (e.g., "ERROR: connection refused at line 42" × 300)
• Whitespace-normalized duplicates

Example transformation:

Before:  300 lines of identical "ERROR: connection refused"
After:   1 representative line                    (≈95% reduction)

Caveman: Deep Structural Analysis

The Caveman engine (caveman.ts) performs heavyweight semantic compression on the already-reduced payload. It understands code and log structure, removing patterns that preserve meaning but consume tokens.

What Caveman removes:

• Code fences (```) and their redundant language tags
• URLs replaced with placeholder hashes
• Version strings (e.g., v2.3.1-beta.4 → vX.X.X)
• CONSTANT_CASE identifiers in logs
• Timestamp normalization

Running Caveman after RTK is critical: the expensive analysis operates on a 95% smaller input, keeping total pipeline latency acceptable.

Using Token Compression in Practice

Programmatic Usage

Import applyStackedCompression and specify your engine stack via the configuration array:

import { applyStackedCompression } from "open-sse/services/compression/strategySelector";

const requestBody = {
  model: "claude-sonnet-5",
  messages: [
    { role: "user", content: "Run the build and show me the log." },
    {
      role: "assistant",
      content: [
        { type: "text", text: "Running the build now." },
        {
          type: "tool_use",
          id: "toolu_01X",
          name: "Bash",
          input: { command: "npm run build" }
        }
      ]
    },
    {
      role: "user",
      content: [
        {
          type: "tool_result",
          tool_use_id: "toolu_01X",
          content: [{ type: "text", text: generateHugeLog() }] // 300+ duplicate lines
        }
      ]
    }
  ]
};

// Apply the standard RTK + Caveman stack
const compressed = applyStackedCompression(requestBody, [
  { engine: "rtk", intensity: "standard" },
  { engine: "caveman", intensity: "full" }
]);

if (compressed.compressed) {
  console.log(`✅ ${compressed.stats?.savingsPercent?.toFixed(1)}% token savings`);
} else {
  console.log("⚠️ Fallback: compression rejected by validation");
}

CLI Preview and Application

The OmniRoute CLI (bin/cli/commands/compression.mjs) supports manual inspection and batch processing:

# Preview compression plan without modifying data

omniroute compression preview --file ./request.json

# Apply default stack and write compressed output

omniroute compression apply --file ./request.json --out ./request.compressed.json

React Component Integration

For dashboard visibility, use the CompressionPanel component:

import { CompressionPanel } from "@/components/compression/CompressionPanel";

function ChatMessage({ message }) {
  return (
    <div>
      <MessageContent message={message} />
      <CompressionPanel payload={message} />
    </div>
  );
}

Safety Mechanisms

Validation Guardrails

The Zod schemas in compressionConfigSchemas.ts define acceptable compression boundaries. The validateCompression() function checks that:

• Required code fences remain intact for markdown parsing
• Tool result identifiers are preserved
• No structural JSON is corrupted

If validation fails, the system falls back to the original payload rather than risk downstream errors.

Budget-Gate Enforcement

The telemetry system prevents runaway compression. Each run records:

• stats.savingsPercent
• stats.fallbackApplied

This data feeds a compression-budget gate that blocks aggressive configurations proven to strip useful context in production traffic.

Token Savings in Practice

Consider a typical CI/CD log scenario:

Stage	Token Count	Reduction
Original payload	12,000 tokens	—
After RTK	600 tokens	95%
After Caveman	400 tokens	97% cumulative

The stacked pipeline achieves >90% reduction where single-engine approaches plateau at 50-70%.

Summary

• Stacked compression chains RTK then Caveman to eliminate both repetitive lines and structural redundancy
• The pipeline is opt-in via the request-level compression field; omitted requests pass through unchanged
• Validation fallbacks guarantee payload integrity—corrupted compressions abort to original content
• Telemetry and budget gates prevent over-compression that would hide critical context
• All orchestration happens in strategySelector.ts, with engines isolated in rtk.ts and caveman.ts

Frequently Asked Questions

What compression ratio can OmniRoute achieve?

According to the source code test suite in stacked-compression-tool-result-savings.test.ts, the RTK + Caveman stack consistently achieves >90% token savings on typical tool_result payloads. Individual results vary: highly repetitive logs may reach 95-97%, while diverse content might see 60-80%. The telemetry system records actual savings per request.

Is token compression enabled by default?

No. OmniRoute's token compression is opt-in per request. Developers must explicitly include a compression configuration array when calling applyStackedCompression. If the field is omitted, the request transmits unchanged. This design prevents accidental information loss in production systems.

How does OmniRoute prevent compression from breaking my prompts?

After the engine pipeline completes, validateCompression() runs against Zod schemas defined in compressionConfigSchemas.ts. This validation checks structural integrity: required code fences, tool identifiers, and JSON validity. If validation fails, the system discards the compressed version and sends the original payload, logging the fallback decision to compressionRunTelemetry.

Can I use only RTK or only Caveman?

Yes. The applyStackedCompression function accepts any array of engine configurations. Single-engine stacks are valid: [{ engine: "rtk", intensity: "standard" }] or [{ engine: "caveman", intensity: "full" }]. However, the documentation emphasizes that stacked configurations capture complementary redundancy classes that neither engine handles alone.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}