How OmniRoute's Stacked Compression Reduces Token Usage: The RTK + Caveman Pipeline

OmniRoute's stacked compression pipeline sequentially applies the RTK (rapid token-killer) and Caveman engines to LLM request payloads, removing line-level duplication and deep structural redundancies to achieve 80–95% token reduction while preserving semantic integrity.

The open-source OmniRoute framework tackles the escalating cost of LLM API calls through a deterministic, multi-stage compression strategy implemented in the open-sse/services/compression/ directory. By orchestrating specialized engines in series, the system targets different classes of content waste without requiring changes to downstream prompt engineering. This approach proves particularly effective when processing repetitive error logs, verbose build outputs, or bloated API responses that would otherwise consume expensive context window capacity.

How the Stacked Compression Pipeline Works

The core orchestration happens in open-sse/services/compression/strategySelector.ts, where the applyStackedCompression function manages the sequential execution of compression engines. The pipeline follows a strict progression: input inspection, RTK deduplication, Caveman deep analysis, validation against guardrails, and telemetry recording.

Each engine catches a distinct category of redundancy, and their combined effect is multiplicative rather than additive. While RTK alone might reduce repetitive log lines, and Caveman alone might strip code fences, their stacked application achieves the documented >90% token savings on typical tool_result blocks.

Strategy Selection and Orchestration

The applyStackedCompression function serves as the entry point for the compression system. It inspects incoming request bodies for compressible sections—typically tool_result blocks containing verbose logs or API responses—and determines which engines to apply based on the configuration passed.

This architecture is opt-in via the request-level compression field. If the field is omitted, the request transmits unchanged, ensuring backward compatibility with existing implementations.

RTK Line-Level Deduplication Engine

The RTK (rapid token-killer) engine, defined in open-sse/services/compression/rtk.ts via the rtkCompression function, executes the first pass with O(N) time complexity. It maintains a small hash set of observed lines to identify and collapse consecutive duplicates.

RTK excels at high-frequency repetition scenarios. For example, when processing 300 lines of "ERROR: connection refused at line 42," the engine reduces these to a single representative line, achieving approximately 95% reduction in this phase alone. The linear time complexity ensures minimal latency impact during this initial pass.

Caveman Deep-Deduplication Engine

Following RTK's line-level cleanup, the Caveman engine—implemented in open-sse/services/compression/caveman.ts as cavemanCompression—performs heavyweight semantic analysis. Because RTK has already dramatically reduced payload size, Caveman can afford computationally expensive pattern matching without hurting overall request latency.

Caveman removes structural redundancies that line-level deduplication misses, including:

• Code fences and syntactic markers
• Duplicate URLs and version strings
• CONSTANT_CASE identifiers
• Timestamp patterns and other boilerplate

This second pass ensures that even after aggressive line deduplication, remaining structural noise is eliminated. The test suite in tests/unit/compression/stacked-compression-tool-result-savings.test.ts confirms that this stacked approach consistently achieves >90% token savings on deliberately noisy Anthropic tool results.

Validation and Safety Guardrails

After both compression stages, the system executes validateCompression() to ensure the transformed payload remains compatible with downstream LLM provider requirements. The validation layer uses Zod schemas defined in src/shared/validation/compressionConfigSchemas.ts to guard against over-aggressive compression.

If the compressed content would violate structural requirements—for instance, by removing a required code fence or breaking JSON syntax—the pipeline aborts and transmits the original payload unchanged. This safety mechanism ensures that token reduction never compromises payload integrity or causes downstream guardrail violations.

Telemetry and Budget Controls

Every compression run is instrumented through the database layer in src/lib/db/compression.ts, which writes metrics to the compressionRunTelemetry table. This persistence layer records token counts, savings percentages, fallback decisions, and engine-specific performance data.

The telemetry feeds into a compression-budget gate that prevents runaway token reduction from hiding diagnostically useful information. This budget mechanism analyzes historical compression data to ensure that aggressive settings do not strip context essential for accurate LLM responses, particularly in debugging scenarios where error frequency matters.

Implementation Examples

Developers can invoke OmniRoute's stacked compression through the library API, CLI tools, or React components.

Node.js/TypeScript Integration

Apply the RTK + Caveman stack programmatically before sending requests to LLM providers:

import { applyStackedCompression } from "open-sse/services/compression/strategySelector";

const requestBody = {
  model: "claude-sonnet-5",
  messages: [
    { role: "user", content: "Run the build and show me the log." },
    {
      role: "assistant",
      content: [
        { type: "text", text: "Running the build now." },
        { type: "tool_use", id: "toolu_01X", name: "Bash", input: { command: "npm run build" } },
      ],
    },
    {
      role: "user",
      content: [
        {
          type: "tool_result",
          tool_use_id: "toolu_01X",
          content: [{ type: "text", text: generateHugeLog() }], // 300+ duplicate lines
        },
      ],
    },
  ],
};

// Apply RTK + Caveman (the most common stack)
const compressed = applyStackedCompression(requestBody, [
  { engine: "rtk", intensity: "standard" },
  { engine: "caveman", intensity: "full" },
]);

if (compressed.compressed) {
  console.log(`✅ Compression applied – ${compressed.stats?.savingsPercent?.toFixed(1)}% token savings`);
} else {
  console.log("⚠️ Compression not applied – fallback or validation issue");
}

CLI Usage

Preview and apply compression directly from the terminal using the entry point in bin/cli/commands/compression.mjs:

# Show a preview of the compression plan for a JSON request file

omniroute compression preview --file ./request.json

# Apply the default stacked pipeline (RTK + Caveman) and write the compressed payload to a new file

omniroute compression apply --file ./request.json --out ./request.compressed.json

React Component Integration

Display compression statistics within your application UI:

import { CompressionPanel } from "@/components/compression/CompressionPanel";

function ChatMessage({ message }) {
  return (
    <div>
      <MessageContent message={message} />
      {/* Show the compression stats if the message was compressed */}
      <CompressionPanel payload={message} />
    </div>
  );
}

Summary

• OmniRoute's stacked compression combines the RTK and Caveman engines in open-sse/services/compression/strategySelector.ts to eliminate both line-level and structural redundancy in LLM payloads.
• The pipeline achieves 80–95% token reduction on typical tool results, with test suites confirming >90% savings on noisy Anthropic tool_result blocks.
• Safety guardrails in src/shared/validation/compressionConfigSchemas.ts automatically fall back to uncompressed payloads if compression would break downstream requirements or strip essential context.
• Telemetry persistence via src/lib/db/compression.ts enables budget gates that prevent over-compression from hiding critical diagnostic information during high-volume operations.
• The system is opt-in via request-level configuration; unconfigured requests transmit unchanged, ensuring full backward compatibility.

Frequently Asked Questions

What is the typical token reduction percentage when using OmniRoute's stacked compression?

In typical "tool-result" scenarios involving repetitive logs or error output, OmniRoute's stacked compression achieves 80–95% token reduction. The test suite in tests/unit/compression/stacked-compression-tool-result-savings.test.ts specifically documents >90% savings on deliberately noisy Anthropic tool_result blocks, with RTK handling the bulk of line-level duplication and Caveman removing remaining structural redundancies.

How does the validation fallback mechanism protect against over-compression?

After the RTK and Caveman engines process the payload, the validateCompression() function checks the result against Zod schemas defined in src/shared/validation/compressionConfigSchemas.ts. If the compressed content would violate structural requirements—such as missing required code fences or breaking syntax guardrails—the system aborts compression and sends the original payload unchanged, ensuring that token reduction never compromises payload integrity.

Can developers configure which compression engines to use or adjust their intensity?

Yes. The applyStackedCompression function accepts an array of engine configurations allowing developers to specify which engines to run and at what intensity. For example, you can run RTK at "standard" intensity followed by Caveman at "full" intensity, or implement custom stacks for specific payload types. If the compression field is omitted from the request, the system transmits the payload unchanged, making the pipeline strictly opt-in.

Where does OmniRoute store compression analytics and token savings data?

Compression telemetry is persisted through the database layer in src/lib/db/compression.ts, specifically written to the compressionRunTelemetry table. This stores token counts, savings percentages, fallback decisions, and engine performance metrics. This data powers the compression-budget gate that prevents aggressive compression from hiding useful diagnostic information across high-volume LLM operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}