Token Saving with OmniRoute's RTK and Caveman Stacked Compression: A Complete Guide

OmniRoute's stacked compression mode combining RTK and Caveman engines typically achieves 30%–45% token reduction, saving 150–200 tokens on a 500-token conversational payload.

This implementation in diegosouzapw/OmniRoute runs two complementary compression engines sequentially to maximize token efficiency. The RTK engine performs history deduplication, while Caveman aggressively prunes older, low-utility content. Understanding this stacked architecture helps optimize your LLM request costs and stay within provider token limits.

How Stacked Compression Works in OmniRoute

The stacked compression mode processes each request through two distinct stages:

• RTK (Redis-Token-Keeper) — First pass: Removes identical or near-identical user-assistant exchanges through deterministic deduplication
• Caveman — Second pass: Applies aggressive pruning to older messages and long-form blocks that exceed configured token budgets

When mode === "stacked" is specified, the output from RTK becomes the input to Caveman. Only tokens surviving both stages reach the final payload.

Token Saving Calculation

The core routine in open-sse/services/compression/strategySelector.ts calculates savings as follows:

// Simplified from applyCompression / applyCompressionAsync
const originalTokens = countTokens(body);
const compressedBody = runEnginePipeline(body, mode);
const compressedTokens = countTokens(compressedBody);
const tokensSaved = originalTokens - compressedTokens;

// Persisted to compression_analytics table
db.prepare(`
  INSERT INTO compression_analytics
  (timestamp, mode, engine, original_tokens, compressed_tokens, tokens_saved)
  VALUES (?, ?, ?, ?, ?, ?)
`).run(
  Date.now(),
  mode,
  mode === "stacked" ? "stacked" : engine,
  originalTokens,
  compressedTokens,
  tokensSaved,
);

For stacked runs, the tokens_saved value represents the combined total from both engines.

Measured Token Savings from Test Suite

The OmniRoute test suite provides concrete figures for stacked compression performance:

Test File	Total Tokens Saved	RTK Contribution	Caveman Contribution
stacked-pipeline-engines-fallback-6463.test.ts	150	~100 tokens	~50 tokens
rtk-caveman-stacked-pipeline.test.ts	180	~110 tokens	~70 tokens

These tests demonstrate the additive nature of stacked compression: RTK handles immediate redundancy elimination, then Caveman extracts additional savings from remaining older content.

Expected Savings Range

Based on typical conversational payloads:

• Absolute savings: 150–200 tokens per request
• Percentage reduction: 30%–45%
• Variation factors: Request length, content type, and conversation history depth

Querying Token Saving Analytics

Request-Level Totals

Retrieve the most recent stacked compression result from the compression_analytics table:

import { db } from "@/lib/db/core";

const row = db
  .prepare(`
    SELECT original_tokens, compressed_tokens, tokens_saved
    FROM compression_analytics
    WHERE mode = 'stacked'
    ORDER BY timestamp DESC
    LIMIT 1
  `)
  .get();

console.log(
  `Original: ${row.original_tokens} tokens, ` +
  `Compressed: ${row.compressed_tokens} tokens, ` +
  `Saved: ${row.tokens_saved} tokens (${(
    (row.tokens_saved / row.original_tokens) *
    100
  ).toFixed(1)}%)`,
);

Per-Engine Breakdown

OmniRoute stores granular attribution in compression_engine_breakdown:

const breakdown = db
  .prepare(`
    SELECT engine, tokens_saved
    FROM compression_engine_breakdown
    WHERE request_id = ?
  `)
  .all(requestId);

breakdown.forEach(({ engine, tokens_saved }) => {
  console.log(`${engine} saved ${tokens_saved} tokens`);
});
// Output:
// rtk saved 110 tokens
// caveman saved 70 tokens

Source Files and Implementation Details

File	Purpose
open-sse/services/compression/strategySelector.ts	Core entry point with applyCompression/applyCompressionAsync functions that orchestrate stacked execution and record tokens_saved
open-sse/services/compression/engineCatalog.ts	Maps engine identifiers (rtk, caveman) to implementation classes
open-sse/services/compression/engines/rtk/	RTK deduplication engine directory
open-sse/services/compression/engines/cavemanAdapter.ts	Caveman pruning wrapper
src/lib/db/compressionAnalytics.ts	SQLite schema for compression_analytics and query helpers
tests/unit/compression/stacked-pipeline-engines-fallback-6463.test.ts	Validates automatic stacked pipeline derivation from engines map
tests/unit/compression/rtk-caveman-stacked-pipeline.test.ts	Explicit stacked pipeline test asserting combined token saving

Summary

• Stacked mode runs RTK then Caveman sequentially, with each engine contributing additive savings
• Total token saving is stored in compression_analytics.tokens_saved as the sum of both engines' contributions
• Typical reduction: 30%–45% (150–200 tokens saved) on standard conversational payloads
• Per-engine breakdown available via compression_engine_breakdown table for optimization analysis
• Entry point: applyCompression() in strategySelector.ts handles mode selection and analytics recording

Frequently Asked Questions

How does OmniRoute calculate token saving percentage?

OmniRoute computes percentage as (tokens_saved / original_tokens) * 100 using the values stored in compression_analytics. The original_tokens count is captured before any compression, and compressed_tokens after the full engine pipeline completes. For stacked mode, this reflects the cumulative effect of both RTK and Caveman stages.

Can I use RTK or Caveman individually instead of stacked?

Yes. The engineCatalog.ts file maps both engines for independent use. Set mode to rtk or caveman to run a single engine. Stacked mode is recommended for maximum savings, but single-engine mode reduces latency when deterministic deduplication alone suffices.

Why does token saving vary between requests?

Variation stems from content characteristics: repetitive exchanges (high RTK savings), lengthy older messages (high Caveman savings), or minimal history (low overall savings). The test suite shows 150–180 tokens saved, but production workloads with verbose logs or long conversational threads may exceed this range.

Where is the compression analytics data stored?

All metrics persist to SQLite via src/lib/db/compressionAnalytics.ts. The compression_analytics table captures request-level totals, while compression_engine_breakdown stores per-engine contributions. Both tables are queryable through standard SQL or the TypeScript helpers provided in the same file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}