# How RTK + Caveman Stacked Compression Saves Tokens in Tool-Heavy Sessions

> Discover how OmniRoute's RTK + Caveman stacked compression slashes token counts by over 90% in tool-heavy sessions. Learn how this pipeline preserves vital diagnostic data.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-20

---

**OmniRoute's stacked compression pipeline chains RTK (fast rule-based filtering) with Caveman (model-aware semantic compression) to reduce token counts by over 90% in tool-heavy sessions while preserving critical diagnostic information.**

The `diegosouzapw/OmniRoute` repository implements a two-stage compression system specifically designed for LLM interactions that generate large, repetitive tool outputs. When `compression.mode` is set to `"stacked"`, the pipeline automatically routes messages through RTK followed by Caveman, stripping both syntactic and semantic redundancy before the payload reaches the model.

---

## What Is Stacked Compression in OmniRoute?

Stacked compression is a **pipeline architecture** that processes messages through multiple specialized engines in sequence. In [`open-sse/services/compression/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/types.ts), the `CompressionConfig` interface defines `mode: "stacked"` as the trigger for this multi-stage behavior.

Unlike single-engine compression, stacking allows each engine to target a distinct class of redundancy. RTK handles deterministic, pattern-based reduction. Caveman then operates on the pre-cleaned output to apply model-aware scoring and relevance-based pruning.

This division of labor matters because tool-heavy sessions—build logs, file reads, CLI output—produce payloads with **two distinct problems**: repetitive formatting noise and low-information semantic content. A single-pass compressor must either miss one or over-aggressively damage the other.

---

## Stage 1: RTK Rapid Tokenizer Engine

RTK ([`open-sse/services/compression/engines/rtkEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtkEngine.ts)) is the **fast, deterministic first pass**. It applies a catalog of filters that require no model inference and complete in milliseconds.

### Key RTK Operations

- **`maxLinesPerResult`** and **`maxCharsPerResult`**: Hard truncates oversized `tool_result` blocks at configurable boundaries
- **`deduplicate: true`**: Removes identical consecutive lines, eliminating repetitive log output
- **Comment stripping**: Removes code comments from tool-generated snippets
- **Force-preserve rules**: Protects safety-critical tokens including URLs, API keys, and specific regex patterns

The engine's output is a trimmed payload that retains semantic content while discarding structural noise. Tests in [`tests/unit/compression/rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/rtk-smart-truncate.test.ts) verify this behavior across **RTK smart-truncate**, **RTK deduplication**, and **RTK filter catalog** scenarios.

---

## Stage 2: Caveman Model-Aware Compression

Caveman receives RTK's output through [`open-sse/services/compression/engines/cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/cavemanAdapter.ts). This second stage performs **deeper, context-sensitive reduction** that requires understanding token information value.

### Caveman Compression Strategies

- **Relevance scoring**: Weighs each token against session context and model-specific importance patterns
- **Session-dedup**: Eliminates semantic duplicates even when phrasing differs
- **Headroom trimming**: Aggressive reduction when approaching token budget limits
- **Hard-budget guard**: Prevents over-truncation that would break output coherence
- **Fidelity guard**: Validates that final output satisfies target token constraints

The `intensity` configuration (set to `"high"` in demanding scenarios) directly influences scoring thresholds. High-value tokens—URLs, numeric identifiers, preserved docstrings—receive protected status regardless of intensity settings.

---

## Combined Token Savings: The 90% Reduction Mechanism

When RTK and Caveman execute in sequence, each removes a **non-overlapping category of redundancy**:

| Stage | Redundancy Type | Typical Reduction | Mechanism |
|-------|-----------------|-------------------|-----------|
| RTK | Syntactic repetition (duplicate lines, large unstructured blobs) | 40-60% | Deterministic filtering |
| Caveman | Semantic redundancy (low-scoring tokens, contextual irrelevance) | 30-50% of remaining | Model-aware scoring |

The multiplicative effect frequently exceeds **90% total token savings** on deliberately noisy `tool_result` blocks. This is demonstrated in [`tests/unit/compression/stacked-compression-tool-result-savings.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/stacked-compression-tool-result-savings.test.ts), which validates the pipeline against realistic tool-heavy payloads that would otherwise overwhelm standard LLM context windows.

---

## Why Tool-Heavy Sessions Require Stacked Compression

Tool-heavy interactions generate three specific problems that stacked compression solves:

1. **Volume**: Build logs and file listings quickly exceed 100K+ tokens
2. **Repetition**: CLI output contains massive structural duplication
3. **Noise density**: High token count with low information-per-token ratio

By applying RTK first, OmniRoute **stays within provider token budgets** without truncation errors. Caveman then ensures the LLM receives the **maximum-information subset** of available output. The pipeline remains deterministic for safety-critical tokens through operator-defined preservation rules, satisfying security compliance requirements.

---

## Enabling and Configuring Stacked Compression

### Basic Activation

```typescript
// Enable stacked compression via request payload
await fetch("/api/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    messages: [
      { role: "user", content: "Run `git status` and show the result." },
    ],
    compression: { mode: "stacked" },
  }),
});

```

### Fine-Grained Configuration

```typescript
// Persistent configuration for both engines
const DEFAULT_RTK_CONFIG = {
  enabled: true,
  maxLinesPerResult: 120,
  deduplicate: true,
};

const DEFAULT_CAVEMAN_CONFIG = {
  enabled: true,
  intensity: "high",
  preserveDocstrings: true,
};

await fetch("/api/v1/compression/config", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    rtkConfig: DEFAULT_RTK_CONFIG,
    cavemanConfig: DEFAULT_CAVEMAN_CONFIG,
  }),
});

```

The configuration service persists settings across requests, while per-request `compression.mode` overrides allow session-specific pipeline selection.

---

## Core Implementation Files

| Component | Path | Function |
|-----------|------|----------|
| Mode resolution | [`open-sse/services/compression/resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/resolveCompressionPlan.ts) | Selects between `off`, `rtk`, `codex-responses`, `stacked` |
| Pipeline orchestration | [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) | Builds ordered engine list (`rtk → caveman`) |
| RTK engine | [`open-sse/services/compression/engines/rtkEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtkEngine.ts) | Line-filtering, deduplication, truncation |
| Caveman adapter | [`open-sse/services/compression/engines/cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/cavemanAdapter.ts) | Stacked-stage wrapper for Caveman |
| Staged execution | [`open-sse/services/compression/stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stackedStepCore.ts) | Progress tracking, bail-out guards, telemetry |
| Savings validation | [`tests/unit/compression/stacked-compression-tool-result-savings.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/stacked-compression-tool-result-savings.test.ts) | >90% reduction verification |
| RTK correctness | [`tests/unit/compression/rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/rtk-smart-truncate.test.ts) | Filter and dedup validation |
| Scoring logic | [`tests/unit/compression/heatmap-build.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/compression/heatmap-build.test.ts) | Token score generation testing |

---

## Summary

- **RTK + Caveman stacked compression** in OmniRoute achieves 90%+ token reduction by chaining deterministic filtering with model-aware semantic pruning
- RTK ([`rtkEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtkEngine.ts)) handles syntactic noise: line limits, deduplication, comment stripping, with force-preserve rules for safety tokens
- Caveman ([`cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanAdapter.ts)) scores remaining tokens by relevance, applying intensity-configurable thresholds and budget guards
- The `compression.mode: "stacked"` trigger activates this pipeline through [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) and [`stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stackedStepCore.ts)
- Tool-heavy sessions benefit most: build logs, CLI output, and file reads stay within context windows without losing diagnostic value
- Full test coverage in [`stacked-compression-tool-result-savings.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stacked-compression-tool-result-savings.test.ts) validates real-world savings

---

## Frequently Asked Questions

### What triggers the stacked compression pipeline?

The pipeline activates when a request includes `compression: { mode: "stacked" }`. [`resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resolveCompressionPlan.ts) parses this configuration and [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) constructs the ordered engine list. No server-side configuration changes are required—clients opt-in per-request or persist preferences via the `/api/v1/compression/config` endpoint.

### How does RTK protect important tokens while still saving space?

RTK implements **force-preserve rules**—regex patterns that mark URLs, API keys, and operator-defined critical strings as non-truncatable. These rules run before any filtering or deduplication. The engine also respects `maxLinesPerResult` boundaries that truncate aggressively but predictably, avoiding mid-token splits that could corrupt preserved content.

### When should I use stacked versus single-engine compression?

Use **stacked** for tool-heavy sessions with large, repetitive outputs (builds, logs, bulk file operations) where both volume and noise density are high. Use **RTK-only** for simple truncation needs where speed matters more than maximum compression. Use **Caveman-only** when inputs are already clean but require semantic relevance scoring. The `codex-responses` mode provides an alternative pipeline optimized for OpenAI-style streaming responses.

### Can I adjust the 90% savings target?

Yes. Tune `maxLinesPerResult` and `maxCharsPerResult` in RTK config to control first-pass aggression. Adjust Caveman's `intensity` (`"low"`, `"medium"`, `"high"`) to shift scoring thresholds. The hard-budget and fidelity guards in [`stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stackedStepCore.ts) ensure these adjustments don't produce incoherent output—they trigger early termination or fallback to less aggressive modes rather than violate safety constraints.