# How RTK and Caveman Compression Reduce Token Usage in OmniRoute

> Discover how RTK and Caveman compression in OmniRoute slash token usage by 78%-95%. Learn how this pipeline optimizes LLM input for significant cost and performance gains.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: performance
- Published: 2026-08-11

---

**RTK and Caveman compression in OmniRoute work as a stacked pipeline—RTK applies command-aware filtering first (≈80% token reduction), then Caveman applies prose-aware compression (additional ≈46% reduction)—achieving 78%–95% overall token savings before text reaches the LLM.**

OmniRoute's dual-engine compression system targets the single largest cost driver in LLM-powered workflows: token volume. By chaining two specialized engines—RTK for machine-generated output and Caveman for narrative condensation—the system multiplicatively shrinks payloads while preserving critical semantic information. This article explains the architecture, configuration, and measurable impact of how RTK and Caveman compression reduce token usage in production deployments.

## RTK Compression: Command-Aware First Pass

The **RTK (Run-Time-Kit)** engine handles the initial heavy lifting on noisy, structured output from developer tools.

### Architecture and Pipeline

Located at [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts), the RTK engine implements a seven-stage pipeline:

1. **Command detection** — [`commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/commandDetector.ts) classifies incoming content by tool type (git, npm, Docker, test runners, etc.)
2. **Filter loading** — Priority-ordered filter resolution: project-level (`.rtk/filters.*`), then global (`DATA_DIR/rtk/filters.*`), then built-in defaults in `open-sse/services/compression/engines/rtk/filters/`
3. **DSL rule application** — Filters execute ANSI stripping, pattern replacement, match-output extraction, include/drop logic, and truncation per [`filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/filterSchema.ts)
4. **Per-filter deduplication** — `rules.deduplicate` collapses repeated lines within individual filter scopes
5. **Engine-wide deduplication** — Final pass with `deduplicateThreshold` (default 3 identical consecutive lines)
6. **Optional grouping** — `groupSimilarLines` collapses near-identical adjacent lines when `rtkConfig.enableGrouping` is enabled
7. **Intensity-based truncation** — Head/tail limits vary by intensity level

### Intensity Configuration

| Level | Head/Tail Lines | Use Case |
|-------|-----------------|----------|
| `minimal` | 24 lines | Maximum fidelity, audit scenarios |
| `standard` (default) | 20 lines | Balanced compression |
| `aggressive` | 16 lines | Cost-critical, high-volume pipelines |

RTK never transmits raw output to downstream LLMs unless `rawOutputRetention` is explicitly configured for debugging.

### Measured Savings

Per the RTK benchmark documentation in [`docs/compression/RTK_COMPRESSION.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/compression/RTK_COMPRESSION.md), the engine achieves **60%–90% token reduction** depending on input verbosity. OmniRoute adopts **≈80%** as the operational average.

## Caveman Compression: Prose-Aware Second Pass

The **Caveman** engine receives RTK-compressed output and applies heuristic-based narrative compression.

### Core Implementation

The engine entry point at [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) orchestrates rules defined in [`open-sse/services/compression/cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/cavemanRules.ts):

- **Line deduplication** — Tuned threshold for prose rather than structured logs
- **Hedging removal** — Strips filler phrases ("as far as I can tell", "it seems that")
- **Structural pruning** — Collapses repetitive stack traces, long log sequences
- **Error preservation** — Guarantees survival of stack traces, assertion failures, and critical error metadata

### Engine-Wide Final Pass

After per-rule application, Caveman runs identical deduplication to RTK: collapsing runs of ≥3 identical consecutive lines. This ensures consistency across the stacked pipeline.

### Incremental Savings

Caveman contributes **≈46% additional reduction** on the already-RTK-compressed payload. The multiplicative effect is what drives total savings toward the upper bound.

## Stacked Compression: Calculating Total Token Reduction

When engines operate in sequence (`rtk → caveman`), savings compound rather than add:

```

Stacked saving = 1 − (1 − RTK_save) × (1 − Caveman_save)
               = 1 − (1 − 0.80) × (1 − 0.46)
               ≈ 89% overall token reduction

```

Accounting for RTK's reported variability (60%–90%), the full system achieves **78%–95% total token reduction** before LLM ingestion.

## Practical Configuration Examples

### Preview Stacked Compression Pipeline

```json
POST /api/compression/preview
{
  "mode": "stacked",
  "messages": [
    {
      "role": "tool",
      "content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed (1 total)\nTests 1 failed (2 total)\nDuration 1.23s"
    }
  ],
  "config": {
    "rtkConfig": { "intensity": "standard" },
    "cavemanConfig": { "intensity": "full" }
  }
}

```

The response includes `CompressionStats` with `rtkSavedPct` and `cavemanSavedPct` fields, plus the compressed payload typically measuring ~10% of original token count.

### Single-Engine RTK Endpoint

```json
POST /api/context/rtk/test
{
  "command": "npm test",
  "text": "FAIL tests/example.test.ts\nAssertionError: expected true\n    at Object.<anonymous> (/project/tests/example.test.ts:15:15)",
  "config": { "intensity": "aggressive" }
}

```

Returns compressed text (~20% of original) and per-engine statistics.

### Enable Raw Output Retention

```json
PUT /api/context/rtk/config
{
  "rawOutputRetention": "always",
  "maxCharsPerResult": 12000
}

```

Raw originals are stored to `DATA_DIR/rtk/raw-output/` and retrievable via:

```bash
GET /api/context/rtk/raw-output/<request_id>

```

### Programmatic Intensity Adjustment

```typescript
import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";

// Maximum compression for cost optimization
updateEngineConfig("rtk", { intensity: "aggressive" });
updateEngineConfig("caveman", { intensity: "full" });

```

This pushes total reduction toward the ~95% upper bound by truncating more aggressively (16 lines/section) and applying full Caveman heuristics.

## Key Source Files and Components

| Component | Path | Purpose |
|-----------|------|---------|
| RTK documentation | [`docs/compression/RTK_COMPRESSION.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/compression/RTK_COMPRESSION.md) | Savings methodology, intensity definitions |
| RTK engine | [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts) | Core pipeline implementation |
| Command detector | [`open-sse/services/compression/engines/rtk/commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/commandDetector.ts) | Tool output classification |
| Filter schema | [`open-sse/services/compression/engines/rtk/filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/filterSchema.ts) | Zod validation for filter DSL |
| Filters directory | `open-sse/services/compression/engines/rtk/filters/` | Built-in filter definitions |
| Caveman engine | [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) | Second-stage compression orchestration |
| Caveman rules | [`open-sse/services/compression/cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/cavemanRules.ts) | Heuristic rule definitions |
| Compression API | [`src/app/api/v1/compression/preview/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/compression/preview/route.ts) | HTTP endpoint for preview/stacked modes |
| Configuration storage | [`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts) | SQLite `key_value` persistence for settings |
| Regression tests | [`tests/golden-set/compression-caveman-v2.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/golden-set/compression-caveman-v2.test.ts) | Validation of advertised savings percentages |

## Summary

- **RTK compression** achieves ~80% token reduction through command detection, filter DSL rules, deduplication, and intensity-controlled truncation
- **Caveman compression** adds ~46% incremental reduction via prose heuristics, hedging removal, and error-preserving structural pruning
- **Stacked operation** produces 78%–95% total savings through multiplicative (not additive) composition
- **Configuration granularity** spans per-request, per-engine, and global persistence layers
- **Observability** includes raw output retention, preview endpoints, and statistical reporting for cost attribution

## Frequently Asked Questions

### How does OmniRoute ensure critical error information survives compression?

Both engines implement **error-preservation heuristics**. RTK's filters in [`commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/commandDetector.ts) recognize test failure patterns and apply less aggressive truncation. Caveman's rules in [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts) explicitly whitelist stack traces, assertion messages, and error metadata during structural pruning. The [`tests/golden-set/compression-caveman-v2.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/golden-set/compression-caveman-v2.test.ts) suite validates that essential diagnostic content persists through the full pipeline.

### Can I disable Caveman and use only RTK compression?

Yes. The `/api/compression/preview` endpoint accepts `"mode": "rtk"` to execute single-engine compression. However, this sacrifices the additional ~46% savings Caveman provides. For maximum cost efficiency, stacked mode is recommended unless profiling reveals Caveman overhead exceeds its benefit for your specific workload patterns.

### What is the latency cost of stacked compression?

The RTK engine operates in sub-millisecond time for typical CLI output (<10KB). Caveman adds comparable overhead for prose processing. Combined, the pipeline typically adds <5ms per request—negligible compared to LLM API round-trip times (100ms–2000ms). The token savings translate directly to reduced LLM latency and cost, yielding net performance improvement.

### How does `rawOutputRetention` impact storage costs?

When enabled, raw outputs are written to `DATA_DIR/rtk/raw-output/` with automatic rotation based on `maxCharsPerResult` (default 12,000 characters). For high-throughput systems, configure retention to `"errors"` to store only failed compressions, or implement external cleanup policies on the raw-output directory. Storage growth is predictable: uncompressed retention consumes ~5× the compressed payload size.