How to Configure RTK and Caveman Token Compression for Tool-Heavy Sessions in OmniRoute

OmniRoute reduces token costs in tool-intensive workflows by applying RTK (Rust Token Killer) and Caveman compression engines either individually or stacked together, achieving 78–95% token reduction without requiring client-side payload changes.

OmniRoute's dual-engine compression system targets the high token overhead typical of CLI-heavy AI sessions. By configuring RTK and Caveman Token Compression, developers can filter command output noise and strip boilerplate from logs before the provider translation step, significantly reducing API costs while maintaining semantic fidelity.

Understanding Compression Modes

OmniRoute offers five distinct compression strategies controlled via the x-omniroute-compression header or Compression API. For complete engine specifications, see docs/compression/COMPRESSION_ENGINES.md.

Available Engine Modes

• off: Disables compression entirely, sending raw tool output (0% savings).
• lite/standard/aggressive/ultra: Built-in heuristics for generic text compression (15–90% savings).
• RTK: Command-output aware filtering with deduplication and truncation (60–90% savings).
• Caveman: Prose-oriented rules that remove boilerplate from logs and build output (20–40% savings).
• stacked: Applies RTK first, then Caveman on the remaining payload for compounding reductions (78–95% savings).

Token Savings Calculation

According to the architecture documentation in docs/i18n/zh-TW/docs/architecture/ARCHITECTURE.md, stacked mode follows the formula 1 − (1 − RTK) × (1 − Caveman), yielding approximately 78–95% token reduction when both engines process the payload sequentially.

Enabling the Compression Engines

Compression activation occurs pre-translation, requiring no changes to downstream request payloads.

HTTP Header Configuration

Set the engine mode via the x-omniroute-compression header:

curl -H "x-omniroute-compression: stacked" \
     -H "Content-Type: application/json" \
     -d '{"text":"npm test"}' \
     /api/compression/preview

REST API Endpoints

The API surface defined in src/app/api/context/rtk/config/route.ts and src/app/api/compression/preview/route.ts exposes:

• PUT /api/context/rtk/config – Configure RTK-specific options including enableRenderers and preserveDocstrings.
• GET /api/context/rtk/filters – Retrieve built-in and custom RTK filter definitions.
• POST /api/context/rtk/test – Preview compression against sample payloads.
• POST /api/compression/preview – Test any compression mode including stacked configurations.

MCP Server Integration

For agentic workflows, the MCP server defined in open-sse/mcp-server/README.md exposes the omniroute_set_compression_engine tool, allowing agents to switch between off, caveman, rtk, and stacked modes programmatically.

Dashboard Configuration

The React-based dashboard implemented in src/app/(dashboard)/dashboard/context/rtk/CavemanContextPageClient.tsx provides a UI for toggling engines, editing filter catalogs, and viewing raw-output retention settings at the /dashboard/context/rtk route.

Configuring the RTK Engine

RTK (Rust Token Killer) operates through JSON filter definitions located in open-sse/services/compression/engines/rtk/filters/. For comprehensive filter schema documentation, refer to docs/compression/RTK_COMPRESSION.md.

Filter Architecture

Each filter specifies command patterns, JSON-path selectors, deduplication rules, and optional redaction logic. The system loads these definitions through open-sse/services/compression/engines/rtk/filterLoader.ts, which validates schemas against built-in TOML definitions.

Loading Custom Filters

Upload custom TOML schema v1 files using the CLI documented in skills/cli-compression/SKILL.md:

omniroute compression load-filter --file ./custom-filter.toml

Alternatively, apply filters via the API:

await fetch('/api/context/rtk/config', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    enableRenderers: true,
    preserveDocstrings: false,
    filters: ['git-diff', 'test-jest'],
  }),
});

Trust Settings for Project Filters

The OMNIROUTE_RTK_TRUST_PROJECT_FILTERS environment variable (default false) controls hash verification for .rtk/trust.json files. Enable this only in controlled development environments per docs/reference/ENVIRONMENT.md.

Raw Output Preservation

Set enableRenderers: true in the RTK configuration to persist original command output for debugging. This field survives DB round-trips following the persistence fix documented in the project's CHANGELOG.

Configuring the Caveman Engine

Caveman specializes in prose-oriented compression for tool outputs like Gradle, .NET CLI, and Docker logs.

Language-Specific Rule Packs

Rule packs reside under open-sse/services/compression/engines/caveman/ and can be toggled individually via the dashboard UI. Each pack targets specific boilerplate patterns common to particular build systems or logging formats.

Stacked Mode Architecture

When stacked mode is active, RTK executes first to filter command-specific noise, followed by Caveman processing on the remaining payload. The dashboard displays individual contribution percentages for each engine stage.

Recommended Setup for Tool-Heavy Sessions

For workflows invoking multiple CLI tools, test suites, or build pipelines:

1. Activate stacked mode via CLI or header to maximize token reduction.
2. Enable raw-output rendering temporarily when debugging compression artifacts.
3. Import custom filters for proprietary commands not covered by the built-in catalog using omniroute compression load-filter.
4. Verify trust settings in production environments to ensure .rtk/trust.json integrity.

Complete Configuration Example

# Set compression to stacked (RTK → Caveman)

omniroute compression set --mode stacked

# Enable raw RTK output retention for debugging

omniroute compression set --rtk-raw true

# Load a custom filter for internal CLI tools

omniroute compression load-filter --file ./internal-tools.toml

Source: CLI implementation details in skills/cli-compression/SKILL.md.

Programmatic Preview

Test compression impact before deployment using the endpoint defined in src/app/api/compression/preview/route.ts:

const response = await fetch('/api/compression/preview', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'x-omniroute-compression': 'stacked',
  },
  body: JSON.stringify({
    text: `npm run test && git diff --stat`,
  }),
});

const { originalTokens, compressedTokens, savingsPercent } = await response.json();

Summary

• RTK and Caveman Token Compression in OmniRoute operate pre-translation to reduce provider token costs by 78–95% in stacked mode.
• RTK handles command-output filtering through JSON definitions in open-sse/services/compression/engines/rtk/filters/, configurable via PUT /api/context/rtk/config.
• Caveman applies prose-oriented boilerplate removal through language packs in open-sse/services/compression/engines/caveman/.
• Stacked mode compounds savings by running RTK before Caveman, controlled via the x-omniroute-compression: stacked header or the omniroute_set_compression_engine MCP tool.
• Debug capabilities include raw-output preservation (enableRenderers: true) and the POST /api/compression/preview endpoint for testing configurations.

Frequently Asked Questions

How do I enable stacked compression mode for all requests?

Set the global compression mode using the CLI command omniroute compression set --mode stacked or configure your HTTP client to include the header x-omniroute-compression: stacked on every request. For agentic systems, invoke the omniroute_set_compression_engine MCP tool with the parameter engine: "stacked" as defined in open-sse/mcp-server/README.md.

What is the difference between RTK and Caveman compression?

RTK (Rust Token Killer) specializes in command-output aware filtering, deduplication, and truncation for structured CLI output, achieving 60–90% token reduction through the logic implemented in open-sse/services/compression/engines/rtk/filterLoader.ts. Caveman applies prose-oriented rules to strip boilerplate from logs and build output, typically saving 20–40% of tokens. When used in stacked mode, RTK processes the payload first, then Caveman operates on the remaining content for maximum efficiency approaching 95% savings.

How do I add support for a custom internal CLI tool?

Create a TOML schema v1 file defining command patterns and JSON-path selectors for your tool's output. Load it using omniroute compression load-filter --file ./your-filter.toml as documented in skills/cli-compression/SKILL.md, or upload it via the PUT /api/context/rtk/config endpoint. Ensure OMNIROUTE_RTK_TRUST_PROJECT_FILTERS is set to true only in controlled environments if your filter requires trust verification.

Can I preview compression results before applying them?

Yes. Send a POST request to /api/compression/preview (implemented in src/app/api/compression/preview/route.ts) with your sample payload and desired compression mode in the x-omniroute-compression header. The endpoint returns token counts for both original and compressed text, allowing you to calculate exact savings before enabling compression in production sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}