How the RTK Compression Engine Detects and Filters Command Output in OmniRoute

The RTK compression engine identifies command output using regex-based heuristics in detectCommandClass(), then applies catalog-driven filters—such as strip, deduplicate, and smartTruncate—to condense terminal logs while preserving semantic content for LLM reasoning.

The RTK (Rule-Based Terminal Knowledge) compression engine is a core component of OmniRoute’s prompt-compression pipeline, specifically designed to process raw terminal output from tools like kubectl, docker build, and gradle. According to the OmniRoute source code, this engine implements a two-stage architecture that first classifies command output into specific categories, then applies targeted filter rules to reduce token count without losing critical context.

Detection and Filtering Pipeline

The RTK engine processes tool and assistant messages through a structured five-step workflow defined in open-sse/services/compression/rtkEngine.ts:

1. Command Detection – The detectCommandClass() function scans text for recognizable command-line patterns, classifying output into categories like kubectl or docker-build.
2. Catalog Lookup – The engine loads matching filter definitions from DATA_DIR/rtk/filters.json via loadBuiltinFilters().
3. Rule Application – The applyFilters() function executes regex-driven transformations including line deduplication, timestamp stripping, and smart truncation.
4. Grouping – Optional merging of related command blocks when enableGrouping is activated.
5. Result Emission – The applyRtkCompression() function returns the compressed payload and optionally preserves a pointer to the original block.

How Detection Works

Detection occurs through a multi-layer heuristic system implemented in detectCommandClass() within open-sse/services/compression/rtkEngine.ts.

Line-by-Line Pattern Matching

The detector analyzes each line for three specific indicators:

• Prompt Prefixes – Characters like $, >>>, or # that indicate shell input lines.
• File Path Patterns – Absolute paths such as /usr/... or C:\... commonly appearing in tool output.
• Command Markers – Known tool names embedded in the filter catalog (e.g., "kubectl").

Regex-Based Classification

Each entry in the filter catalog defines a detect regex. The engine evaluates these patterns sequentially against the input text, and the first successful match determines the command class. If no patterns match, the engine falls back to an "unknown" classification and applies only minimal safe filters like basic deduplication.

This logic is unit-tested in tests/unit/compression/rtk-command-detector.test.ts, which validates detection accuracy across various terminal output formats.

How Filtering Works

Once classified, the engine applies rules from the built-in catalog using helper functions typically referenced in open-sse/services/compression/rtkRules.ts. The core filter operations include:

• strip – Removes lines matching specific regex patterns, such as timestamps, progress bars, or ANSI color codes.
• deduplicate – Collapses consecutive identical lines or patterns using the deduplicateLines() helper to eliminate repetitive log noise.
• smartTruncate – Retains the first N and last M lines of oversized outputs, discarding the middle section when blocks exceed configured token limits.
• grouping – Merges related command blocks before truncation when enableGrouping is true, controlled by the groupingThreshold parameter.

The applyFilters() function orchestrates these transformations by iterating over the selected filter's rule set and invoking the appropriate helper for each action.

Configuration and Customization

The RTK engine behavior is governed by a Zod schema defined in open-sse/services/compression/types.ts:

{
  enabled: boolean,
  intensity: number,          // 0-10 scale controlling aggressiveness
  enableGrouping: boolean,
  groupingThreshold: number,
  filters: string[],          // whitelist of filter names
  // additional flags like stripCodeComments, preserveDocstrings
}

Adding Custom Filters

Developers can extend the engine with project-specific filters using registerCustomRtkFilter():

import { registerCustomRtkFilter } from '@/open-sse/services/compression/rtkEngine';

await registerCustomRtkFilter('stripAnsi', {
  detect: /\x1b\[[0-9;]*m/g,
  strip: true,
  deduplicate: false,
});

Security Safeguards

Untrusted custom filters undergo ReDoS (Regular Expression Denial of Service) validation through the test suite in tests/unit/compression/rtk-filter-redos-guard.test.ts, ensuring that user-supplied regex patterns cannot cause catastrophic backtracking.

Practical Usage Examples

Compressing a Message Payload

import { applyRtkCompression } from '@/open-sse/services/compression/rtkEngine';
import { DEFAULT_RTK_CONFIG } from '@/open-sse/services/compression/types';

const compressed = await applyRtkCompletion(msg, DEFAULT_RTK_CONFIG);
console.log('Compressed token count:', compressed.tokens);

Enabling Grouping for Build Logs

const config = { 
  ...DEFAULT_RTK_CONFIG, 
  enableGrouping: true, 
  groupingThreshold: 5 
};
await applyRtkCompression(buildLogMessage, config);

Summary

• The RTK compression engine uses detectCommandClass() in rtkEngine.ts to classify terminal output via regex heuristics and command markers.
• Filter rules are loaded from DATA_DIR/rtk/filters.json and applied through applyFilters(), supporting operations like strip, deduplicate, and smartTruncate.
• Configuration is type-safe via Zod schema in types.ts, offering tunable intensity (0-10) and optional enableGrouping.
• Custom filters can be registered via registerCustomRtkFilter() but are guarded against ReDoS attacks through dedicated test coverage.
• The engine preserves semantic meaning while reducing token count, making it suitable for preparing verbose command output for LLM consumption.

Frequently Asked Questions

How does the RTK compression engine detect command output?

The engine's detectCommandClass() function scans text line-by-line for shell prompt prefixes ($, #), file paths, and known command markers defined in the filter catalog. It applies regex patterns sequentially until finding a match, classifying the output into specific command classes like kubectl or docker-build. If no match occurs, it defaults to a minimal "unknown" classification.

What filter rules are available in the RTK engine?

The engine supports four primary rule types implemented in the rules module: strip (regex-based line removal), deduplicate (collapsing consecutive identical lines), smartTruncate (middle-section removal for long outputs), and grouping (merging related command blocks). Each rule is configured per command class in the JSON filter catalog.

Can I add custom filters to the RTK engine?

Yes. Developers can call registerCustomRtkFilter() from rtkEngine.ts to inject project-specific regex patterns and rule configurations. All custom filters must pass ReDoS safety checks validated in rtk-filter-redos-guard.test.ts to prevent regex-based denial of service attacks.

What does the intensity parameter control?

The intensity parameter (0-10) defined in the Zod config schema controls how aggressively the engine removes lines during filtering. Higher values increase the threshold for smartTruncate and expand the scope of strip operations, while lower values preserve more original content. This setting allows fine-tuning between token economy and output completeness.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}