How to Configure the RTK Compression Engine with Custom Rules and Filter Packs

Configure custom RTK compression in OmniRoute by creating signed filter packs in .rtk/filters.json and adjusting engine intensity via the /api/context/rtk/config API.

The RTK (Real-Time Kontext) compression engine is OmniRoute's command-aware compression stage that runs before provider translation, intelligently reducing token volume based on tool output patterns. This guide shows you how to create custom filter packs, establish trust for project-level filters, and tune engine behavior using the official configuration API.

Understanding RTK Filter Sources and Loading Order

RTK discovers filter definitions from three hierarchically ordered sources. The engine applies the first matching filter based on this priority: project → global → builtin.

Source	Location	Trust Requirements	Load Function
Project	<cwd>/.rtk/filters.{json,toml}	Signed with SHA-256 hash in .rtk/trust.json or OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1	collectProjectFilterSources() in open-sse/services/compression/engines/rtk/filterLoader.ts:28-40
Global	$DATA_DIR/rtk/filters.{json,toml} (default ~/.omniroute/rtk)	Implicitly trusted	collectGlobalFilterSources() in filterLoader.ts:53-63
Builtin	open-sse/services/compression/engines/rtk/filters/	Trusted (shipped with binary)	collectBuiltinFilterSources() in filterLoader.ts:65-89

The loader caches the priority-sorted filter list in memory. When a request reaches matchRtkFilter() (lines 300-331 of open-sse/services/compression/engines/rtk/index.ts), the engine selects the highest-priority match and applies its rule set through applyRtkCompression (implemented in src/app/api/context/rtk/test/route.ts).

Filter Pack Schema and Validation Rules

RTK accepts two formats: a modern JSON DSL and a legacy RTK-AI format. Both are validated with Zod in open-sse/services/compression/engines/rtk/filterSchema.ts.

Core Schema Fields

Field	Description	Schema Location
id, label, description	Human-readable identifiers	filterSchema.ts:74-86
category	Command type: git, test, build, shell, docker, package, infra, cloud, generic	filterSchema.ts:4-13
match.commands / match.patterns	Command triggers or regex patterns	filterSchema.ts:41-44
rules.replace	Pattern-to-replacement transforms	filterSchema.ts:15-19
rules.matchOutput	Regex-based error/message mapping	filterSchema.ts:22-26
rules.deduplicate	Enable line-level deduplication	filterSchema.ts:55
tests	Inline test vectors for CI validation (tests/unit/rtk-*.test.ts)	filterSchema.ts:30-36

The loader includes ReDoS protection via isReDoSProne (lines 161-163), which rejects patterns containing nested quantifiers.

RTK Engine Configuration Options

Beyond filter catalogs, control the engine through the RTK config stored in the settings table (src/lib/db/compression.ts). The public schema is declared in open-sse/services/compression/engines/rtk/configSchema.ts and exposed via REST endpoints /api/context/rtk/config.

Configurable Engine Parameters

// open-sse/services/compression/engines/rtk/configSchema.ts
export const RTK_SCHEMA: EngineConfigField[] = [
  { key: "intensity", type: "select", options: ["minimal","standard","aggressive"], defaultValue: DEFAULT_RTK_CONFIG.intensity },
  { key: "applyToToolResults", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToToolResults },
  { key: "applyToAssistantMessages", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToAssistantMessages },
  { key: "applyToCodeBlocks", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToCodeBlocks },
  { key: "maxLinesPerResult", type: "number", min: 0, max: 5000, defaultValue: DEFAULT_RTK_CONFIG.maxLinesPerResult },
  { key: "maxCharsPerResult", type: "number", min: 0, max: 500000, defaultValue: DEFAULT_RTK_CONFIG.maxCharsPerResult },
  { key: "deduplicateThreshold", type: "number", min: 2, max: 100, defaultValue: DEFAULT_RTK_CONFIG.deduplicateThreshold },
  { key: "rawOutputRetention", type: "select", options: ["never","failures","always"], defaultValue: DEFAULT_RTK_CONFIG.rawOutputRetention },
  { key: "enableRenderers", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.enableRenderers },
];

These fields are validated by validateRtkEngineConfig() (lines 77-88). The intensity setting controls heuristic aggressiveness for line truncation and deduplication.

Step-by-Step: Creating and Deploying Custom Filter Packs

Step 1: Create a Project Filter Pack

Create .rtk/filters.json in your project root:

[
  {
    "id": "gradle-noop",
    "label": "Gradle No-Op Lines",
    "description": "Drop repetitive 'UP-TO-DATE' lines from Gradle builds",
    "category": "build",
    "priority": 80,
    "match": {
      "commands": ["gradle", "gradlew"],
      "patterns": []
    },
    "rules": {
      "dropPatterns": ["> Task .* UP-TO-DATE", "> Task .* SKIPPED"],
      "keepPatterns": ["> Task .* SUCCESSFUL", "BUILD SUCCESSFUL", "FAILURE"]
    },
    "tests": [
      {
        "name": "gradle-noop-sample",
        "input": "> Task :compileJava UP-TO-DATE\\n> Task :processResources UP-TO-DATE\\nBUILD SUCCESSFUL",
        "expected": "BUILD SUCCESSFUL"
      }
    ]
  }
]

Step 2: Sign the Filter Pack (Recommended)

Generate a trust file to satisfy project filter verification:

shasum -a 256 .rtk/filters.json | awk '{print $1}'

Save to .rtk/trust.json:

{
  "filtersSha256": "8f2c4e7d9a3b5a6c1e2d3f4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d"
}

For local development, bypass signing with:

export OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1

Step 3: Test the Filter Pack

Verify compression behavior before deployment:

curl -X POST https://your.omniroute.instance/api/context/rtk/test \
  -H "Content-Type: application/json" \
  -d '{
    "text":"> Task :compileJava UP-TO-DATE\n> Task :processResources UP-TO-DATE\nBUILD SUCCESSFUL",
    "command":"gradle"
  }' | jq .

Expected response:

{
  "compressedText": "BUILD SUCCESSFUL",
  "techniquesUsed": ["rtk-drop"],
  "filterId": "gradle-noop"
}

Step 4: Configure Engine Intensity

Adjust RTK behavior via the configuration API:

curl -X PUT https://your.omniroute.instance/api/context/rtk/config \
  -H "Content-Type: application/json" \
  -d '{"intensity":"aggressive","applyToToolResults":true}'

Step 5: Monitor Active Filters

List the merged filter catalog with diagnostics:

curl https://your.omniroute.instance/api/context/rtk/filters | jq .

Key Implementation Files

File	Purpose
open-sse/services/compression/engines/rtk/filterLoader.ts	Filter discovery, validation, and caching across all three sources
open-sse/services/compression/engines/rtk/filterSchema.ts	Zod schemas for filter packs with ReDoS protection
open-sse/services/compression/engines/rtk/configSchema.ts	Engine configuration field definitions
src/shared/validation/compressionConfigSchemas.ts	Top-level schema merging RTK with generic compression settings
src/lib/db/compression.ts	SQLite persistence and default application
src/app/api/context/rtk/config/route.ts	GET/PUT endpoint for engine configuration
src/app/api/context/rtk/filters/route.ts	GET endpoint for active filter catalog
src/app/api/context/rtk/test/route.ts	POST endpoint for filter preview testing

Summary

• Configure the RTK compression engine by placing signed filter packs in .rtk/filters.json (or .toml) with optional trust.json for verification
• Three filter sources are prioritized: project (signed), global (~/.omniroute/rtk), and builtin (shipped with OmniRoute)
• Engine intensity and payload targeting are controlled via PUT /api/context/rtk/config, with settings persisted in SQLite
• Test filters interactively using POST /api/context/rtk/test before deploying to production
• All schemas are Zod-validated, with built-in ReDoS protection for regex patterns

Frequently Asked Questions

How do I force OmniRoute to reload my custom filter pack?

Filters reload automatically on the next request. To force immediate reloading and view diagnostics, call GET /api/context/rtk/filters — this refreshes the cache and returns validation warnings if your pack has schema errors, according to filterLoader.ts:43-48.

What happens if my project filter fails the SHA-256 trust check?

The loader skips the project source entirely and falls back to global or builtin filters. Set OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1 to bypass verification during development, or generate the correct hash using shasum -a 256 and store it in .rtk/trust.json for production.

Can I use TOML instead of JSON for filter packs?

Yes. The loader accepts both formats: filters.json or filters.toml. The parser auto-detects the format based on file extension in collectProjectFilterSources() and collectGlobalFilterSources().

What's the difference between intensity levels in RTK configuration?

The intensity setting scales internal heuristics: minimal preserves almost all output with basic deduplication; standard applies moderate line truncation and pattern filtering; aggressive maximizes compression through stricter thresholds, early termination, and deeper deduplication as implemented in the engine's applyRtkCompression pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}