# How to Configure the RTK Compression Engine with Custom Rules and Filter Packs

> Learn to configure RTK compression in OmniRoute. Create signed filter packs in filters.json and adjust engine intensity using the API for custom optimization.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Configure custom RTK compression in OmniRoute by creating signed filter packs in [`.rtk/filters.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/filters.json) and adjusting engine intensity via the `/api/context/rtk/config` API.**

The **RTK (Real-Time Kontext) compression engine** is OmniRoute's command-aware compression stage that runs before provider translation, intelligently reducing token volume based on tool output patterns. This guide shows you how to create custom **filter packs**, establish trust for project-level filters, and tune engine behavior using the official configuration API.

## Understanding RTK Filter Sources and Loading Order

RTK discovers filter definitions from three hierarchically ordered sources. The engine applies the **first matching filter** based on this priority: **project → global → builtin**.

| Source | Location | Trust Requirements | Load Function |
|--------|----------|-------------------|---------------|
| **Project** | `<cwd>/.rtk/filters.{json,toml}` | Signed with SHA-256 hash in [`.rtk/trust.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/trust.json) or `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` | `collectProjectFilterSources()` in `open-sse/services/compression/engines/rtk/filterLoader.ts:28-40` |
| **Global** | `$DATA_DIR/rtk/filters.{json,toml}` (default `~/.omniroute/rtk`) | Implicitly trusted | `collectGlobalFilterSources()` in `filterLoader.ts:53-63` |
| **Builtin** | `open-sse/services/compression/engines/rtk/filters/` | Trusted (shipped with binary) | `collectBuiltinFilterSources()` in `filterLoader.ts:65-89` |

The loader caches the priority-sorted filter list in memory. When a request reaches `matchRtkFilter()` (lines 300-331 of [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts)), the engine selects the highest-priority match and applies its rule set through `applyRtkCompression` (implemented in [`src/app/api/context/rtk/test/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/context/rtk/test/route.ts)).

## Filter Pack Schema and Validation Rules

RTK accepts two formats: a modern JSON DSL and a legacy RTK-AI format. Both are validated with **Zod** in [`open-sse/services/compression/engines/rtk/filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/filterSchema.ts).

### Core Schema Fields

| Field | Description | Schema Location |
|-------|-------------|---------------|
| `id`, `label`, `description` | Human-readable identifiers | `filterSchema.ts:74-86` |
| `category` | Command type: `git`, `test`, `build`, `shell`, `docker`, `package`, `infra`, `cloud`, `generic` | `filterSchema.ts:4-13` |
| `match.commands` / `match.patterns` | Command triggers or regex patterns | `filterSchema.ts:41-44` |
| `rules.replace` | Pattern-to-replacement transforms | `filterSchema.ts:15-19` |
| `rules.matchOutput` | Regex-based error/message mapping | `filterSchema.ts:22-26` |
| `rules.deduplicate` | Enable line-level deduplication | `filterSchema.ts:55` |
| `tests` | Inline test vectors for CI validation (`tests/unit/rtk-*.test.ts`) | `filterSchema.ts:30-36` |

The loader includes **ReDoS protection** via `isReDoSProne` (lines 161-163), which rejects patterns containing nested quantifiers.

## RTK Engine Configuration Options

Beyond filter catalogs, control the engine through the **RTK config** stored in the settings table ([`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts)). The public schema is declared in [`open-sse/services/compression/engines/rtk/configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/configSchema.ts) and exposed via REST endpoints `/api/context/rtk/config`.

### Configurable Engine Parameters

```typescript
// open-sse/services/compression/engines/rtk/configSchema.ts
export const RTK_SCHEMA: EngineConfigField[] = [
  { key: "intensity", type: "select", options: ["minimal","standard","aggressive"], defaultValue: DEFAULT_RTK_CONFIG.intensity },
  { key: "applyToToolResults", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToToolResults },
  { key: "applyToAssistantMessages", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToAssistantMessages },
  { key: "applyToCodeBlocks", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.applyToCodeBlocks },
  { key: "maxLinesPerResult", type: "number", min: 0, max: 5000, defaultValue: DEFAULT_RTK_CONFIG.maxLinesPerResult },
  { key: "maxCharsPerResult", type: "number", min: 0, max: 500000, defaultValue: DEFAULT_RTK_CONFIG.maxCharsPerResult },
  { key: "deduplicateThreshold", type: "number", min: 2, max: 100, defaultValue: DEFAULT_RTK_CONFIG.deduplicateThreshold },
  { key: "rawOutputRetention", type: "select", options: ["never","failures","always"], defaultValue: DEFAULT_RTK_CONFIG.rawOutputRetention },
  { key: "enableRenderers", type: "boolean", defaultValue: DEFAULT_RTK_CONFIG.enableRenderers },
];

```

These fields are validated by `validateRtkEngineConfig()` (lines 77-88). The **`intensity`** setting controls heuristic aggressiveness for line truncation and deduplication.

## Step-by-Step: Creating and Deploying Custom Filter Packs

### Step 1: Create a Project Filter Pack

Create [`.rtk/filters.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/filters.json) in your project root:

```json
[
  {
    "id": "gradle-noop",
    "label": "Gradle No-Op Lines",
    "description": "Drop repetitive 'UP-TO-DATE' lines from Gradle builds",
    "category": "build",
    "priority": 80,
    "match": {
      "commands": ["gradle", "gradlew"],
      "patterns": []
    },
    "rules": {
      "dropPatterns": ["> Task .* UP-TO-DATE", "> Task .* SKIPPED"],
      "keepPatterns": ["> Task .* SUCCESSFUL", "BUILD SUCCESSFUL", "FAILURE"]
    },
    "tests": [
      {
        "name": "gradle-noop-sample",
        "input": "> Task :compileJava UP-TO-DATE\\n> Task :processResources UP-TO-DATE\\nBUILD SUCCESSFUL",
        "expected": "BUILD SUCCESSFUL"
      }
    ]
  }
]

```

### Step 2: Sign the Filter Pack (Recommended)

Generate a trust file to satisfy project filter verification:

```bash
shasum -a 256 .rtk/filters.json | awk '{print $1}'

```

Save to [`.rtk/trust.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/trust.json):

```json
{
  "filtersSha256": "8f2c4e7d9a3b5a6c1e2d3f4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d"
}

```

For local development, bypass signing with:

```bash
export OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1

```

### Step 3: Test the Filter Pack

Verify compression behavior before deployment:

```bash
curl -X POST https://your.omniroute.instance/api/context/rtk/test \
  -H "Content-Type: application/json" \
  -d '{
    "text":"> Task :compileJava UP-TO-DATE\n> Task :processResources UP-TO-DATE\nBUILD SUCCESSFUL",
    "command":"gradle"
  }' | jq .

```

Expected response:

```json
{
  "compressedText": "BUILD SUCCESSFUL",
  "techniquesUsed": ["rtk-drop"],
  "filterId": "gradle-noop"
}

```

### Step 4: Configure Engine Intensity

Adjust RTK behavior via the configuration API:

```bash
curl -X PUT https://your.omniroute.instance/api/context/rtk/config \
  -H "Content-Type: application/json" \
  -d '{"intensity":"aggressive","applyToToolResults":true}'

```

### Step 5: Monitor Active Filters

List the merged filter catalog with diagnostics:

```bash
curl https://your.omniroute.instance/api/context/rtk/filters | jq .

```

## Key Implementation Files

| File | Purpose |
|------|---------|
| [`open-sse/services/compression/engines/rtk/filterLoader.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/filterLoader.ts) | Filter discovery, validation, and caching across all three sources |
| [`open-sse/services/compression/engines/rtk/filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/filterSchema.ts) | Zod schemas for filter packs with ReDoS protection |
| [`open-sse/services/compression/engines/rtk/configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/configSchema.ts) | Engine configuration field definitions |
| [`src/shared/validation/compressionConfigSchemas.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/compressionConfigSchemas.ts) | Top-level schema merging RTK with generic compression settings |
| [`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts) | SQLite persistence and default application |
| [`src/app/api/context/rtk/config/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/context/rtk/config/route.ts) | GET/PUT endpoint for engine configuration |
| [`src/app/api/context/rtk/filters/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/context/rtk/filters/route.ts) | GET endpoint for active filter catalog |
| [`src/app/api/context/rtk/test/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/context/rtk/test/route.ts) | POST endpoint for filter preview testing |

## Summary

- **Configure the RTK compression engine** by placing signed filter packs in [`.rtk/filters.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/filters.json) (or `.toml`) with optional [`trust.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/trust.json) for verification
- **Three filter sources** are prioritized: project (signed), global (`~/.omniroute/rtk`), and builtin (shipped with OmniRoute)
- **Engine intensity** and payload targeting are controlled via `PUT /api/context/rtk/config`, with settings persisted in SQLite
- **Test filters interactively** using `POST /api/context/rtk/test` before deploying to production
- **All schemas are Zod-validated**, with built-in ReDoS protection for regex patterns

## Frequently Asked Questions

### How do I force OmniRoute to reload my custom filter pack?

Filters reload automatically on the next request. To force immediate reloading and view diagnostics, call `GET /api/context/rtk/filters` — this refreshes the cache and returns validation warnings if your pack has schema errors, according to `filterLoader.ts:43-48`.

### What happens if my project filter fails the SHA-256 trust check?

The loader skips the project source entirely and falls back to global or builtin filters. Set `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` to bypass verification during development, or generate the correct hash using `shasum -a 256` and store it in [`.rtk/trust.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/.rtk/trust.json) for production.

### Can I use TOML instead of JSON for filter packs?

Yes. The loader accepts both formats: [`filters.json`](https://github.com/diegosouzapw/OmniRoute/blob/main/filters.json) or [`filters.toml`](https://github.com/diegosouzapw/OmniRoute/blob/main/filters.toml). The parser auto-detects the format based on file extension in `collectProjectFilterSources()` and `collectGlobalFilterSources()`.

### What's the difference between `intensity` levels in RTK configuration?

The `intensity` setting scales internal heuristics: **minimal** preserves almost all output with basic deduplication; **standard** applies moderate line truncation and pattern filtering; **aggressive** maximizes compression through stricter thresholds, early termination, and deeper deduplication as implemented in the engine's `applyRtkCompression` pipeline.