How to Create Custom Compression Engines Using OmniRoute's Pluggable Framework

Implement the five-member engine contract in TypeScript and register your engine via registerCompressionEngine() in open-sse/services/compression/engines/registry.ts to extend OmniRoute with custom compression logic.

OmniRoute's compression subsystem is built around a pluggable engine registry that enables runtime-extensible text optimization. Each engine implements a shared contract and registers at startup, making it instantly available to compression modes, stacked pipelines, the dashboard UI, and public API endpoints. This guide walks through creating and registering a custom engine using the actual source implementation from the diegosouzapw/OmniRoute repository.

Understanding the Engine Contract

Every compression engine must expose five specific members, as defined in open-sse/services/compression/types.ts and documented in the engine guide at lines 29-34:

• id — Stable string identifier (e.g., "caveman", "rtk", "noop")
• compress(input, config) — Primary entry point returning { text, stats } with token-saving metadata
• apply(text, config) — Legacy path used by stacked pipelines; returns transformed text only
• getConfigSchema() — JSON-Schema-like description of configuration shape
• validateConfig(config) — Returns { valid, errors[] } for runtime config validation

The registry enforces uniqueness and schema compliance. In open-sse/services/compression/engines/registry.ts, the helper functions registerCompressionEngine(), unregisterCompressionEngine(), and assertValidEngine() throw at startup if an ID collides or default config fails validation, as noted at lines 35-37 of the engine documentation.

Step 1: Create the Engine Implementation

Create a new directory under open-sse/services/compression/engines/ with an index.ts file exporting the contract. Below is a minimal "noop" engine that passes text through unchanged—useful for testing or as a wrapper base.

// open-sse/services/compression/engines/noop/index.ts

/** No-Op compression engine – useful for testing or as a base for wrappers. */
import { EngineContract, CompressionConfig, CompressionResult } from '@/open-sse/services/compression/types';

/** Stable identifier used throughout the registry and API. */
export const id = 'noop';

/** The engine never modifies the text; it just returns it as-is. */
export async function compress(
  input: string,
  _config: CompressionConfig,
): Promise<CompressionResult> {
  console.debug('[noop] compress called – passing through');
  return {
    text: input,
    stats: {
      tokensSaved: 0,
      originalTokens: 0,
      compressedTokens: 0
    }
  };
}

/** Legacy apply method – forwards to `compress`. */
export async function apply(text: string, config: CompressionConfig): Promise<string> {
  return (await compress(text, config)).text;
}

/** Simple, empty config schema – the engine takes no options. */
export function getConfigSchema() {
  return {
    type: 'object',
    properties: {},
    additionalProperties: false
  };
}

/** Validation always succeeds for the empty schema. */
export function validateConfig(_config: unknown) {
  return { valid: true, errors: [] };
}

/** Export the contract expected by the registry. */
export const engine: EngineContract = {
  id,
  compress,
  apply,
  getConfigSchema,
  validateConfig
};

Reference the caveman engine at open-sse/services/compression/engines/caveman/index.ts for a production-ready implementation with full config validation and complex transformation logic.

Step 2: Register the Engine

Import and register your engine in the central registry. Built-in engines are registered by strategySelector.ts before any request runs, populating the global registry shared by the preview endpoint, runtime compression, and tests.

// open-sse/services/compression/engines/registry.ts

import { engine as noopEngine } from './noop/index';

// Existing built-in registrations …
registerCompressionEngine(noopEngine);

The registerCompressionEngine() call throws if:

• The id already exists in the registry
• The default configuration fails validateConfig()

This ensures safety at startup rather than runtime failure.

Step 3: Add Configurable Parameters (Optional)

For engines with tunable behavior, extend getConfigSchema() with JSON-Schema properties:

export function getConfigSchema() {
  return {
    type: 'object',
    properties: {
      maxLines: { type: 'number', default: 50, description: 'Maximum lines to preserve' },
      trimWhitespace: { type: 'boolean', default: true }
    },
    required: ['maxLines'],
    additionalProperties: false
  };
}

export function validateConfig(config: unknown) {
  const errors: string[] = [];
  if (typeof (config as any)?.maxLines !== 'number') {
    errors.push('maxLines must be a number');
  }
  return { valid: errors.length === 0, errors };
}

The dashboard under Dashboard → Context & Cache → Compression automatically reads this schema to generate configuration forms.

Step 4: Verify via Preview Endpoint

Test your engine using the CLI or HTTP preview before deployment:

# CLI preview with single-engine mode

omniroute compression preview --mode=noop --input "Sample text to process"

# Stacked pipeline with multiple engines

omniroute compression preview --mode=stacked --pipeline='[
  { "engine": "noop" },
  { "engine": "caveman", "config": { "aggressive": true } }
]'

For HTTP access, the preview route at src/app/api/compression/preview/route.ts resolves engines through the same registry:

curl -X POST http://localhost:3000/api/compression/preview \
  -H "Content-Type: application/json" \
  -d '{"mode":"noop","text":"Hello world"}'

Step 5: Write Unit Tests

Add tests under tests/unit/compression/ to guarantee correct integration:

// tests/unit/compression/noop.test.ts

import { assertEquals } from "https://deno.land/std/testing/asserts.ts";
import { compress, apply, validateConfig } from '@/open-sse/services/compression/engines/noop/index';

Deno.test('noop engine returns original text via compress', async () => {
  const input = 'The quick brown fox jumps over the lazy dog.';
  const result = await compress(input, {});
  assertEquals(result.text, input);
  assertEquals(result.stats.tokensSaved, 0);
});

Deno.test('noop engine returns original text via apply', async () => {
  const input = 'Preserves legacy interface';
  const text = await apply(input, {});
  assertEquals(text, input);
});

Deno.test('noop engine accepts any config', () => {
  const result = validateConfig({ unexpected: true });
  assertEquals(result.valid, true);
  assertEquals(result.errors, []);
});

Run with:

node --import tsx/esm --test tests/unit/compression/noop.test.ts

Key Source Files for Custom Compression Development

File	Purpose
open-sse/services/compression/engines/registry.ts	Central registry with registerCompressionEngine() — entry point for all engine registration
open-sse/services/compression/types.ts	TypeScript interfaces defining EngineContract, CompressionConfig, CompressionResult
open-sse/services/compression/engines/caveman/index.ts	Reference implementation showing schema validation and complex transformation
open-sse/services/compression/pipelineEngine.ts	Stacked mode sequencing logic for multi-engine pipelines
src/app/api/compression/preview/route.ts	HTTP endpoint that resolves engines via registry for live preview
docs/compression/COMPRESSION_ENGINES.md	High-level design documentation and extension patterns

Summary

• Engine-centric architecture: OmniRoute's compression is registry-driven, not hardcoded
• Five-member contract: Implement id, compress, apply, getConfigSchema, validateConfig
• Startup safety: registerCompressionEngine() enforces unique IDs and valid defaults
• Immediate availability: Registered engines work in all modes, APIs, pipelines, and dashboard
• Test coverage: Unit tests and CLI preview validate integration before deployment

Frequently Asked Questions

Can multiple custom engines coexist in the same OmniRoute instance?

Yes. Register each engine with a unique id in registry.ts. Engines operate independently and can be combined in stacked pipelines. The registry maintains a flat namespace—no nesting or versioning required.

What's the difference between compress and apply?

compress() is the modern primary interface returning structured results with statistics. apply() is the legacy interface returning only transformed text, maintained for backward compatibility with stacked pipeline sequencing in pipelineEngine.ts. New implementations should implement both, with apply typically delegating to compress.

How do I debug engine execution in production?

Enable debug logging in your engine implementation (as shown in the noop example), then set LOG_LEVEL=debug in environment configuration. The preview endpoint at /api/compression/preview also returns full stats objects showing engine selection and token metrics without modifying persisted data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}