# Architecture of the OmniRoute Compression Pipeline with Risk Gates

> Explore the OmniRoute compression pipeline architecture. Learn how risk gates and circuit breakers ensure safe, open-fail processing and prevent original request corruption.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-08-01

---

**OmniRoute implements a stacked, mode-driven compression pipeline that processes prompts through multiple engines before reaching the model executor, using per-step risk gates and circuit breakers to ensure that any compression failure fails open and never corrupts the original request.**

The architecture of the compression pipeline with risk gates in the diegosouzapw/OmniRoute repository provides a production-ready solution for reducing token costs while maintaining strict safety guarantees. This system processes every request through a seven-stage pipeline that can aggressively compress prompts using modes ranging from `lite` to `ultra`, yet automatically falls back to the original prompt if any engine produces risky output.

## How the Compression Pipeline Works

The pipeline executes as a pre-processing layer before requests reach [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts). Each stage transforms the request or validates safety constraints, creating a resilient chain where failures are contained rather than propagated.

### Stage 1: Effective Mode Selection

The entry point `getEffectiveMode()` in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) determines which compression strategy to apply. It evaluates the request’s combo configuration, override flags, auto-trigger rules, and cache-aware markers to select from modes including `off`, `lite`, `standard`, `aggressive`, `ultra`, `rtk`, and `stacked`. This decision dictates how aggressively the system will attempt to reduce token counts.

### Stage 2: Compression Plan Resolution

Once the mode is determined, `selectCompressionPlan()` in [`resolveCompressionPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resolveCompressionPlan.ts) translates that mode into an ordered list of **engine descriptors**. Each descriptor specifies the engine name, configuration parameters, and **TV1 bail-out thresholds** that define when an engine should abandon processing if it cannot achieve sufficient compression.

### Stage 3: Stacked Engine Execution

The `applyStackedCompression()` function orchestrates the actual transformation by iterating through the engine list. Implemented in [`stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stackedStepCore.ts), this stage calls `applyCompressionAsync()` for each engine sequentially. Every engine receives the output of the previous step and returns either a transformed prompt or a bail-out signal indicating it could not safely compress the content.

### Stage 4: Per-Step Risk Gates

Before any engine’s output advances to the next stage, [`riskGateStep.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskGateStep.ts) executes **risk-gate logic** against the patterns defined in [`riskPatterns.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskPatterns.ts). This validation layer checks for dangerous artifacts such as runaway token growth, malformed JSON structures, or unauthorized system-prompt modifications. The gate evaluates each pattern through `applyRiskGate()` and returns one of three actions: **ALLOW**, **SKIP**, or **FAIL**.

### Stage 5: Engine-Level Circuit Breaker

A process-local circuit breaker in [`pipelineEngineBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/pipelineEngineBreaker.ts) monitors exception frequency across requests. When an engine exceeds configured failure thresholds, the breaker opens and automatically excludes that engine from subsequent compression plans. This cross-request resilience prevents a single faulty engine from repeatedly degrading performance.

### Stage 6: Telemetry and Final Hand-off

After pipeline completion, [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts) aggregates metrics including original versus compressed token counts, savings percentages, and per-engine diagnostics. The final payload—whether compressed or the original prompt—passes to [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts) for model execution. If the response requires decompression, that processing occurs after the model returns its output.

## Risk Gate Mechanics and Decision Logic

Risk gates serve as the primary safety mechanism within the architecture of the compression pipeline with risk gates, ensuring that only validated transformations reach the model.

**Pattern Definition** – The [`riskPatterns.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskPatterns.ts) file contains regex-based definitions that capture specific failure modes. These patterns detect anomalies like JSON syntax errors, unexpected token inflation, or structural corruption that could confuse the downstream model.

**Step-Wise Evaluation** – The [`riskGateStep.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskGateStep.ts) module receives each engine’s output and executes the full pattern suite. This evaluation occurs immediately after compression but before the result flows to the next engine or the final output.

**Decision Actions** – The gate returns discrete actions that determine pipeline flow:
- **ALLOW** – Accepts the engine output and passes it to the next compression stage or final output.
- **SKIP** – Discards the current engine’s result and continues with the previous valid prompt, effectively removing that engine from the chain for this request.
- **FAIL** – Aborts the entire compression pipeline immediately and returns the original, uncompressed prompt to the request flow.

Risk-gate outcomes feed into the circuit breaker statistics, ensuring that engines triggering frequent skips or failures are automatically suppressed in future requests.

## Safety Guarantees and Failure Modes

The pipeline architecture provides three critical safety guarantees that prevent compression errors from impacting production traffic.

**Fail-Open Behavior** – If any compression engine throws an unhandled exception, the system catches the error and falls back to the original prompt. This ensures that requests never fail due to compression logic errors.

**TV1 Bail-Out Thresholds** – Defined in [`stepDetailConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stepDetailConfig.ts), these per-step thresholds allow engines to short-circuit their own processing when they cannot achieve minimum token reduction targets. This prevents wasted compute on ineffective compression attempts.

**Cross-Request Resilience** – The circuit breaker tracks failure frequency across multiple requests rather than just within a single pipeline execution. When failure rates exceed configurable limits, the engine is bypassed entirely until it recovers, protecting the overall system health.

## Implementation Examples

### Invoking the Pipeline Internally

```typescript
// Manually invoke the compression pipeline (used by internal APIs)
import { applyStackedCompression } from "@omniroute/open-sse/services/compression/strategySelector";

const requestBody = { messages: [{ role: "user", content: BIG_PROMPT }] };
const config = { compression: { mode: "aggressive" } }; // Overrides any combo defaults

const { compressedBody, stats } = await applyStackedCompression(requestBody, config);
console.log(`Saved ${stats.savingsPct}% tokens`);

```

### HTTP Preview Endpoint

```bash

# Using the HTTP `/api/compression/preview` endpoint

curl -X POST https://your.omniroute/api/compression/preview \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"<huge prompt>"}],"compression":{"mode":"rtk"}}'

```

### Testing Risk Gate Behavior

```typescript
// Inspecting risk-gate stats in a test
import { applyCompressionAsync } from "@omniroute/open-sse/services/compression/strategySelector";

test("risk gate blocks malformed JSON", async () => {
  const badPrompt = "```json {invalid}";
  const { result, gateAction } = await applyCompressionAsync(badPrompt, { engine: "rtk" });
  expect(gateAction).toBe("SKIP"); // engine was ignored, original prompt kept
});

```

## Summary

- **Seven-stage pipeline** – Mode selection, plan resolution, stacked execution, risk gates, circuit breakers, telemetry, and final hand-off create a complete preprocessing layer.
- **Risk gates provide three actions** – ALLOW permits the transformation, SKIP removes the current engine from the chain, and FAIL aborts compression entirely.
- **Fail-open guarantees** – Every failure path returns the original prompt, ensuring model inputs remain valid even when compression engines malfunction.
- **Circuit breaker protection** – Cross-request monitoring in [`pipelineEngineBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/pipelineEngineBreaker.ts) automatically suppresses flaky engines to maintain system reliability.
- **Configurable thresholds** – TV1 bail-out logic in [`stepDetailConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stepDetailConfig.ts) lets engines abandon ineffective compression early, optimizing compute usage.

## Frequently Asked Questions

### What are risk gates in OmniRoute's compression pipeline?

Risk gates are validation checkpoints defined in [`riskGateStep.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskGateStep.ts) that inspect each compression engine’s output against regex patterns from [`riskPatterns.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/riskPatterns.ts) before allowing the result to proceed. They detect dangerous artifacts like malformed JSON or runaway token growth, returning ALLOW, SKIP, or FAIL actions that control whether the transformation is accepted, the engine is bypassed, or the entire pipeline aborts.

### How does the circuit breaker protect against faulty compression engines?

The circuit breaker in [`pipelineEngineBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/pipelineEngineBreaker.ts) tracks exception and failure rates across multiple requests, not just within a single pipeline execution. When an engine exceeds configurable failure thresholds, the breaker opens and automatically excludes that engine from subsequent compression plans until it recovers, preventing repeated degradation of user requests.

### What happens when a compression engine fails or produces invalid output?

When an engine fails, the risk gate returns either **SKIP**—which removes that engine from the current chain and continues with the previous prompt—or **FAIL**, which immediately aborts the entire pipeline and returns the original uncompressed prompt. Additionally, unhandled exceptions trigger fail-open behavior, ensuring the original request always reaches the model executor in [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts).

### Which entry point should developers use to invoke the compression pipeline manually?

Developers should import `applyStackedCompression` from [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) to programmatically compress prompts, or use the HTTP `/api/compression/preview` endpoint for testing compression results without executing the model. Both methods accept a configuration object specifying the compression mode and return detailed statistics including token savings percentages and per-engine diagnostics.