# How the OmniRoute Prompt Compression Adaptive Context-Budget Dial Works

> Understand how the OmniRoute prompt compression adaptive context-budget dial maximizes token use. It dynamically adjusts input prompts, ensuring requests stay within limits while retaining information.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-13

---

**The adaptive context-budget dial dynamically adjusts the maximum token allowance for input prompts by subtracting reserved tokens for responses and chain-of-thought reasoning from the model’s total context window, ensuring every request stays within provider limits while maximizing information retention.**

The diegosouzapw/OmniRoute repository implements a sophisticated prompt compression system that prevents runtime token overflow through an adaptive context-budget dial. This mechanism calculates a hard floor for prompt tokens based on the model’s context window, the requested `max_tokens`, and any explicit thinking budgets, then enforces that limit across all compression strategies. Understanding this dial is essential for debugging token limit errors and optimizing prompt density.

## Where the Logic Lives in the Source Code

### Strategy Selection and Floor Injection

The compression strategy selector determines which compression mode to apply and injects the adaptive budget floor. In [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts), the code contains the comment "Adaptive context-budget floor/escalation (D-C4)" and implements the logic that adds the calculated floor value to the compression plan based on the model’s window and the request’s `max_tokens`.

### Budget Calculation in the Request Handler

The chat core handler orchestrates the compression pipeline and enforces the final budget calculations. The file [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) includes the comment "Adaptive context-budget (Sub-project C): model context window + request max_tokens drive..." and contains the arithmetic that derives the `promptBudgetFloor` before invoking compression engines.

### Provider Registry and Effort Translation

Provider-specific context limits are defined in [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts), where each entry specifies `maxContextTokens` for its respective model. The translation of numeric budgets into categorical effort levels—`low`, `medium`, `high`, and `xhigh`—is handled by helper functions such as `budgetToEffort` and `applyThinking`, which are tested in [`tests/unit/xai-translators.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/xai-translators.test.ts).

## How the Adaptive Dial Calculates the Context Budget

The dial operates in six distinct steps to guarantee that `promptTokens + responseRoom + thinkingRoom ≤ modelWindow`.

### Step 1: Gather Model Limits

The system retrieves the provider’s total context window from the registry. For example, `gpt-4` might provide 8192 tokens, while other models offer 128k or 200k.

### Step 2: Determine Request-Side Constraints

The incoming request specifies a `max_tokens` value (the desired response length) and optionally a `thinking.budget_tokens` value for chain-of-thought reasoning.

### Step 3: Compute the Prompt Budget Floor

The core calculation establishes the maximum usable tokens for the compressed prompt:

```typescript
// Logic derived from chatCore.ts
const modelWindow = provider.maxContextTokens;              // total available
const responseRoom = request.max_tokens ?? defaultResponse; // reserved for output
const thinkingRoom = request.thinking?.budget_tokens ?? 0;  // reserved for CoT
const promptBudgetFloor = modelWindow - responseRoom - thinkingRoom;

```

If the computed floor is negative, the request is rejected immediately with a clear error before reaching the compression stage.

### Step 4: Inject the Floor into the Compression Plan

The compression service receives the `promptBudgetFloor` and instructs all active engines—**lite**, **caveman**, and **RTK**—to shrink the prompt until it fits under this ceiling. The strategy selector in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) ensures this floor is passed as a non-negotiable constraint.

### Step 5: Map the Final Budget to Provider Effort Levels

After compression, the remaining `budget_tokens` is translated into provider-specific effort buckets. The `budgetToEffort` function converts numeric values into `low`, `medium`, `high`, or `xhigh` categories, ensuring downstream translators (OpenAI, Claude, Gemini) emit the correct `thinking` or `reasoning_effort` fields.

### Step 6: Final Sanity Check

Before dispatching the request, the handler verifies the invariant:

```typescript
if (promptTokens + responseRoom + thinkingRoom > modelWindow) {
  // Re-run compression with stricter floor or reject
}

```

If the sum would still overflow, the compression step repeats with a progressively stricter floor until the constraint holds.

## Practical Implementation Examples

### Programmatic Use with Automatic Budget Calculation

```typescript
import { compressRequest } from "@omniroute/open-sse/services/compression";

const req = {
  model: "gpt-4",
  max_tokens: 1024,
  // No explicit thinking budget → adaptive dial computes floor automatically
  prompt: "…very long prompt text…",
};

const compressed = await compressRequest(req);
// compressed.prompt is guaranteed to be ≤ (modelWindow - 1024 - thinkingRoom)

```

### Debugging the Computed Floor

To inspect the values driving the adaptive dial during development:

```typescript
// Within a request handler (chatCore.ts context)
console.log({
  modelWindow: provider.maxContextTokens,
  responseRoom: request.max_tokens,
  thinkingRoom: request.thinking?.budget_tokens ?? 0,
  calculatedFloor: provider.maxContextTokens - 
                   (request.max_tokens ?? 0) - 
                   (request.thinking?.budget_tokens ?? 0)
});

```

### Disabling the Adaptive Floor for Experiments

You can bypass the dial by creating a compression configuration with `adaptiveContextBudget: false`. When the strategy selector detects this flag, it skips the floor injection, allowing manual control over token limits for A/B testing compression strategies.

## Summary

- **The adaptive context-budget dial** prevents token overflow by calculating a hard floor: `modelWindow - max_tokens - thinkingBudget`.
- **[`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts)** performs the arithmetic and enforces the limit before dispatch.
- **[`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts)** injects the floor into the compression plan managed by the lite, caveman, and RTK engines.
- **[`tests/unit/xai-translators.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/xai-translators.test.ts)** provides the `budgetToEffort` mapping that converts numeric floors into provider-specific effort levels.
- The system guarantees that the sum of prompt tokens, response room, and thinking budget never exceeds the model’s context window, rejecting requests early if the initial configuration is impossible.

## Frequently Asked Questions

### What happens if the computed prompt budget floor is negative?

The request is rejected immediately with a validation error. According to the implementation in [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts), if `modelWindow - max_tokens - thinkingRoom` yields a negative number, the system returns an error before invoking any compression engines, preventing runtime failures against the upstream provider.

### Which compression engines respect the adaptive context-budget dial?

All built-in engines—**lite**, **caveman**, and **RTK**—respect the floor when it is passed through the strategy selector. The [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) file ensures that the `promptBudgetFloor` is included in the compression plan, forcing each engine to truncate or summarize content until the prompt fits within the allowed token count.

### How does the dial handle different provider thinking budget formats?

The dial normalizes everything to a numeric token count first, then maps to provider-specific schemas. After calculating the floor in [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts), the system uses `budgetToEffort` from [`xai-translators.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/xai-translators.test.ts) to convert the remaining budget into categorical levels like `low` or `high`, which translators then format as `thinking_budget` for Claude or `reasoning_effort` for OpenAI.

### Can I disable the adaptive dial for specific requests?

Yes. By setting `adaptiveContextBudget: false` in your compression combo configuration, you instruct the strategy selector to skip the floor injection. This is useful for experiments where you want to manually control token allocation or test compression behavior without automatic constraints, though it risks provider-side rejection if limits are exceeded.