How the OmniRoute Prompt Compression Adaptive Context-Budget Dial Works

The adaptive context-budget dial dynamically adjusts the maximum token allowance for input prompts by subtracting reserved tokens for responses and chain-of-thought reasoning from the model’s total context window, ensuring every request stays within provider limits while maximizing information retention.

The diegosouzapw/OmniRoute repository implements a sophisticated prompt compression system that prevents runtime token overflow through an adaptive context-budget dial. This mechanism calculates a hard floor for prompt tokens based on the model’s context window, the requested max_tokens, and any explicit thinking budgets, then enforces that limit across all compression strategies. Understanding this dial is essential for debugging token limit errors and optimizing prompt density.

Where the Logic Lives in the Source Code

Strategy Selection and Floor Injection

The compression strategy selector determines which compression mode to apply and injects the adaptive budget floor. In open-sse/services/compression/strategySelector.ts, the code contains the comment "Adaptive context-budget floor/escalation (D-C4)" and implements the logic that adds the calculated floor value to the compression plan based on the model’s window and the request’s max_tokens.

Budget Calculation in the Request Handler

The chat core handler orchestrates the compression pipeline and enforces the final budget calculations. The file open-sse/handlers/chatCore.ts includes the comment "Adaptive context-budget (Sub-project C): model context window + request max_tokens drive..." and contains the arithmetic that derives the promptBudgetFloor before invoking compression engines.

Provider Registry and Effort Translation

Provider-specific context limits are defined in open-sse/config/providerRegistry.ts, where each entry specifies maxContextTokens for its respective model. The translation of numeric budgets into categorical effort levels—low, medium, high, and xhigh—is handled by helper functions such as budgetToEffort and applyThinking, which are tested in tests/unit/xai-translators.test.ts.

How the Adaptive Dial Calculates the Context Budget

The dial operates in six distinct steps to guarantee that promptTokens + responseRoom + thinkingRoom ≤ modelWindow.

Step 1: Gather Model Limits

The system retrieves the provider’s total context window from the registry. For example, gpt-4 might provide 8192 tokens, while other models offer 128k or 200k.

Step 2: Determine Request-Side Constraints

The incoming request specifies a max_tokens value (the desired response length) and optionally a thinking.budget_tokens value for chain-of-thought reasoning.

Step 3: Compute the Prompt Budget Floor

The core calculation establishes the maximum usable tokens for the compressed prompt:

// Logic derived from chatCore.ts
const modelWindow = provider.maxContextTokens;              // total available
const responseRoom = request.max_tokens ?? defaultResponse; // reserved for output
const thinkingRoom = request.thinking?.budget_tokens ?? 0;  // reserved for CoT
const promptBudgetFloor = modelWindow - responseRoom - thinkingRoom;

If the computed floor is negative, the request is rejected immediately with a clear error before reaching the compression stage.

Step 4: Inject the Floor into the Compression Plan

The compression service receives the promptBudgetFloor and instructs all active engines—lite, caveman, and RTK—to shrink the prompt until it fits under this ceiling. The strategy selector in strategySelector.ts ensures this floor is passed as a non-negotiable constraint.

Step 5: Map the Final Budget to Provider Effort Levels

After compression, the remaining budget_tokens is translated into provider-specific effort buckets. The budgetToEffort function converts numeric values into low, medium, high, or xhigh categories, ensuring downstream translators (OpenAI, Claude, Gemini) emit the correct thinking or reasoning_effort fields.

Step 6: Final Sanity Check

Before dispatching the request, the handler verifies the invariant:

if (promptTokens + responseRoom + thinkingRoom > modelWindow) {
  // Re-run compression with stricter floor or reject
}

If the sum would still overflow, the compression step repeats with a progressively stricter floor until the constraint holds.

Practical Implementation Examples

Programmatic Use with Automatic Budget Calculation

import { compressRequest } from "@omniroute/open-sse/services/compression";

const req = {
  model: "gpt-4",
  max_tokens: 1024,
  // No explicit thinking budget → adaptive dial computes floor automatically
  prompt: "…very long prompt text…",
};

const compressed = await compressRequest(req);
// compressed.prompt is guaranteed to be ≤ (modelWindow - 1024 - thinkingRoom)

Debugging the Computed Floor

To inspect the values driving the adaptive dial during development:

// Within a request handler (chatCore.ts context)
console.log({
  modelWindow: provider.maxContextTokens,
  responseRoom: request.max_tokens,
  thinkingRoom: request.thinking?.budget_tokens ?? 0,
  calculatedFloor: provider.maxContextTokens - 
                   (request.max_tokens ?? 0) - 
                   (request.thinking?.budget_tokens ?? 0)
});

Disabling the Adaptive Floor for Experiments

You can bypass the dial by creating a compression configuration with adaptiveContextBudget: false. When the strategy selector detects this flag, it skips the floor injection, allowing manual control over token limits for A/B testing compression strategies.

Summary

• The adaptive context-budget dial prevents token overflow by calculating a hard floor: modelWindow - max_tokens - thinkingBudget.
• open-sse/handlers/chatCore.ts performs the arithmetic and enforces the limit before dispatch.
• open-sse/services/compression/strategySelector.ts injects the floor into the compression plan managed by the lite, caveman, and RTK engines.
• tests/unit/xai-translators.test.ts provides the budgetToEffort mapping that converts numeric floors into provider-specific effort levels.
• The system guarantees that the sum of prompt tokens, response room, and thinking budget never exceeds the model’s context window, rejecting requests early if the initial configuration is impossible.

Frequently Asked Questions

What happens if the computed prompt budget floor is negative?

The request is rejected immediately with a validation error. According to the implementation in chatCore.ts, if modelWindow - max_tokens - thinkingRoom yields a negative number, the system returns an error before invoking any compression engines, preventing runtime failures against the upstream provider.

Which compression engines respect the adaptive context-budget dial?

All built-in engines—lite, caveman, and RTK—respect the floor when it is passed through the strategy selector. The strategySelector.ts file ensures that the promptBudgetFloor is included in the compression plan, forcing each engine to truncate or summarize content until the prompt fits within the allowed token count.

How does the dial handle different provider thinking budget formats?

The dial normalizes everything to a numeric token count first, then maps to provider-specific schemas. After calculating the floor in chatCore.ts, the system uses budgetToEffort from xai-translators.test.ts to convert the remaining budget into categorical levels like low or high, which translators then format as thinking_budget for Claude or reasoning_effort for OpenAI.

Can I disable the adaptive dial for specific requests?

Yes. By setting adaptiveContextBudget: false in your compression combo configuration, you instruct the strategy selector to skip the floor injection. This is useful for experiments where you want to manually control token allocation or test compression behavior without automatic constraints, though it risks provider-side rejection if limits are exceeded.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}