OmniRoute Guardrails Framework: Hot-Reloadable Security Policies for LLM Proxies

OmniRoute's Guardrails framework provides a hot-reloadable, pluggable security layer that intercepts every LLM request and response through configurable preCall and postCall hooks, enabling live policy updates without server restarts.

The OmniRoute open-source LLM proxy implements a comprehensive hot-reloadable security policy framework called Guardrails. This system allows operators to enforce safety constraints, mask sensitive data, and reroute requests dynamically while the server remains operational.

Core Architecture of the Guardrails Framework

The framework centers on three fundamental components that enable flexible, runtime-modifiable security policies.

BaseGuardrail Abstract Class

All security policies extend the BaseGuardrail abstract class defined in src/lib/guardrails/base.ts. This class establishes the contract that every guardrail must implement through two asynchronous methods:

• preCall – Intercepts and potentially modifies the request payload before it reaches the upstream LLM provider
• postCall – Processes the provider's response before returning it to the client

The base class also exports critical TypeScript types including GuardrailContext (request metadata), GuardrailResult (modification/blocking decisions), and GuardrailExecutionResult (execution summaries). These types ensure type safety across the entire guardrail pipeline.

Hot-Reloadable Registry

The src/lib/guardrails/registry.ts file implements a global registry that enables the framework's signature hot-reload capability. Rather than statically importing guardrails at startup, the registry uses dynamic import() statements to load guardrail classes on demand.

When source files change, the registry clears its module cache and re-instantiates guardrail classes. Because each request fetches the latest instances from the registry, policy modifications become effective immediately without process restarts. The registry also maintains a priority-ordered map of enabled guardrails, ensuring consistent execution order.

Execution Flow and Lifecycle

Understanding the request lifecycle reveals how hot-reloadable policies integrate into OmniRoute's request handling.

Pre-Call and Post-Call Interception

Every API request flows through the guardrail pipeline in two phases:

1. Pre-call phase – The registry invokes each enabled guardrail's preCall method sequentially. Guardrails can block the request entirely, modify the payload (such as masking PII or describing images), or pass the request through unchanged.
2. Post-call phase – After the upstream provider returns a response, each guardrail's postCall method sanitizes the output, strips sensitive data, or appends metadata.

The GuardrailContext object passed to both methods contains essential metadata including the target model, provider identifier, request headers, and logging utilities.

Priority-Based Policy Ordering

Guardrails execute according to a numeric priority property defined in the BaseGuardrail class. Lower numbers execute first, allowing foundational policies (like authentication) to run before transformation policies (like PII masking). This ordering persists across hot-reloads, as the registry re-sorts the guardrail map whenever policies refresh.

Built-in Security Policy Implementations

OmniRoute ships with several production-ready guardrails that demonstrate the framework's capabilities.

Prompt Injection Detection

The PromptInjectionGuardrail in src/lib/guardrails/promptInjection.ts implements heuristic and pattern-based detection for malicious prompt injection attempts. Located at lines 249-256, this guardrail analyzes incoming messages for jailbreak patterns and can block requests before they reach the LLM provider.

PII Masking

src/lib/guardrails/piiMasker.ts contains the PIIMaskerGuardrail, which identifies and redacts personally identifiable information. This guardrail operates opt-in by default and never mutates payloads unless explicitly enabled, ensuring backward compatibility.

Vision Bridge Rerouting

The VisionBridgeGuardrail in src/lib/guardrails/visionBridge.ts illustrates sophisticated policy logic for multimodal requests. This guardrail handles image-bearing requests to models lacking native vision support through three strategies:

• Rerouting – Redirects requests to the fastest available vision-capable model when credentials permit
• Image Description – Generates textual descriptions of images using a vision model and injects them into the payload
• Combo Awareness – Introspects combo model definitions via getComboVisionBridgeDecision (lines 44-93) to determine if image description is required

The guardrail utilizes bridgeCache (from src/lib/guardrails/modalityBridge/bridgeCache.ts) to cache image descriptions and avoid redundant processing. It remains fully dependency-injectable, facilitating unit testing with mock vision models.

Implementing Custom Hot-Reloadable Policies

Developers can extend the framework by implementing the BaseGuardrail interface and registering instances with the hot-reloadable registry.

import { BaseGuardrail, type GuardrailResult, type GuardrailContext } from "@/lib/guardrails/base";
import { registerGuardrail } from "@/lib/guardrails/registry";

class RateLimitGuardrail extends BaseGuardrail {
  name = "custom-rate-limit";
  priority = 10;
  private requests = new Map<string, number>();

  async preCall(_payload: unknown, ctx: GuardrailContext): Promise<GuardrailResult> {
    const key = ctx.headers["x-api-key"] as string;
    const count = this.requests.get(key) || 0;
    
    if (count > 100) {
      return { block: true, message: "Rate limit exceeded" };
    }
    
    this.requests.set(key, count + 1);
    return { block: false };
  }
}

// Register enables hot-reloading
registerGuardrail(new RateLimitGuardrail());

For runtime disabling, the registry supports filtering via request headers:

import { getEnabledGuardrails } from "@/lib/guardrails/registry";

function handleRequest(req) {
  const disabled = req.headers["x-disable-guardrails"]?.split(",") ?? [];
  const enabled = getEnabledGuardrails().filter(g => !disabled.includes(g.name));
  // Execute only enabled guardrails...
}

Summary

• Pluggable Architecture – OmniRoute's BaseGuardrail class in src/lib/guardrails/base.ts defines a clear contract for request/response interception via preCall and postCall methods.
• Hot-Reload Capability – The registry in src/lib/guardrails/registry.ts uses dynamic imports and module cache invalidation to enable live policy updates without restarts.
• Priority System – Guardrails execute in numeric priority order, ensuring predictable policy application across reloads.
• Production Examples – Built-in guardrails include prompt injection detection (promptInjection.ts), PII masking (piiMasker.ts), and the sophisticated Vision Bridge (visionBridge.ts) with caching and combo-model awareness.

Frequently Asked Questions

How does OmniRoute enable hot-reloading of security policies?

OmniRoute achieves hot-reloading through the guardrail registry in src/lib/guardrails/registry.ts, which dynamically imports guardrail modules using import() rather than static imports. When files change, the registry clears its module cache and re-instantiates guardrail classes, making new policies available on the subsequent request without restarting the Node.js process.

What is the difference between preCall and postCall guardrails?

preCall methods execute before the request reaches the upstream LLM provider, allowing guardrails to block malicious prompts, mask PII, or transform payloads like converting images to text. postCall methods execute after receiving the provider's response, enabling output sanitization, metadata injection, or response modification before delivery to the client.

Can I disable specific guardrails per request?

Yes. The registry supports runtime filtering through the getEnabledGuardrails() function, which can exclude guardrails based on request headers such as x-disable-guardrails. Pass a comma-separated list of guardrail names to disable specific policies for individual requests while maintaining global defaults.

How does the Vision Bridge guardrail handle non-vision models?

The VisionBridgeGuardrail in src/lib/guardrails/visionBridge.ts detects image content in requests to non-vision models and applies either rerouting or description strategies. It first checks reroute eligibility and credential availability (lines 59-77), then either redirects to a vision-capable model or generates textual descriptions via a vision model, caching results in bridgeCache to optimize subsequent identical requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}