Deploying Embedded Services with OmniRoute: A Complete Guide to Running Auxiliary Binaries

OmniRoute's embedded-services layer lets you run auxiliary binaries like local LLMs and vector stores as child processes supervised within the main proxy process tree, exposing them through the same unified API as external providers.

OmniRoute ships with a lightweight embedded-services layer designed to run auxiliary binaries alongside the main proxy without leaving the process tree. This architecture enables seamless integration of local LLMs, vector stores, and custom tools through a unified provider interface. The implementation in diegosouzapw/OmniRoute centers on three core components that manage process lifecycle, installation, and registration.

Architecture of the Embedded-Services Layer

The embedded-services system is built around three primary concepts that work together to manage auxiliary binaries securely and efficiently.

ServiceSupervisor

The ServiceSupervisor class handles process lifecycle management, including starting, monitoring, and restarting child processes. Located in src/lib/services/types.ts, this component records the PID, exit status, and startup errors in the services table. The supervisor ensures binaries remain healthy and automatically respawns failed services when configured.

Installer Utilities

Installer utilities provide a vetted runNpm wrapper that performs safe npm install operations inside service directories. Implemented in src/lib/services/installers/utils.ts, these utilities handle platform-specific quirks and report installation failures without executing untrusted shell commands. This approach prevents command injection while allowing dynamic service installation.

Service Registry

The Service registry maintains metadata for each embedded service, including name, binary path, default arguments, and health-check endpoints. Defined in src/lib/services/registry.ts, this registry enables the router to treat local services identically to external providers. Registration entries specify the unique identifier, executable command, runtime arguments, and health verification URLs.

Request Flow for Embedded Services

When a request targets an embedded service, OmniRoute processes it through a standardized five-step pipeline:

1. API route capture – Requests like GET /v1/providers/<service>/models are intercepted by the generic provider-router in open-sse/services/.
2. Registry resolution – The router queries the service registry to resolve the backend plugin ID for the requested service.
3. Process validation – If the service is marked enabled, the ServiceSupervisor ensures the binary is running, spawning it on-demand if necessary.
4. Request forwarding – The request is proxied to the service's local HTTP endpoint, typically http://127.0.0.1:<port>.
5. Response translation – Responses from the embedded service are translated back into the OmniRoute API format using translators in open-sse/translator/.

Configuration and Feature Flags

The embedded-services functionality is controlled by the EMBEDDED_SERVICES_ENABLED feature flag, defined in src/shared/constants/featureFlagDefinitions.ts. This flag defaults to true in production environments. When disabled, the "Embedded Services" entry is removed from the sidebar configuration in src/shared/constants/sidebarVisibility/sections.ts, and the supervisor prevents launching any child processes.

Security and Safety Guarantees

OmniRoute implements multiple safeguards to ensure embedded services operate securely within the host environment.

Process Isolation

Child processes are started using Node.js spawn with a minimal environment. The implementation strictly avoids interpolating untrusted arguments into shell strings, adhering to security hardening rules. This prevents shell injection attacks while maintaining flexibility in service configuration.

Local-Only Routing

All /api/services/* routes are classified as local-only paths by src/server/authz/routeGuard.ts through the isLocalOnlyPath() function. This classification shields embedded service endpoints from external exposure, ensuring they remain accessible only from the local machine.

Graceful Teardown

During server shutdown, ServiceSupervisor.stopAll() sends SIGTERM signals to each child process and awaits clean exits. This mechanism prevents orphaned processes and ensures consistent state in the services database table.

Implementation Examples

Registering a new embedded service requires updating the registry and utilizing the supervisor API:

// src/lib/services/registry.ts – registering a new embedded service
export const serviceRegistry = [
  {
    id: "ninerouter",
    name: "9Router",
    binary: "node",
    args: ["-r", "./src/ninerouter/main.js"],
    healthCheck: "http://127.0.0.1:8000/health",
    enabled: true,
  },
];

The ServiceSupervisor class provides the core lifecycle management:

// src/lib/services/types.ts – the supervisor API
export class ServiceSupervisor {
  constructor(private readonly config: ServiceConfig) {}
  async start(): Promise<void> {
    // spawn the binary with safe env, capture stdout/stderr
    this.process = spawn(this.config.binary, this.config.args, {
      env: { ...process.env, NODE_ENV: "production" },
    });
    // hook into exit events and update the DB row
    this.process.on("exit", (code) => updateServiceStatus(this.config.id, code));
  }
  async stop(): Promise<void> {
    this.process?.kill("SIGTERM");
    await once(this.process!, "close");
  }
}

Handlers integrate embedded services using the registry lookup:

// Using the service from a handler (open-sse/handlers/chatCore.ts)
import { getServiceEndpoint } from "@/services/registry";

export async function handleEmbeddedChat(req: Request) {
  const endpoint =

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}