How Services Are Deployed in the OmniRoute Project: 5 Production Methods

OmniRoute services deploy via Docker containers, Fly.io edge VMs, systemd-managed bare-metal servers, Electron desktop bundles, or remote CLI connections, all running the same Next.js application core configured through environment variables.

The diegosouzapw/OmniRoute repository ships a unified AI gateway that scales from single-user laptops to production-grade VM clusters. Regardless of the target environment, every deployment method leverages the identical compiled bundle in src/app/—a Next.js application that serves HTTP APIs, dashboard UIs, and MCP/A2A transports—differing only in bootstrap configuration and environment variables.

Docker Container Deployment

Docker provides the most reproducible path for running OmniRoute in production or CI pipelines. The containerized approach guarantees identical Node.js 22 runtimes across AMD64 and ARM64 architectures.

Core Configuration Files

The deployment relies on three key files in the repository root:

• Dockerfile – Defines the multi-stage build process that copies source, runs npm ci, and executes npm run build to produce the production image.
• docker-compose.yml – Optional orchestration template demonstrating SQLite volume persistence.
• docs/guides/DOCKER_GUIDE.md – Step-by-step command reference for container operations.

Build and Run Commands

Deploy a container exposing the API on port 20128:

# Build the multi-arch image

docker build -t diegosouzapw/omniroute:latest .

# Run production container

docker run -d \
  -p 20128:20128 \
  -e NODE_ENV=production \
  -e OMNIROUTE_BASE_PATH= \
  --name omniroute \
  diegosouzapw/omniroute:latest

The container executes npm start, which launches the Next.js server handling both API routes (/v1/*) and the dashboard (/).

Fly.io Edge VM Deployment

Fly.io represents the recommended cloud-native deployment target, offering global edge networking, managed persistent volumes, and automatic TLS termination.

Configuration and Setup

Deployment requires generating a fly.toml configuration file and referencing the official guide:

1. fly.toml – Application manifest defining ports, volume mounts, and health checks.
2. docs/ops/FLY_IO_DEPLOYMENT_GUIDE.md – Complete walkthrough for app creation, secret management, and deployment.

Deployment Commands

# Initialize the Fly.io application

flyctl launch --name omniroute --region iad

# Set required encryption and API keys

flyctl secrets set STORAGE_ENCRYPTION_KEY=$(openssl rand -base64 32)

# Deploy to edge VMs

flyctl deploy

Once deployed, Fly.io routes traffic to https://<app>.fly.dev while mounting persistent storage at /data for SQLite or other stateful operations.

VM and Bare-Metal Deployment

For self-hosted scenarios on Ubuntu, Debian, or other Linux distributions, OmniRoute runs directly on the host without containerization overhead.

Systemd Service Configuration

Create a systemd unit file at /etc/systemd/system/omniroute.service:

[Unit]
Description=OmniRoute AI gateway
After=network.target

[Service]
WorkingDirectory=/opt/omniroute
ExecStart=/usr/bin/npm run start
Environment=NODE_ENV=production
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Enable and start the service:

systemctl enable omniroute
systemctl start omniroute

The process listens on 0.0.0.0:20128 by default, or respects the custom PORT environment variable.

Manual Installation Steps

The docs/ops/VM_DEPLOYMENT_GUIDE.md documents the complete bare-metal workflow, including firewall configuration, reverse proxy setup, and the src/server/start.ts entry point execution.

Electron Desktop Deployment

OmniRoute ships as a native desktop application for Windows, macOS, and Linux, bundling the server and UI into platform-specific installers.

Build the desktop client using:

npm run electron:build

This produces installers from the electron/ directory source, documented in electron/README.md. The desktop mode runs the identical Next.js server locally, accessible via localhost without network exposure.

Remote Mode Deployment

The CLI supports remote-control architectures where a local omniroute command drives a server instance deployed via Docker or Fly.io.

CLI-to-Server Architecture

Connection flow utilizes transport layers in src/lib/a2a/ and src/app/api/mcp/:

# Connect local CLI to remote instance

omniroute connect https://my-omniroute.example.com

This stores an authentication token in ~/.omniroute/context.json and routes all subsequent commands through HTTP/MCP transports to the remote /v1 endpoints.

Environment Configuration Across All Modes

All deployment methods share a single configuration surface through environment variables, read at runtime without requiring rebuilds. Critical variables include:

• PORT – HTTP listener port (default: 20128), referenced in src/server/start.ts.
• OMNIROUTE_BASE_PATH – Reverse proxy base path (e.g., /omniroute/) consumed by src/server/nextConfig.ts.
• OMNIROUTE_NO_SUDO – Set to 1 to enable root-less MITM certificate installation in src/mitm/systemCommands.ts.
• BIFROST_ENABLED – Toggle for the Go sidecar high-throughput path in src/app/api/v1/relay/chat/completions/bifrost/route.ts (default: 1).
• OMNIROUTE_LOCAL_ENDPOINTS_ENABLED – Required for bundled Redis launcher functionality in src/lib/security/localEndpoints.ts.
• CORS_ALLOWED_ORIGINS – Production whitelist for src/server/cors/origins.ts, mandatory when exposing OmniRoute behind public proxies.

Complete variable documentation lives in docs/reference/ENVIRONMENT.md.

CI/CD and Release Automation

The repository enforces deployment consistency through docs/ops/RELEASE_CHECKLIST.md, which mandates:

1. Building both the Next.js application (npm run build) and CLI (npm run build:cli).
2. Verifying dist/BUILD_SHA matches the git commit hash.
3. Executing smoke tests against the deployed instance.

GitHub Actions pipelines automate these steps, ensuring every Docker tag and Fly.io release is reproducible and verified.

Summary

• OmniRoute deploys through five primary methods: Docker containers, Fly.io edge VMs, systemd-managed bare-metal servers, Electron desktop apps, and remote CLI connections.
• Every deployment runs the same Next.js bundle from src/app/, differing only in bootstrap configuration and environment variables.
• Docker provides multi-arch images (AMD64/ARM64) with guaranteed Node.js 22 runtimes.
• Fly.io offers the recommended production path with global edge networking and persistent volumes.
• Bare-metal deployments use standard systemd services on Linux hosts, controlled via docs/ops/VM_DEPLOYMENT_GUIDE.md.
• Environment variables like PORT, OMNIROUTE_BASE_PATH, and BIFROST_ENABLED configure runtime behavior without rebuilding.
• Release automation via docs/ops/RELEASE_CHECKLIST.md ensures reproducible builds across all platforms.

Frequently Asked Questions

What is the default port for OmniRoute services?

The default HTTP listener port is 20128, defined in src/server/start.ts and exposed in Docker and Fly.io configurations. You can override this by setting the PORT environment variable before starting the service.

Can I deploy OmniRoute without Docker or containers?

Yes. The VM and bare-metal deployment method documented in docs/ops/VM_DEPLOYMENT_GUIDE.md supports direct execution on Linux servers using systemd. Install Node.js 22, clone the repository, and run npm run start via a systemd service file for production-grade uptime.

How do I enable high-throughput relay mode in production?

Set the BIFROST_ENABLED environment variable to 1 (default) to utilize the Go sidecar for high-throughput operations in src/app/api/v1/relay/chat/completions/bifrost/route.ts. Setting it to 0 falls back to the TypeScript implementation without requiring a code rebuild.

Is it possible to run OmniRoute as a desktop application?

Yes. The Electron desktop deployment packages the entire server and UI into native installers for Windows, macOS, and Linux. Build locally with npm run electron:build or download pre-built installers from GitHub Releases, as documented in electron/README.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}