# How to Structure a Large-Scale Application with OmniRoute: A Layered Monorepo Guide

> Learn to structure large scale applications with OmniRoute. This guide details its modular, layered monorepo approach for building extensible AI gateways.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-17

---

**OmniRoute is designed as a modular, layered monorepo that cleanly separates API routes, streaming logic, domain policies, persistence, and auxiliary services, enabling teams to build an extensible AI gateway that scales across dozens of providers.**

OmniRoute (`diegosouzapw/OmniRoute`) is an open-source AI gateway built to expose an OpenAI-compatible API while orchestrating multiple upstream providers. Learning how to structure a large-scale application with OmniRoute means adopting its directory conventions and request lifecycle patterns that keep the public API, business rules, and provider integrations testable and decoupled.

## High-Level Directory Layout

OmniRoute organizes code into ten distinct layers. Each layer owns a single responsibility and maps directly to the request lifecycle described in [`docs/architecture/ARCHITECTURE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/architecture/ARCHITECTURE.md).

### API Routes Layer

The **API Routes** layer lives in `src/app/api/v1/` and exposes OpenAI-compatible endpoints such as `/v1/chat/completions` and `/v1/embeddings`, alongside management APIs like `/api/providers` and `/api/combos`. The entry point for chat completions is [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts), which validates incoming requests with Zod before delegating to the core handler.

### SSE and Routing Core

The **SSE / Routing Core** under `open-sse/handlers/` and `open-sse/executors/` manages request parsing, model resolution, translation, and streaming to the client. The central orchestrator is [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts), which dispatches provider-specific HTTP calls and manages response streams.

### Domain and Policy Layer

The **Domain / Policy** layer in `src/domain/` enforces centralized business rules before any provider call. Key modules include [`src/domain/comboResolver.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/comboResolver.ts) for combo resolution and [`src/domain/policyEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/policyEngine.ts) for lockout, budgeting, and fallback checks.

### Persistence Layer

The **Persistence** layer in `src/lib/db/` uses a SQLite-backed state with a write-through cache pattern. All mutable state is stored in a single SQLite file (`${DATA_DIR}/storage.sqlite`). The connection, WAL setup, and migrations are handled by [`src/lib/db/core.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/core.ts), while domain-specific CRUD operations live in [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts) and usage history is tracked via [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts).

### Provider Executors and Translators

The **Provider Executors** in `open-sse/executors/` contain one executor per provider family, such as `DefaultExecutor`, `AntigravityExecutor`, and `AzureOpenAIExecutor`. Each implements URL building, header management, retry logic, and token refresh. The **Translators** under `open-sse/translator/` convert provider-specific schemas to the canonical OpenAI format; for example, [`open-sse/translator/request/claude-to-openai.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/request/claude-to-openai.ts) handles Claude request normalization.

### Auth and Security

The **Auth & Security** layer in `src/server/authz/` manages request classification, API-key enforcement, IP filtering, and route guarding. The main pipeline is defined in [`src/server/authz/pipeline.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server/authz/pipeline.ts).

### Auxiliary Services

Beyond the core gateway, OmniRoute supports three auxiliary service categories:

- **Embedded Services** (`src/lib/services/`): Optional locally-run AI tools like 9Router and CLIProxyAPI that appear as regular providers. The lifecycle manager is [`src/lib/services/ServiceSupervisor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/services/ServiceSupervisor.ts).
- **Cloud Agents** (`src/lib/cloudAgent/`): DB-backed task lifecycle for remote agents such as Codex Cloud, Devin, and Jules, coordinated through [`src/lib/cloudAgent/registry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/cloudAgent/registry.ts).
- **Webhooks and Evaluation** ([`src/lib/webhookDispatcher.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/webhookDispatcher.ts), `src/lib/evals/`): Outbound event dispatch and automated quality-assurance suites.

### Dashboard UI

The **Dashboard UI** in `src/app/(dashboard)/dashboard/` provides React-based pages for provider management, combo editing, analytics, and settings. An example page is `src/app/(dashboard)/dashboard/providers/page.tsx`.

## Request Lifecycle for Chat Completions

Understanding the request flow is essential when structuring a large-scale application with OmniRoute. A typical chat completion follows this path:

1. The client posts to `/v1/chat/completions`, handled by [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts).
2. **Zod validation** guarantees type safety before any business logic executes.
3. The route delegates to [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts), which resolves the model and invokes the **domain policy engine** ([`src/domain/policyEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/policyEngine.ts)).
4. The policy engine enforces lockout, budget, and fallback checks before the provider call.
5. [`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts) invokes the appropriate **provider executor** (e.g., [`open-sse/executors/defaultExecutor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/defaultExecutor.ts)), which performs the upstream HTTP request.
6. **Circuit breaker** logic ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) and **account fallback** logic ([`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)) prevent cascading failures by retrying alternate credentials or models on transient errors.
7. The executor streams the response through the translator registry (`open-sse/translator/`) and generic stream helpers ([`open-sse/utils/stream.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/stream.ts)), normalizing chunks into OpenAI-compatible SSE format.
8. Token usage is extracted and persisted via [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts).

Error sanitisation is enforced by [`open-sse/utils/error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/error.ts), ensuring raw stack traces never leak to clients. Outbound requests are guarded by [`src/shared/network/safeOutboundFetch.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/network/safeOutboundFetch.ts) and [`src/shared/network/outboundUrlGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/network/outboundUrlGuard.ts) to block SSRF-prone private URLs.

## Combo Routing and Auto-Combo

OmniRoute supports **named combos** (static sequences) and **Auto-Combo** (dynamic, score-based selection). The engine lives under `open-sse/services/autoCombo/` and evaluates 19 strategies using a nine-factor scoring model that weighs cost, latency, success rate, quota headroom, and breaker state.

- The engine entry point is [`open-sse/services/autoCombo/autoComboEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/autoComboEngine.ts).
- Scoring logic resides in [`open-sse/services/autoCombo/scoringEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoringEngine.ts).
- The **virtual factory** inside [`autoComboEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/autoComboEngine.ts) materializes an ad-hoc combo when no static definition matches.

You can define a custom combo via a dedicated route:

```typescript
// src/app/api/combos/my-cheap-combo/route.ts
import { defineCombo } from '@/open-sse/services/combo/defineCombo';

export const GET = async () => {
  await defineCombo({
    name: 'my-cheap-combo',
    models: [
      'openai/gpt-3.5-turbo',
      'anthropic/claude-instant-1.2',
      'xai/grok-7b',
    ],
    strategy: 'weighted',
    weights: [0.5, 0.3, 0.2],
  });
  return new Response('Combo created', { status: 201 });
};

```

Combo definitions are persisted in the `combos` table via [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts).

To invoke Auto-Combo from a client, prefix the model name with `auto/`, such as `auto/fast`:

```bash
curl -X POST http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto/fast",
    "messages": [{ "role": "user", "content": "Write a short poem about clouds." }]
  }'

```

The `auto/fast` prefix triggers the Auto-Combo engine, which selects the cheapest low-latency models that satisfy the request. For scoring formulas and strategy details, see [`docs/routing/AUTO-COMBO.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/routing/AUTO-COMBO.md).

## Persistence Strategy

All mutable state is persisted in a **single SQLite file** (`${DATA_DIR}/storage.sqlite`). The DB layer follows a **write-through cache** pattern: in-memory Maps hold the current state, and every mutation is synchronously written to SQLite via the `src/lib/db/*` modules. This guarantees crash-safety while keeping runtime reads fast.

Key modules include:

- [`src/lib/db/core.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/core.ts) – connection, WAL setup, and migrations.
- [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts) – CRUD for domain tables such as fallback chains, budgets, and lockouts.
- [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts) – usage-history facade for the `usage_history` table.

## Extending the System

When adding a new provider, combo strategy, or UI tab, follow OmniRoute's layered convention:

1. **Register the provider** in [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts) or the OAuth registry under `src/lib/oauth/providers/`.
2. **Create an executor** that extends `BaseExecutor` in a new file such as [`open-sse/executors/myProviderExecutor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/myProviderExecutor.ts).
3. **Add request/response translators** if the provider's schema differs from OpenAI.
4. **Expose a route** under `src/app/api/v1/providers/[myProvider]/chat/completions/route.ts` if a dedicated per-provider endpoint is required.
5. **Write a unit test** in `tests/unit/` that exercises the new flow; the test suite verifies lint, type-check, and coverage gates automatically.

For example, adding a new OAuth provider such as "MyAI" requires a definition file and an index re-export:

```typescript
// src/lib/oauth/providers/myai.ts
import { OAuthProvider } from '@/src/lib/oauth/types';

export const myai: OAuthProvider = {
  name: 'myai',
  authUrl: 'https://login.myai.com/auth',
  tokenUrl: 'https://login.myai.com/token',
  clientIdEnv: 'MYAI_CLIENT_ID',
  clientSecretEnv: 'MYAI_CLIENT_SECRET',
  scopes: ['api.read', 'api.write'],
};

```

After re-exporting in [`src/lib/oauth/providers/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/oauth/providers/index.ts), run `npm run typecheck:core` to ensure the new type flows through the system.

## Security and Resilience

OmniRoute implements a three-layer resilience model spanning provider, connection, and model scopes, as detailed in [`docs/security/RESILIENCE_GUIDE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/security/RESILIENCE_GUIDE.md).

- **Circuit breakers** ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) prevent flapping providers from overwhelming the system.
- **Connection cooldowns** ([`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)) handle transient errors such as `429` or `500` by retrying alternate credentials.
- **Error sanitisation** ([`open-sse/utils/error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/error.ts)) strips raw stack traces from client-facing responses.
- **Outbound fetch guards** ([`src/shared/network/safeOutboundFetch.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/network/safeOutboundFetch.ts) and [`src/shared/network/outboundUrlGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/network/outboundUrlGuard.ts)) block SSRF-prone private URLs.

## Deployment Topology

OmniRoute runs as a single **Node.js process** combining Next.js and the SSE core. It can be containerized, run on a developer workstation, or deployed on a VM. The architecture follows a three-tier interaction between **Clients → OmniRoute → Upstream Providers**, with optional **Cloud Sync** and **Embedded Services** attached as sidecars.

To start a minimal OmniRoute server:

```typescript
import { createServer } from 'http';
import next from 'next';

const dev = process.env.NODE_ENV !== 'production';
const app = next({ dev, hostname: '0.0.0.0', port: 20128 });
const handle = app.getRequestHandler();

await app.prepare();

createServer((req, res) => {
  // All Next.js API routes (including /v1/*) are handled automatically
  handle(req, res);
}).listen(20128, () => {
  console.log('🚀 OmniRoute listening on http://localhost:20128');
});

```

This pattern is implemented in [`src/server.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server.ts).

## Summary

- OmniRoute uses a **layered monorepo** structure that isolates API routes (`src/app/api/v1/`), routing core (`open-sse/handlers/`), domain policies (`src/domain/`), and persistence (`src/lib/db/`).
- The **request lifecycle** enforces Zod validation, domain policy checks, circuit breakers, and account fallback before streaming normalized responses back to the client.
- **Combo routing** supports both static named combos and dynamic Auto-Combo selection via [`open-sse/services/autoCombo/autoComboEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/autoComboEngine.ts).
- **SQLite persistence** follows a write-through cache pattern for crash-safe, high-performance state management.
- Extending the gateway follows a strict convention: register the provider, create an executor, add translators, expose routes, and write tests.

## Frequently Asked Questions

### What is the best way to add a new AI provider to OmniRoute?

Register the provider in [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts) (or the OAuth registry under `src/lib/oauth/providers/`), then create an executor extending `BaseExecutor` in `open-sse/executors/`. Add request and response translators if the schema deviates from OpenAI, expose a dedicated route under `src/app/api/v1/providers/` if needed, and add unit tests in `tests/unit/`.

### How does OmniRoute handle provider failures and retries?

OmniRoute applies a three-scope resilience model using **circuit breakers** ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) at the provider level and **account fallback** logic ([`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)) at the connection level. If an upstream returns a transient error such as `429` or `500`, the system automatically retries an alternate credential or model before returning an error to the client.

### Where does OmniRoute store configuration and usage data?

All mutable state lives in a single SQLite database (`${DATA_DIR}/storage.sqlite`) managed by [`src/lib/db/core.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/core.ts). The architecture uses a write-through cache so that in-memory Maps serve fast reads while SQLite guarantees durability for provider connections, model aliases, combos, usage history, and circuit-breaker state.

### Can OmniRoute dynamically select models instead of using static routing?

Yes. The **Auto-Combo engine** in [`open-sse/services/autoCombo/autoComboEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/autoComboEngine.ts) dynamically evaluates 19 strategies across a nine-factor scoring model. By sending a request with a model prefix such as `auto/fast`, the engine selects the optimal provider based on real-time cost, latency, success rate, quota headroom, and breaker state.