How OmniRoute Implements Persistent Memory: Architecture and Code Deep Dive

OmniRoute implements persistent memory through a modular backend architecture centered on a MemoryManager singleton that coordinates SQLite as the primary durable store with pluggable fallback backends like Qdrant for vector search.

OmniRoute is an open-source AI routing platform that requires robust memory persistence across sessions. The implementation of persistent memory in OmniRoute relies on a sophisticated backend abstraction that separates storage concerns from retrieval logic. According to the diegosouzapw/OmniRoute source code, this design ensures data durability while supporting hybrid search capabilities through a flexible plugin system.

The MemoryManager Singleton

The MemoryManager class in src/lib/memory/manager.ts serves as the central coordinator for all memory operations. This singleton registers multiple storage backends—such as SQLite, Qdrant, and Obsidian—and designates a primary backend (defaulting to SQLite) with optional fallback backends for redundancy.

The manager exposes unified CRUD, list, and search APIs that delegate operations to the primary backend first. If the primary fails or returns no results, the manager automatically iterates through the fallback list. It also provides health-checking across all backends and runtime lifecycle hooks (initialize, shutdown) for proper resource management.

Primary Storage: The SQLite Backend

src/lib/memory/sqliteBackend.ts implements the MemoryBackend interface by forwarding calls to thin wrapper functions defined in src/lib/memory/store.ts. This backend stores each memory entry in a SQLite database initialized by the migration system located in src/lib/db/migrations/.

The SQLite backend supports full creation, retrieval, update, and deletion operations. It also handles vector-based search capabilities by integrating with the retrieval layer, ensuring that local storage remains the source of truth while supporting semantic query patterns.

Hybrid Retrieval and Vector Search

The retrieval layer in src/lib/memory/retrieval.ts handles hybrid search combining keyword matching with embedding-based similarity. When a search request arrives, the primary backend invokes retrieveMemories, which uses the configured embedding provider to execute similarity queries.

This architecture allows OmniRoute to fall back to dedicated vector stores like Qdrant when configured, while maintaining SQLite as the persistent record. The system supports multiple search strategies including "hybrid", "keyword", and "semantic" to balance precision with recall.

Fallback Backend Architecture

OmniRoute supports pluggable fallback backends that implement the same MemoryBackend contract. The Qdrant backend (src/lib/memory/qdrant.ts) stores vectors for fast semantic search, while the Obsidian backend (src/lib/memory/obsidianBackend.ts) provides an alternative storage format.

When the primary SQLite backend returns no result or fails health checks, the MemoryManager automatically queries fallback backends for get, search, and write-through update operations. This design enables graceful degradation and redundancy without changing the application-level API.

Working with Persistent Memory: Code Examples

The following examples demonstrate how to interact with OmniRoute's persistent memory layer using the memoryManager singleton:

Creating and Retrieving Memories

import { memoryManager } from "@/lib/memory/manager";

// Create a persistent memory entry
await memoryManager.create({
  apiKeyId: "key-123",
  sessionId: "sess-abc",
  type: "chat",
  key: "msg-001",
  content: "Hello, world!",
  metadata: { user: "alice" },
});
// The entry is persisted in the SQLite DB via store.ts helpers

// Retrieve a single memory by key
const mem = await memoryManager.get("msg-001");
console.log(mem?.content); // → "Hello, world!"

Updating and Listing Memories

// Update a memory (writes through to fallbacks automatically)
await memoryManager.update("msg-001", { content: "Hi there!" });

// List memories for a specific session with pagination
const { data, total } = await memoryManager.list({
  apiKeyId: "key-123",
  sessionId: "sess-abc",
  type: "chat",
  limit: 20,
  offset: 0,
});
console.log(`Found ${total} messages.`);

Performing Hybrid Search

// Search using hybrid keyword + semantic retrieval
const results = await memoryManager.search({
  apiKeyId: "key-123",
  query: "greeting",
  maxTokens: 512,
  // Optional: strategy = "hybrid" | "keyword" | "semantic"
});
results.forEach((m) => console.log(m.content));

Key Implementation Files

• src/lib/memory/manager.ts – Singleton that registers backends, routes CRUD/search operations, and manages fallback logic.
• src/lib/memory/sqliteBackend.ts – Primary SQLite implementation of the MemoryBackend interface.
• src/lib/memory/store.ts – Low-level SQLite helpers (createMemory, getMemory, etc.) used by the backend.
• src/lib/memory/retrieval.ts – Hybrid retrieval logic combining keyword matching with embedding-based similarity search.
• src/lib/memory/qdrant.ts – Fallback backend storing vectors in Qdrant for fast semantic search.
• src/lib/memory/obsidianBackend.ts – Alternate fallback backend for Obsidian-based storage.

Summary

• OmniRoute implements persistent memory through a modular backend architecture centered on the MemoryManager singleton in src/lib/memory/manager.ts.
• SQLite serves as the primary durable backend, with low-level operations handled in src/lib/memory/store.ts and the interface implemented in src/lib/memory/sqliteBackend.ts.
• Hybrid retrieval in src/lib/memory/retrieval.ts combines keyword and semantic search, supporting multiple strategies including "hybrid", "keyword", and "semantic".
• Fallback backends like Qdrant and Obsidian provide redundancy and specialized vector storage while maintaining a consistent MemoryBackend contract.
• All write operations support write-through to fallbacks, ensuring data consistency across multiple storage systems.

Frequently Asked Questions

How does OmniRoute handle memory persistence across application restarts?

OmniRoute persists memory across restarts by storing data in SQLite databases managed through the SQLiteBackend class. The database schema is created and maintained by the migration system in src/lib/db/migrations/, ensuring that memory entries remain durable on disk between sessions.

What is the difference between the primary backend and fallback backends in OmniRoute?

The primary backend (default SQLite) handles all read and write operations first and serves as the source of truth. Fallback backends (such as Qdrant or Obsidian) are queried automatically if the primary backend fails or returns no results for get and search operations, and they receive write-through updates for redundancy.

Can I use Qdrant as the primary storage instead of SQLite?

Yes, you can configure Qdrant as the primary backend by registering it with the MemoryManager and designating it as primary, though the default configuration uses SQLite. Any backend implementing the MemoryBackend interface—including the Qdrant implementation in src/lib/memory/qdrant.ts—can serve as either primary or fallback.

How does the hybrid search strategy work in OmniRoute's persistent memory?

The hybrid search strategy, implemented in src/lib/memory/retrieval.ts, combines traditional keyword matching with embedding-based similarity search. When you specify strategy: "hybrid" in a search request, the system uses both text matching and vector similarity (via the configured embedding provider) to return the most relevant memories from the persistent store.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}