Architecture of OmniRoute's Persistent Memory System Using FTS5 and Qdrant

OmniRoute implements a tiered persistence layer that stores all conversational memories in SQLite with FTS5 full-text indexing, while supporting both Qdrant and sqlite-vec for semantic vector search, automatically degrading from vector-based retrieval to keyword search when vector stores are unavailable.

OmniRoute is an open-source AI routing platform that implements a sophisticated persistent memory system to store and retrieve conversational context across sessions. The architecture combines SQLite's native FTS5 extension for fast keyword matching with optional Qdrant vector database integration to enable hybrid semantic retrieval. This dual-engine approach ensures high availability while supporting both exact-match and vector-similarity search patterns.

SQLite Foundation with FTS5 Indexing

All memory data persists in a SQLite database, leveraging the memory_fts virtual table for high-performance text search. This foundation ensures data durability while providing the speed necessary for real-time conversational retrieval.

The memories Table Schema

Every conversational memory resides in the memories table, defined in migration src/lib/db/migrations/022_add_memory_fts5.sql. This table stores raw conversation content, metadata, TTL settings, and unique keys for identification. The schema supports both ephemeral and long-lived memories through configurable expiration timestamps, with CRUD operations abstracted in src/lib/db/memoryVec.ts.

FTS5 Virtual Table Implementation

For fast keyword searches, OmniRoute creates an FTS5 virtual table named memory_fts that indexes the content and key columns of the memories table. This implementation, refined in migration src/lib/db/migrations/023_fix_memory_fts_uuid.sql, uses stable integer rowids to maintain referential integrity. When vector stores are disabled or unavailable, the system queries this FTS5 index directly using the buildFtsRows function to perform ranked text matching against the indexed columns.

Vector Store Architecture: Qdrant and sqlite-vec

OmniRoute abstracts vector storage behind a unified interface that supports two distinct backends: an external Qdrant cluster for production semantic search, and the sqlite-vec extension for in-process approximate nearest neighbor (ANN) queries. The selection logic in src/lib/memory/retrieval.ts evaluates settings.vectorStore to determine which backend to invoke.

Qdrant Integration

When settings.vectorStore equals "qdrant", OmniRoute connects to an external Qdrant cluster over HTTP. The configuration, defined in src/lib/memory/qdrant.ts, supports collection management, semantic search, and optional quantization modes including none, int8, and binary. The upsertSemanticMemoryPoint function handles vector upserts, while searchSemanticMemory executes semantic queries against the Qdrant collection, returning IDs that are resolved back to full records via fetchMemoriesByIds.

sqlite-vec Fallback

For deployments without external dependencies, OmniRoute falls back to sqlite-vec, an in-process SQLite extension that provides ANN capabilities. This backend activates when settings.vectorStore equals "sqlite-vec", storing vectors directly within the SQLite database file alongside the memories table. The system uses the same abstraction layer to ensure consistent behavior between Qdrant and sqlite-vec backends, including access to the memory_vec_meta table for vector metadata.

Tiered Retrieval Hierarchy and Fallback Logic

The retrieval engine implements a graceful degradation strategy that prioritizes semantic accuracy while ensuring availability. This tiered approach is orchestrated in src/lib/memory/retrieval.ts, which attempts retrieval in order of sophistication: Qdrant first, then sqlite-vec, and finally FTS5 keyword search.

Semantic Retrieval Flow

For semantic queries, the system first checks if Qdrant is enabled and healthy. If so, it calls searchSemanticMemory from src/lib/memory/qdrant.ts to retrieve candidate IDs, then fetches the full memory records from SQLite using fetchMemoriesByIds. If Qdrant returns no results or fails, the system automatically attempts sqlite-vec; if that also fails, it degrades to the FTS5 index with the tier marker "fts5".

Hybrid Search with Reciprocal Rank Fusion

When using the sqlite-vec backend for hybrid retrieval, OmniRoute performs reciprocal rank fusion (RRF) to combine vector similarity scores with FTS5 keyword match scores. This approach weights semantic relevance against exact term matches, providing more accurate results than either method alone. The RRF implementation ensures that memories matching both semantically and lexicographically receive higher rankings, with the final ordering logged for observability.

Engine Observability and Configuration

OmniRoute exposes the current memory backend status through the /api/settings/memory/engine-status endpoint. This runtime inspection, defined in src/shared/schemas/memory.ts, reports whether FTS5, the embedding source, vector store, Qdrant health, and optional rerank services are active. The rerank service, typically available at http://127.0.0.1:20128/v1/rerank, can reorder results after primary retrieval for improved accuracy.

Implementation Examples

The following examples demonstrate how to interact with the tiered memory system:

// Retrieve memories with semantic strategy, respecting the tier hierarchy
import { retrieveMemories } from "@/lib/memory/retrieval";

const apiKeyId = "user-api-key-123";
const result = await retrieveMemories(apiKeyId, {
  query: "how to reset my password",
  retrievalStrategy: "semantic",
  maxTokens: 2000,
});

console.log(result.map(m => ({
  id: m.id,
  tier: (m as any).tier, // "qdrant" | "vector" | "fts5"
})));

// Preview retrieval (dry-run) for the Playground UI
import { retrievePreview } from "@/lib/memory/retrieval";

const preview = await retrievePreview(null, "project roadmap", {
  strategy: "hybrid",
  maxTokens: 3000,
  limit: 10,
});

console.log(preview.resolution); // shows which backend (qdrant / sqlite-vec) was used

// Upserting a memory into Qdrant from the insertion pipeline
import { upsertSemanticMemoryPoint } from "@/lib/memory/qdrant";

await upsertSemanticMemoryPoint({
  id: "mem-001",
  apiKeyId: "user-api-key-123",
  sessionId: "sess-456",
  key: "reset-password",
  content: "To reset your password, click the 'Forgot password' link …",
  metadata: {},
  createdAt: new Date().toISOString(),
  expiresAt: null,
});

Summary

• OmniRoute stores all memories in SQLite with an FTS5 virtual table (memory_fts) for fast keyword retrieval, defined in migrations 022_add_memory_fts5.sql and 023_fix_memory_fts_uuid.sql.
• The system supports two vector backends: external Qdrant clusters for production semantic search and sqlite-vec for in-process ANN when settings.vectorStore is configured.
• Retrieval follows a tiered fallback hierarchy: Qdrant → sqlite-vec → FTS5, ensuring availability even when vector stores fail or return empty results.
• Hybrid search combines vector scores with FTS5 results using reciprocal rank fusion (RRF) when using the sqlite-vec backend.
• Engine status is observable via the /api/settings/memory/engine-status endpoint, exposing health of all memory subsystems including optional rerank services.

Frequently Asked Questions

How does OmniRoute handle vector search when Qdrant is unavailable?

When Qdrant is unreachable or returns no results, OmniRoute automatically falls back to the sqlite-vec extension for in-process semantic search. If sqlite-vec is also unavailable or disabled, the system degrades gracefully to pure FTS5 keyword search against the memory_fts virtual table, ensuring continuous operation with the tier marker "fts5".

What is the difference between semantic and hybrid retrieval in OmniRoute?

Semantic retrieval uses vector similarity exclusively to find memories related in meaning to the query, while hybrid retrieval combines vector similarity scores with FTS5 keyword match scores using reciprocal rank fusion. Hybrid mode typically yields better accuracy by considering both conceptual similarity and exact term matches.

Where is the FTS5 virtual table defined in the OmniRoute source code?

The FTS5 virtual table memory_fts is created in the database migration file src/lib/db/migrations/022_add_memory_fts5.sql, with UUID stabilization fixes applied in src/lib/db/migrations/023_fix_memory_fts_uuid.sql. This table indexes the content and key columns of the memories table for fast full-text queries.

How can I check which memory backend is currently active?

Query the /api/settings/memory/engine-status endpoint, which returns the current state of all memory subsystems including whether FTS5, the embedding source, vector store, and Qdrant health checks are active. The response schema is defined in src/shared/schemas/memory.ts and indicates the active retrieval tier.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}