How OmniRoute Handles Context Extraction, Injection, and Summarization

OmniRoute's memory subsystem processes every LLM interaction through three asynchronous stages—extracting structured facts from responses using regex patterns, injecting relevant memories into outbound requests via provider-aware message formatting, and compressing older entries through token-budget-aware summarization—to maintain persistent, query-aware context without blocking the main request pipeline.

OmniRoute implements a sophisticated memory layer that persists user preferences and decisions across chat sessions by handling context extraction, injection, and summarization through three tightly-coupled modules. The system is defined across src/lib/memory/extraction.ts, src/lib/memory/injection.ts, and src/lib/memory/summarization.ts, ensuring that relevant historical context is automatically harvested, retrieved, and compacted to stay within token budgets while maintaining conversational continuity.

Extracting Facts from LLM Responses

The extraction module in src/lib/memory/extraction.ts scans every LLM response for user-expressed preferences, decisions, and behavioral patterns. When extractFacts receives response text, it applies three groups of regular-expression patterns—PREFERENCE_PATTERNS, DECISION_PATTERNS, and PATTERN_PATTERNS—to identify extractable information.

Pattern Matching and MemoryType Classification

For each regex match, the system sanitizes the text and generates a stable factKey to deduplicate entries within the same batch. Facts are classified by MemoryType: FACTUAL for preferences and patterns, and EPISODIC for decisions. These are emitted as ExtractedFact objects and stored via createMemory with metadata including category, extractedAt, and source: "llm_response".

import { extractFacts } from "@/lib/memory/extraction";

const response = "I really prefer dark mode. I will use the new theme next week.";
await extractFacts(response, "apiKey123", "session456");
// → stores a factual "preference" and an episodic "decision" memory

Non-Blocking Persistence Pipeline

To ensure extraction never blocks the main request-handling pipeline, the asynchronous extractFacts function schedules storage work using setImmediate. This allows the system to persist insights to the SQLite store via createMemory while the conversation continues uninterrupted.

Injecting Memories into Provider Requests

Before an outbound request reaches the LLM provider, the injection module in src/lib/memory/injection.ts pulls relevant memories for the current API key and session, prepending them to the message list via injectMemory. The module formats selected memories into a labeled string ("Memory context: …") and applies provider-specific injection strategies.

import { injectMemory, shouldInjectMemory } from "@/lib/memory/injection";

const request = {
  model: "gpt-4o",
  messages: [{ role: "user", content: "Show me my settings." }],
};

if (shouldInjectMemory(request)) {
  const memories = await getMemories("apiKey123", "session456");
  const enriched = injectMemory(request, memories, "openai");
  // enriched contains a leading system message with formatted memory context
}

Provider-Specific Message Strategies

The injection strategy adapts to provider capabilities. System-message-supported providers receive context as a leading system message (or merged with an existing system message), while providers without system support—such as o1-mini and glm—receive the context as a user message. For providers requiring the system message to be the very first entry (e.g., xiaomi-mimo), the injectSystemFirst helper enforces that rule.

Cache-Aware Context Positioning

When prompt-caching is active via the cacheSafe option, the memory message is inserted just before the last user turn rather than at the beginning. This positioning keeps the cacheable prefix stable, maximizing cache hits while still providing relevant context to the model.

Summarizing and Compacting Historical Context

The summarization module in src/lib/memory/summarization.ts maintains token budgets by compacting older memories. The summarizeMemories function traverses all memories for an API key (optionally scoped to a session) and retains as many as fit within the configurable maxTokens ceiling.

import { summarizeMemoriesOlderThan } from "@/lib/memory/summarization";

const result = await summarizeMemoriesOlderThan("apiKey123", 30, false);
// → creates a single semantic summary memory and removes 30-day-old entries

Token-Ceiling-Aware Compaction

Excess memories are rewritten using generateSummary, which produces a concise three-sentence summary. The function tallies token savings and updates the SQLite store via low-level DB helpers like getDbInstance, createMemory, and deleteMemory.

Semantic Aggregation of Old Memories

For bulk cleanup, summarizeMemoriesOlderThan creates a single "semantic" summary memory that aggregates many old entries, then deletes the originals. This enables a clean-up-while-preserving-knowledge workflow, ensuring long-term user context remains accessible without exhausting context windows.

Summary

• Context extraction in src/lib/memory/extraction.ts uses regex patterns to classify facts as FACTUAL or EPISODIC, storing them asynchronously via setImmediate to avoid blocking.
• Memory injection in src/lib/memory/injection.ts formats and inserts memories using provider-aware strategies, handling system message constraints for xiaomi-mimo, o1-mini, and glm while respecting cacheSafe positioning.
• Summarization in src/lib/memory/summarization.ts enforces maxTokens budgets by rewriting excess memories with generateSummary and supports bulk semantic aggregation via summarizeMemoriesOlderThan.
• The subsystem operates entirely within a SQLite store using helpers from src/lib/memory/store.ts and retrieval logic from src/lib/memory/retrieval.ts.

Frequently Asked Questions

How does OmniRoute prevent duplicate facts from being stored?

The extraction module generates a stable factKey for each matched pattern and removes duplicates within the same batch before calling createMemory. This deduplication occurs in src/lib/memory/extraction.ts during the fact classification phase.

Which providers require special handling during memory injection?

Providers without system message support—such as o1-mini and glm—receive memory context as user messages, while xiaomi-mimo requires the system message to be the very first entry. The injectMemory function in src/lib/memory/injection.ts handles these quirks automatically.

What triggers the memory summarization process?

Summarization can run periodically or on-demand via summarizeMemoriesOlderThan, which targets entries older than a specified day threshold. The process compacts memories to fit within the maxTokens budget by generating three-sentence summaries and deleting original entries.

How does the system ensure extraction doesn't slow down responses?

The extractFacts function uses setImmediate to schedule storage operations asynchronously. This ensures that parsing, factKey generation, and createMemory calls occur outside the main request-handling thread, maintaining low latency for LLM interactions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}