How ai-memory's Zero-LLM Mode Works: Complete Feature Set Without an LLM

ai-memory operates fully without an LLM by using synthetic compression for data ingestion, BM25/FTS5 search ranking, and SQL-based structured briefings, making all core features available offline and cost-free.

The akitaonrails/ai-memory repository is designed to function entirely without configuring a large-language-model provider. This zero-LLM mode ensures deterministic, offline operation while preserving essential memory management capabilities through lightweight synthetic compression and database-native operations.

Synthetic Default Compression Strategy

When operating in zero-LLM mode, ai-memory employs a synthetic-default-compression approach instead of LLM-driven summarization. According to the design documentation in docs/research-agentmemory.md at line 101, this strategy extracts obvious structural elements—titles, file names, and short narratives derived from raw tool I/O—without invoking external models.

This compression preserves searchability while eliminating token costs. The system stores observations from hooks, CLI commands, or API calls as-is, applying only lightweight structural extraction to keep storage efficient.

RRF-Only Search Architecture

Zero-LLM search bypasses vector embeddings entirely. The implementation in crates/ai-memory-mcp/src/server.rs at line 1730 explicitly documents "RRF-only, zero-LLM behaviour," utilizing pure SQL aggregation and BM25-based FTS5 ranking.

When you execute memory_query, the system returns results through reciprocal rank fusion (RRF) of database rows without LLM reranking. The reranker module at crates/ai-memory-llm/src/reranker.rs line 9 contains a guard clause that returns inputs unchanged when no LLM provider is configured, ensuring zero latency from model inference.

Structured Briefings Without Model Calls

The memory_briefing function generates comprehensive activity snapshots entirely from database queries. As documented in docs/design-decisions.md at line 213, this returns structured data including 7-day and 30-day activity windows, pending handoffs, recent pages, and _rules/ contents—no LLM invocation required.

Similarly, memory_explore attempts LLM-driven prose generation only when configured. The fallback logic in crates/ai-memory-mcp/src/server.rs lines 3707-3711 emits a warning and returns the same structured snapshot when no provider exists, ensuring the feature remains functional.

Available Zero-LLM Features

Observation Capture stores raw input from hooks and CLI commands without transformation.

Query and Search operates via BM25/FTS5 keyword matching through the memory_query command.

Rule-Based Linting validates duplicate titles and broken cross-references through memory_lint --skip-llm, as the contradiction detection pass is explicitly optional per the server comment at crates/ai-memory-mcp/src/server.rs line 367.

Manual Consolidation allows memory_consolidate execution, though automatic LLM-driven contradiction resolution is bypassed.

Features Requiring LLM Configuration

Only three capabilities require an LLM provider: vector-based semantic search reranking, automatic contradiction detection during consolidation, and LLM-driven prose generation in exploration digests. The default build ships with only the built-in rusqlite store, avoiding heavy ONNX/torch dependencies as noted in docs/issues-mempalace.md at line 183.

Practical Zero-LLM Usage Examples

All commands function without setting AI_MEMORY_LLM_* environment variables:

# Store observation via CLI (zero synthetic compression)

ai-memory hook observe "User opened file src/main.rs"

# Structured briefing (pure SQL aggregation)

ai-memory memory_briefing --project myproj --window 7d

Example output:

{
  "activity_7d": 42,
  "pending_handoffs": [],
  "recent_pages": ["README.md", "src/main.rs"],
  "_rules": [...]
}

# Keyword search (BM25/FTS5 ranking)

ai-memory memory_query "rust async"

# Rule-based lint without LLM

ai-memory memory_lint --project myproj --skip-llm

# HTTP API query (read-only, zero-LLM)

curl http://localhost:49374/api/v1/memory_query \
     -d '{"query":"error handling"}' \
     -H "Content-Type: application/json"

Summary

• Zero-LLM mode uses synthetic compression instead of LLM summarization, extracting only titles, file names, and tool I/O narratives.
• Search operates via BM25/FTS5 with RRF-only ranking, bypassing vector embeddings and LLM rerankers entirely.
• Structured briefings return database snapshots (7-day/30-day windows, handoffs, rules) without model invocation.
• Core features—observation capture, keyword search, rule-based linting, and manual consolidation—remain fully functional offline.
• Optional dependencies like ONNX and torch are excluded from the default build, which ships with only rusqlite storage.

Frequently Asked Questions

Can I use ai-memory completely offline without any API keys?

Yes. ai-memory is designed to function entirely without network connectivity or LLM providers. The default rusqlite storage backend requires no external services, and all core features—including observation capture, BM25 search, and structured briefings—operate locally using SQL queries and file system operations.

What search capabilities do I lose in zero-LLM mode?

You lose only semantic vector search and LLM-based result reranking. The system falls back to BM25/FTS5 keyword search with reciprocal rank fusion, which handles exact and partial keyword matching efficiently. The reranker at crates/ai-memory-llm/src/reranker.rs line 9 passes through results unchanged when no LLM is configured, maintaining query functionality without semantic understanding.

Does zero-LLM mode affect data storage or compression?

Data storage remains unchanged, but compression strategy shifts from LLM-driven summarization to synthetic default compression. As implemented in the codebase, this extracts structural metadata (titles, file names) from observations without natural language processing. Your raw observations remain intact and searchable, though they lack the condensed prose summaries that LLM compression would generate.

How do I verify that a command is running in zero-LLM mode?

Execute commands without setting AI_MEMORY_LLM_* environment variables. The system will emit warnings for features that attempt LLM fallbacks, such as memory_explore, which explicitly notes the fallback to structured snapshots in crates/ai-memory-mcp/src/server.rs lines 3707-3711. Successful execution without API key errors confirms zero-LLM operation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}