How Does the memory_query Tool Work for Searching Memory in ai-memory

The memoryquery CLI utility executes a multi-stage retrieval pipeline that parses free-form queries, resolves project scopes via the ScopeResolver API, searches SQLite FTS5 indexes, optionally re-ranks results with vector embeddings using cosine-similarity RRF, and returns structured output while maintaining strict workspace isolation.

The memoryquery command in the akitaonrails/ai-memory repository provides ad-hoc search capabilities over the persistent memory store. Written in Rust, this tool enables developers to retrieve relevant observations and pages through a hybrid search architecture that combines full-text indexing with optional semantic vector ranking.

The memory_query Search Pipeline

The tool implements a five-stage pipeline that transforms raw input into structured results. Each stage is isolated to specific modules within the workspace crate structure.

Input Parsing and CLI Entry

The search process begins at crates/ai-memory-cli/src/lib.rs, where the memoryquery subcommand is implemented. The parser accepts either a free-form query string or a JSON-encoded query object containing raw text, metadata filters (e.g., author, time ranges), and ranking directives. This input normalization ensures that downstream components receive a standardized query representation regardless of how the user invoked the CLI.

Scope Resolution and Project Isolation

Before executing any lookup, the query must resolve to a concrete Scope (workspace + project). The ScopeResolver API in crates/ai-memory-core/src/routing_snippet.rs handles this mapping, guaranteeing that searches remain isolated per project. This resolution step prevents cross-contamination between workspaces by ensuring every subsequent database operation includes the resolved workspace_id and project_id as mandatory filter predicates.

Full-Text Indexing with SQLite FTS5

The primary search mechanism leverages SQLite FTS5 full-text indexes defined in the data layer. In crates/ai-memory-store/src/search.rs, the normalized query text is fed to the FTS5 engine, which executes a TF-IDF-based ranking to return an initial list of matching observations. This stage operates against the read-only connection pool, ensuring that concurrent ingest operations never block query execution.

Optional Vector-Based Re-Ranking

When an embedding provider is configured, the raw FTS5 results undergo semantic enrichment. The ai-memory-consolidate crate at crates/ai-memory-consolidate/src/embed.rs computes vector embeddings for the candidate results and applies reciprocal-rank fusion (RRF) with cosine-similarity scoring. This hybrid approach boosts semantically relevant results that might not contain exact keyword matches, blending lexical and vector search signals into a unified ranking.

Result Post-Processing and Formatting

The final stage occurs in crates/ai-memory-web/src/templates.rs, where the hit list is trimmed to the requested limit and enriched with metadata including author, timestamps, and file paths. The renderer supports both JSON output for programmatic consumption and human-readable table formatting for CLI display.

Architecture Safeguards During Query Execution

The memory_query implementation respects three critical invariants defined in the core architecture:

• Single-writer SQLite actor: All query reads traverse the read-only connection pool, while writes are serialized through a dedicated actor. This design ensures that search operations never block or encounter locks from concurrent ingestion processes.

• Typed identifiers: Every search result carries the triple identifier (workspace_id, project_id, path), enforcing strict cross-project isolation at the data level.

• Atomic page views: Results reference pages that are always accessed via the Wiki::write_page and Wiki::apply_batch APIs. This guarantees that memoryquery never reads partially written or corrupted page states, even during active batch updates to crates/ai-memory-core/src/page.rs and crates/ai-memory-core/src/observation.rs.

Practical Usage Examples

Execute searches using the ai-memory binary with the memoryquery subcommand:

# Basic free-text search with JSON output

ai-memory memoryquery "project kickoff"

# → {"hits":[{"path":"docs/PROJECT_OVERVIEW.md","score":1.42,"author":"alice"}...]}

# Filter by author and time range

ai-memory memoryquery --author bob --since 2024-01-01 "performance regression"

# Enable vector-based semantic re-ranking (requires embedding provider)

ai-memory memoryquery --embed "optimizing latency"

Summary

• The memoryquery tool is implemented in crates/ai-memory-cli/src/lib.rs and provides a CLI interface to the ai-memory search infrastructure.
• Searches execute through a scoped resolution phase using ScopeResolver in crates/ai-memory-core/src/routing_snippet.rs to enforce project isolation.
• Full-text retrieval uses SQLite FTS5 indexes via crates/ai-memory-store/src/search.rs with TF-IDF scoring.
• Optional semantic re-ranking occurs in crates/ai-memory-consolidate/src/embed.rs using cosine-similarity RRF fusion.
• Output formatting is handled by crates/ai-memory-web/src/templates.rs, supporting both JSON and table views.
• The architecture guarantees non-blocking reads through a single-writer SQLite actor and atomic page access patterns.

Frequently Asked Questions

How does memory_query prevent searches from leaking across projects?

The tool leverages the ScopeResolver API in crates/ai-memory-core/src/routing_snippet.rs to bind every query to a specific workspace and project identifier before execution. This scope resolution ensures that all subsequent SQL queries include mandatory workspace_id and project_id filters, providing hard isolation boundaries at the database level.

Can the tool perform vector searches without FTS5?

No. According to the source implementation, vector-based re-ranking in crates/ai-memory-consolidate/src/embed.rs operates as a secondary enrichment layer on top of the initial FTS5 result set. The SQLite FTS5 index provides the candidate pool, which is then re-ordered using cosine-similarity scoring if embeddings are available.

Where are the data structures for search results defined?

The core entities returned by memoryquery are defined in crates/ai-memory-core/src/observation.rs and crates/ai-memory-core/src/page.rs. These files implement the persistent structs that represent memory observations and wiki pages, including the metadata fields (author, timestamps, paths) attached to each search result.

How does memory_query handle concurrent database writes?

The implementation uses a single-writer SQLite actor pattern where all query operations read from a dedicated read-only connection pool. This architecture, enforced in the store layer, ensures that active ingestions or consolidations never block search queries, and that memoryquery always reads consistent, committed states from the underlying storage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}