How the ai-memory Project Is Structured: A Complete Architecture Guide

The ai-memory repository is a Rust workspace organized into nine specialized crates that implement a capture-consolidate-recall-handoff data flow, enforcing strict architectural invariants like single-writer SQLite access and typed triple identifiers.

The ai-memory project provides persistent, searchable memory for AI agents through a carefully layered architecture. As implemented in akitaonrails/ai-memory, the codebase separates concerns into distinct crates under a unified workspace, ensuring each module has a single responsibility and a typed public API.

Workspace Layout and Crate Organization

The project uses a Cargo workspace defined in Cargo.toml, partitioning functionality into nine crates under the crates/ directory.

Core Domain Crates

Three crates handle the fundamental data model and persistence logic:

• ai-memory-core – Contains domain types, error definitions, and ID structures with no I/O dependencies.
• ai-memory-store – Implements the SQLite writer actor pattern, managing a WriterHandle that serializes all writes through a dedicated thread, alongside the reader pool and decay mathematics for memory relevance.
• ai-memory-wiki – Manages atomic markdown writes to the filesystem and includes a git watcher for version tracking.

Interface and Integration Crates

These crates handle external communication protocols and AI provider abstractions:

• ai-memory-mcp – Implements RMCP transport, a tool router, and admin routes. According to docs/ARCHITECTURE.md, this crate exposes 18 MCP tools for agent integration.
• ai-memory-hooks – Defines payload schemas, request sanitizers, and the /hook HTTP endpoint for receiving observations.
• ai-memory-llm – Provides abstractions for LLM and embedding providers with strict auth boundaries, supporting zero-LLM operation mode.
• ai-memory-web – Serves a read-only web UI and JSON API under /api/v1.

Pipeline and CLI Crates

The ingestion pipeline and user interfaces reside in:

• ai-memory-consolidate – Orchestrates the ingest → lint → sweep → auto-improve pipeline, optionally using LLMs to compile observations into coherent Markdown.
• ai-memory-workstream – Contains native transcript handlers and launch adapters for capturing agent output.
• ai-memory-cli – The ai-memory binary entry point providing subcommands like init, search, and handoff.

Data Directory Architecture

When initialized, the server creates a structured data directory that separates raw inputs from derived indices:

<data_dir>/
├── wiki/    # Markdown source of truth, git-versioned

├── raw/     # Immutable sanitized work-stream transcript segments

├── db/      # SQLite index (FTS5, entities, embeddings)

├── models/  # Reserved for local embedding models

└── logs/    # Rolling tracing output

Hooks POST sanitized observations to the server, which the ai-memory-store writer serializes into wiki pages under wiki/. Retrieval queries execute against the SQLite index in db/, fusing full-text search results with optional vector scores from configured embedding providers.

Runtime Data Flow: Capture to Handoff

The architecture implements a four-phase cognitive cycle for AI agent memory:

Capture Phase

Agent harnesses emit observations via lifecycle hooks located in hooks/pool/. These scripts (available in POSIX and PowerShell variants) POST structured payloads to the server's sanitization layer in ai-memory-hooks.

Consolidation Pipeline

At session end, ai-memory-consolidate compiles raw observations into coherent Markdown pages. This process runs through the pipeline: ingest → lint → sweep → auto-improve, with the optional auto-improve reviewer scheduling proposal approvals as documented in docs/auto-improve-loop.md.

Recall and Search

The recall mechanism combines three retrieval strategies:

1. FTS5 full-text search against the SQLite index
2. Entity matching for structured data retrieval
3. Vector similarity via the Embedder trait in ai-memory-llm, merged using reciprocal rank fusion (RRF)

Source-authority adjustments bias results based on provenance.

Handoff Mechanism

Typed handoff objects deliver the "where-did-we-leave-off?" baton to subsequent sessions. These objects are owned and claimed exactly once, ensuring deterministic session continuity. The CLI command ai-memory handoff --next retrieves the active handoff for the current workspace.

Cross-Cutting Architectural Invariants

The project enforces five strict invariants across all crates, documented in README.md and docs/ARCHITECTURE.md:

• One-writer SQLite actor – All database writes funnel through the WriterHandle in ai-memory-store, preventing connection contention.
• Typed 3-tuple identity – Every row uses a composite key of (workspace_id, project_id, path), enforced by types in ai-memory-core.
• No environment access outside config – Config::load() is the sole entry point for std::env::var calls.
• Atomic file writes – The wiki crate ensures Markdown updates are atomic before git commits.
• Zero circular dependencies – The crate graph is strictly acyclic, with ai-memory-core at the root.

Extensibility and Provider Model

The architecture supports pluggable components through trait-based abstractions:

• LLM providers – Configured via docs/llm-providers.md; the system runs in zero-LLM mode using only local rules.
• Embedding providers – Implemented via the Embedder trait in ai-memory-llm, allowing local or remote vector generation.
• Auto-improve loop – An optional reviewer component in ai-memory-consolidate that schedules proposal approvals without blocking the main pipeline.

Getting Started with the Architecture

Initialize a new memory instance and explore the structure:

# Initialize the data directory structure

ai-memory --data-dir ~/.local/share/ai-memory \
          --config ~/.config/ai-memory/config.toml init

# Install MCP server and hooks for Claude Code

ai-memory install-mcp   --client claude-code --apply
ai-memory install-hooks --agent  claude-code --apply

Search the knowledge base and manage handoffs programmatically:

# Search using FTS5 + entity matching

ai-memory search "authentication token" --limit 5

# Retrieve the next session handoff

ai-memory handoff --next

Interact with the library crates directly:

use ai_memory_store::ScopeResolver;
use ai_memory_wiki::Wiki;

let resolver = ScopeResolver::new(&config)?;
let wiki = Wiki::open(&data_dir)?;
let page = wiki.read_page("projectX/notes.md")?;
println!("Page contents:\n{}", page);

Summary

• Nine-crate workspace with strict separation between domain logic (ai-memory-core), persistence (ai-memory-store), interfaces (ai-memory-mcp), and pipelines (ai-memory-consolidate).
• Git-backed wiki storage with atomic writes in crates/ai-memory-wiki/ and SQLite indexing in crates/ai-memory-store/.
• Single-writer actor pattern ensures safe concurrent access to the SQLite database through WriterHandle.
• Capture-consolidate-recall-handoff flow provides end-to-end memory management for AI agents.
• MCP server architecture exposes 18 tools via ai-memory-mcp, enabling integration with Claude Code and other agents.

Frequently Asked Questions

What is the crate structure in ai-memory?

The project organizes code into nine crates under crates/: ai-memory-core for domain types, ai-memory-store for SQLite persistence with a single-writer actor, ai-memory-wiki for markdown management, ai-memory-mcp for the MCP server, ai-memory-hooks for HTTP endpoints, ai-memory-llm for provider abstractions, ai-memory-consolidate for the processing pipeline, ai-memory-web for the UI, and ai-memory-cli for the command-line interface.

How does ai-memory handle concurrent writes?

All writes serialize through a dedicated SQLite writer actor (WriterHandle in ai-memory-store). This single-writer pattern prevents database locking issues by funneling every mutation through one thread, while read operations use a separate connection pool for concurrent query performance.

What storage backends does ai-memory use?

The system uses SQLite for the search index (with FTS5 and entity tables), filesystem storage for git-versioned Markdown in wiki/ and immutable transcripts in raw/, and optional remote embedding providers (via ai-memory-llm) for vector search. Local embedding models can be stored in the models/ directory.

How does the MCP server fit into the architecture?

The ai-memory-mcp crate implements the server using RMCP transport and exposes 18 tools for agent integration. It serves as the primary interface for the capture-consolidate-recall-handoff cycle, handling tool routing and admin routes while delegating persistence to ai-memory-store and ai-memory-wiki.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}