Understanding the Purpose of Each Crate in the ai-memory Repository

The akitaonrails/ai-memory project organizes its Rust codebase into twelve single-responsibility crates that separate concerns between core domain types, SQLite persistence, markdown operations, LLM providers, HTTP transport, and user-facing interfaces.

The akitaonrails/ai-memory repository implements a memory system for AI agents through a strictly layered workspace architecture documented in AGENTS.md. Understanding the purpose of each crate in the ai-memory repository reveals how the project enforces boundaries between storage, business logic, and presentation layers while maintaining cross-cutting invariants like single-writer SQLite access and atomic wiki writes.

Core Foundation Crates

ai-memory-core: Domain Types and Shared IDs

The ai-memory-core crate defines the foundational domain types, errors, and IDs shared across the entire codebase. Located in crates/ai-memory-core, this crate contains pure, no-IO logic implementing types such as PagePath, ActiveProject, and Observation that serve as the contract between all other layers.

Key file: crates/ai-memory-core/src/lib.rs centralizes these types and interfaces.

ai-memory-store: SQLite Persistence Layer

The ai-memory-store crate implements the SQLite storage layer using a single-writer actor pattern, reader pool, and decay mathematics. It handles all persistence of pages, observations, handoffs, and metadata through the writer actor implementation.

Key file: crates/ai-memory-store/src/lib.rs contains the Writer initialization logic.

Interface and Transport Crates

ai-memory-wiki: Atomic Markdown Operations

The ai-memory-wiki crate provides atomic markdown writes, file-watcher integration, and Git interaction. All wiki mutations flow through this crate to ensure sanitization, attribution, and index updates remain coherent across the system.

Key file: crates/ai-memory-wiki/src/wiki.rs implements the atomic write logic.

ai-memory-hooks: Lifecycle Hook Sanitization

The ai-memory-hooks crate defines payload schemas and the sanitizer for lifecycle-hook ingestion via the /hook endpoint. All untrusted hook data passes through this crate before reaching storage to enforce security boundaries.

Key file: crates/ai-memory-hooks/src/sanitizer.rs handles payload validation.

ai-memory-mcp: MCP/HTTP Transport

The ai-memory-mcp crate exposes the MCP/HTTP transport using rmcp and axum, routing admin interfaces, web UI, and tool-router endpoints. It converts incoming HTTP requests into typed internal calls for the rest of the system.

Key file: crates/ai-memory-mcp/src/router.rs defines the MCP route structure.

ai-memory-llm: Provider Abstractions

The ai-memory-llm crate implements LLM provider abstractions through the LlmProvider and Embedder traits. It manages authentication boundaries for OpenAI, Anthropic, Gemini, and other providers without leaking provider-specific details to consuming crates.

Key file: crates/ai-memory-llm/src/provider.rs defines the provider interface.

Processing and Application Crates

ai-memory-consolidate: Auto-Improvement Pipeline

The ai-memory-consolidate crate contains the auto-improvement pipeline following a Karpathy-style workflow: ingest → lint → sweep → auto-improve. It runs periodic consolidation of stored pages and embeddings to maintain data quality.

Key file: crates/ai-memory-consolidate/src/consolidator.rs drives the pipeline.

ai-memory-web: Read-Only Web Interface

The ai-memory-web crate serves the read-only web UI at /web and the JSON API at /api/v1. It renders the wiki and provides endpoints for external clients without allowing direct mutations to the underlying storage.

Key file: crates/ai-memory-web/src/server.rs implements the web server.

ai-memory-workstream: Native Transcripts

The ai-memory-workstream crate supplies a read-only native transcript and the ai-memory run command-line adapter that preserves continuity across harnesses. It bridges the gap between the storage layer and CLI execution contexts.

Key file: crates/ai-memory-workstream/src/lib.rs handles transcript operations.

ai-memory-cli: Binary Entry Point

The ai-memory-cli crate implements the ai-memory binary—the primary CLI entry point. It parses arguments, resolves configuration via Config::load(), and wires together all other crates for user-facing commands.

Key file: crates/ai-memory-cli/src/main.rs serves as the application entry point.

Testing and Companion Crates

ai-memory-test-support: Development Utilities

The ai-memory-test-support crate provides dev-only test helpers including temporary directories and fixture builders used by crate-level test suites. It is not included in production builds.

Key file: crates/ai-memory-test-support/src/lib.rs contains the test utilities.

companions/ai-memory-importer: External Data Import

The companions/ai-memory-importer crate functions as a stand-alone importer for bringing external conversation data into the memory system. Unlike workspace members, it builds independently with its own Cargo.toml and is not a workspace member.

Key file: companions/ai-memory-importer/src/main.rs implements the importer CLI.

evals: End-to-End Testing

The evals workspace member contains end-to-end smoke tests, hook shell tests, and fixtures used exclusively for testing the overall system integration. It validates that the crate composition works correctly across the full stack.

Key file: evals/src/main.rs runs the test harness.

How the Crates Compose

The crates interact through explicit dependency injection, with the store's writer actor serving as the central coordination point. The following patterns demonstrate how the layers stack together.

First, the CLI loads configuration and initializes the storage layer:

// 1️⃣ Load configuration (ai-memory-cli)
let cfg = ai_memory_cli::config::Config::load()?;

// 2️⃣ Create a writer actor (ai-memory-store)
let (writer, writer_handle) = ai_memory_store::writer::Writer::new(&cfg)?;

Next, the wiki layer uses the writer handle to perform atomic operations:

// 3️⃣ Open the wiki (ai-memory-wiki) – all writes go through this API
let wiki = ai_memory_wiki::Wiki::new(&cfg, writer_handle.clone())?;

// 4️⃣ Store a new page (ai-memory-store + ai-memory-wiki)
let page = ai_memory_core::page::Page::new("example.md", "Hello world!");
wiki.write_page(page)?;   // atomic write + index update

Higher-level services consume the same writer handle for background processing:

// 5️⃣ Run the auto‑improvement loop (ai-memory-consolidate)
let consolidator = ai_memory_consolidate::Consolidator::new(&cfg);
consolidator.run_once()?;   // ingest → lint → embed → sweep

Finally, the web server exposes read-only access while the workstream handles CLI continuity:

// 6️⃣ Start the web server (ai-memory-web)
let server = ai_memory_web::Server::new(&cfg, writer_handle)?;
tokio::spawn(server.serve());

This architecture maintains the project's cross-cutting invariants: single-writer SQLite access, atomic wiki writes, strict hook sanitization, and typed boundaries between layers.

Summary

• ai-memory-core provides pure domain types like PagePath and Observation with no IO dependencies, serving as the foundation for all crates.
• ai-memory-store manages the SQLite single-writer actor and persistence logic in crates/ai-memory-store/src/lib.rs.
• ai-memory-wiki gates all markdown mutations through atomic writes and Git integration via crates/ai-memory-wiki/src/wiki.rs.
• ai-memory-mcp handles HTTP transport and MCP protocol routing using axum in crates/ai-memory-mcp/src/router.rs.
• ai-memory-hooks sanitizes untrusted payload data before storage ingestion through crates/ai-memory-hooks/src/sanitizer.rs.
• ai-memory-llm abstracts provider-specific implementations behind LlmProvider and Embedder traits.
• ai-memory-consolidate runs the automated Karpathy-style improvement pipeline on stored content.
• ai-memory-web serves the read-only web UI at /web and JSON API at /api/v1 from crates/ai-memory-web/src/server.rs.
• ai-memory-workstream manages native transcript continuity and the ai-memory run adapter.
• ai-memory-cli wires all crates together as the main binary entry point at crates/ai-memory-cli/src/main.rs.
• ai-memory-test-support, ai-memory-importer, and evals provide testing utilities, external data import, and integration validation respectively.

Frequently Asked Questions

What is the relationship between ai-memory-store and ai-memory-wiki?

The ai-memory-store crate provides the low-level SQLite writer actor that handles raw persistence, while ai-memory-wiki sits above it to enforce business rules like atomic markdown writes, Git attribution, and index updates. The wiki crate calls into the store crate but adds sanitization and file-watcher coordination that the storage layer does not handle.

Why does ai-memory-core contain no IO logic?

Keeping ai-memory-core free of IO operations allows it to serve as a universal dependency for all other crates without introducing async runtime requirements or storage coupling. Types like PagePath and Observation defined in crates/ai-memory-core/src/lib.rs can be used in the CLI, web server, and test utilities without dragging in SQLite or HTTP dependencies.

How does the single-writer SQLite pattern work across crates?

The ai-memory-store crate creates a single writer actor during initialization (via Writer::new()) and hands out cloneable handles to other crates like ai-memory-wiki and ai-memory-consolidate. This ensures that only one actor performs writes to the SQLite database, preventing WAL mode conflicts while allowing multiple crates to queue operations through message passing.

What separates ai-memory-mcp from ai-memory-web?

ai-memory-mcp handles the Model Context Protocol (MCP) transport and administrative HTTP routes using rmcp and axum, exposing tool-router endpoints for AI agent integration. ai-memory-web specifically serves the human-facing read-only web UI at /web and the JSON API at /api/v1, focusing on content rendering rather than protocol negotiation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}