# Understanding the Purpose of Each Crate in the ai-memory Repository

> Explore the ai-memory repository and understand the purpose of each Rust crate. Discover how they manage domain types, persistence, LLM integration, and more.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: internals
- Published: 2026-09-09

---

**The akitaonrails/ai-memory project organizes its Rust codebase into twelve single-responsibility crates that separate concerns between core domain types, SQLite persistence, markdown operations, LLM providers, HTTP transport, and user-facing interfaces.**

The akitaonrails/ai-memory repository implements a memory system for AI agents through a strictly layered workspace architecture documented in [`AGENTS.md`](https://github.com/akitaonrails/ai-memory/blob/main/AGENTS.md). Understanding the purpose of each crate in the ai-memory repository reveals how the project enforces boundaries between storage, business logic, and presentation layers while maintaining cross-cutting invariants like single-writer SQLite access and atomic wiki writes.

## Core Foundation Crates

### ai-memory-core: Domain Types and Shared IDs

The **`ai-memory-core`** crate defines the foundational **domain types, errors, and IDs** shared across the entire codebase. Located in `crates/ai-memory-core`, this crate contains pure, no-IO logic implementing types such as `PagePath`, `ActiveProject`, and `Observation` that serve as the contract between all other layers.

Key file: [`crates/ai-memory-core/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/lib.rs) centralizes these types and interfaces.

### ai-memory-store: SQLite Persistence Layer

The **`ai-memory-store`** crate implements the **SQLite storage layer** using a single-writer actor pattern, reader pool, and decay mathematics. It handles all persistence of pages, observations, handoffs, and metadata through the writer actor implementation.

Key file: [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) contains the `Writer` initialization logic.

## Interface and Transport Crates

### ai-memory-wiki: Atomic Markdown Operations

The **`ai-memory-wiki`** crate provides **atomic markdown writes**, file-watcher integration, and Git interaction. All wiki mutations flow through this crate to ensure sanitization, attribution, and index updates remain coherent across the system.

Key file: [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) implements the atomic write logic.

### ai-memory-hooks: Lifecycle Hook Sanitization

The **`ai-memory-hooks`** crate defines **payload schemas** and the **sanitizer** for lifecycle-hook ingestion via the `/hook` endpoint. All untrusted hook data passes through this crate before reaching storage to enforce security boundaries.

Key file: [`crates/ai-memory-hooks/src/sanitizer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/sanitizer.rs) handles payload validation.

### ai-memory-mcp: MCP/HTTP Transport

The **`ai-memory-mcp`** crate exposes the **MCP/HTTP transport** using `rmcp` and `axum`, routing admin interfaces, web UI, and tool-router endpoints. It converts incoming HTTP requests into typed internal calls for the rest of the system.

Key file: [`crates/ai-memory-mcp/src/router.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/router.rs) defines the MCP route structure.

### ai-memory-llm: Provider Abstractions

The **`ai-memory-llm`** crate implements **LLM provider abstractions** through the `LlmProvider` and `Embedder` traits. It manages authentication boundaries for OpenAI, Anthropic, Gemini, and other providers without leaking provider-specific details to consuming crates.

Key file: [`crates/ai-memory-llm/src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/provider.rs) defines the provider interface.

## Processing and Application Crates

### ai-memory-consolidate: Auto-Improvement Pipeline

The **`ai-memory-consolidate`** crate contains the **auto-improvement pipeline** following a Karpathy-style workflow: ingest → lint → sweep → auto-improve. It runs periodic consolidation of stored pages and embeddings to maintain data quality.

Key file: [`crates/ai-memory-consolidate/src/consolidator.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/consolidator.rs) drives the pipeline.

### ai-memory-web: Read-Only Web Interface

The **`ai-memory-web`** crate serves the **read-only web UI** at `/web` and the JSON API at `/api/v1`. It renders the wiki and provides endpoints for external clients without allowing direct mutations to the underlying storage.

Key file: [`crates/ai-memory-web/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-web/src/server.rs) implements the web server.

### ai-memory-workstream: Native Transcripts

The **`ai-memory-workstream`** crate supplies a **read-only native transcript** and the `ai-memory run` command-line adapter that preserves continuity across harnesses. It bridges the gap between the storage layer and CLI execution contexts.

Key file: [`crates/ai-memory-workstream/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-workstream/src/lib.rs) handles transcript operations.

### ai-memory-cli: Binary Entry Point

The **`ai-memory-cli`** crate implements the **`ai-memory` binary**—the primary CLI entry point. It parses arguments, resolves configuration via `Config::load()`, and wires together all other crates for user-facing commands.

Key file: [`crates/ai-memory-cli/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-cli/src/main.rs) serves as the application entry point.

## Testing and Companion Crates

### ai-memory-test-support: Development Utilities

The **`ai-memory-test-support`** crate provides **dev-only test helpers** including temporary directories and fixture builders used by crate-level test suites. It is not included in production builds.

Key file: [`crates/ai-memory-test-support/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-test-support/src/lib.rs) contains the test utilities.

### companions/ai-memory-importer: External Data Import

The **`companions/ai-memory-importer`** crate functions as a **stand-alone importer** for bringing external conversation data into the memory system. Unlike workspace members, it builds independently with its own [`Cargo.toml`](https://github.com/akitaonrails/ai-memory/blob/main/Cargo.toml) and is not a workspace member.

Key file: [`companions/ai-memory-importer/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/companions/ai-memory-importer/src/main.rs) implements the importer CLI.

### evals: End-to-End Testing

The **`evals`** workspace member contains **end-to-end smoke tests, hook shell tests, and fixtures** used exclusively for testing the overall system integration. It validates that the crate composition works correctly across the full stack.

Key file: [`evals/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/evals/src/main.rs) runs the test harness.

## How the Crates Compose

The crates interact through explicit dependency injection, with the store's writer actor serving as the central coordination point. The following patterns demonstrate how the layers stack together.

First, the CLI loads configuration and initializes the storage layer:

```rust
// 1️⃣ Load configuration (ai-memory-cli)
let cfg = ai_memory_cli::config::Config::load()?;

// 2️⃣ Create a writer actor (ai-memory-store)
let (writer, writer_handle) = ai_memory_store::writer::Writer::new(&cfg)?;

```

Next, the wiki layer uses the writer handle to perform atomic operations:

```rust
// 3️⃣ Open the wiki (ai-memory-wiki) – all writes go through this API
let wiki = ai_memory_wiki::Wiki::new(&cfg, writer_handle.clone())?;

// 4️⃣ Store a new page (ai-memory-store + ai-memory-wiki)
let page = ai_memory_core::page::Page::new("example.md", "Hello world!");
wiki.write_page(page)?;   // atomic write + index update

```

Higher-level services consume the same writer handle for background processing:

```rust
// 5️⃣ Run the auto‑improvement loop (ai-memory-consolidate)
let consolidator = ai_memory_consolidate::Consolidator::new(&cfg);
consolidator.run_once()?;   // ingest → lint → embed → sweep

```

Finally, the web server exposes read-only access while the workstream handles CLI continuity:

```rust
// 6️⃣ Start the web server (ai-memory-web)
let server = ai_memory_web::Server::new(&cfg, writer_handle)?;
tokio::spawn(server.serve());

```

This architecture maintains the project's cross-cutting invariants: single-writer SQLite access, atomic wiki writes, strict hook sanitization, and typed boundaries between layers.

## Summary

- **ai-memory-core** provides pure domain types like `PagePath` and `Observation` with no IO dependencies, serving as the foundation for all crates.
- **ai-memory-store** manages the SQLite single-writer actor and persistence logic in [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs).
- **ai-memory-wiki** gates all markdown mutations through atomic writes and Git integration via [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs).
- **ai-memory-mcp** handles HTTP transport and MCP protocol routing using `axum` in [`crates/ai-memory-mcp/src/router.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/router.rs).
- **ai-memory-hooks** sanitizes untrusted payload data before storage ingestion through [`crates/ai-memory-hooks/src/sanitizer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/sanitizer.rs).
- **ai-memory-llm** abstracts provider-specific implementations behind `LlmProvider` and `Embedder` traits.
- **ai-memory-consolidate** runs the automated Karpathy-style improvement pipeline on stored content.
- **ai-memory-web** serves the read-only web UI at `/web` and JSON API at `/api/v1` from [`crates/ai-memory-web/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-web/src/server.rs).
- **ai-memory-workstream** manages native transcript continuity and the `ai-memory run` adapter.
- **ai-memory-cli** wires all crates together as the main binary entry point at [`crates/ai-memory-cli/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-cli/src/main.rs).
- **ai-memory-test-support**, **ai-memory-importer**, and **evals** provide testing utilities, external data import, and integration validation respectively.

## Frequently Asked Questions

### What is the relationship between ai-memory-store and ai-memory-wiki?

The **ai-memory-store** crate provides the low-level SQLite writer actor that handles raw persistence, while **ai-memory-wiki** sits above it to enforce business rules like atomic markdown writes, Git attribution, and index updates. The wiki crate calls into the store crate but adds sanitization and file-watcher coordination that the storage layer does not handle.

### Why does ai-memory-core contain no IO logic?

Keeping **ai-memory-core** free of IO operations allows it to serve as a universal dependency for all other crates without introducing async runtime requirements or storage coupling. Types like `PagePath` and `Observation` defined in [`crates/ai-memory-core/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/lib.rs) can be used in the CLI, web server, and test utilities without dragging in SQLite or HTTP dependencies.

### How does the single-writer SQLite pattern work across crates?

The **ai-memory-store** crate creates a single writer actor during initialization (via `Writer::new()`) and hands out cloneable handles to other crates like **ai-memory-wiki** and **ai-memory-consolidate**. This ensures that only one actor performs writes to the SQLite database, preventing WAL mode conflicts while allowing multiple crates to queue operations through message passing.

### What separates ai-memory-mcp from ai-memory-web?

**ai-memory-mcp** handles the Model Context Protocol (MCP) transport and administrative HTTP routes using `rmcp` and `axum`, exposing tool-router endpoints for AI agent integration. **ai-memory-web** specifically serves the human-facing read-only web UI at `/web` and the JSON API at `/api/v1`, focusing on content rendering rather than protocol negotiation.