# ai-memory Workspace Crate Responsibilities: A Modular Rust Architecture

> Understand the responsibilities of each Rust crate in the ai-memory workspace. Explore modular architecture for LLM integrations, persistence, and UI.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: architecture
- Published: 2026-09-11

---

**The ai-memory workspace partitions functionality across eleven specialized crates, enforcing strict separation between pure domain logic (core), SQLite persistence (store), atomic file operations (wiki), network protocols (mcp), LLM integrations (llm), automation pipelines (consolidate), and user interfaces (cli).**

The ai-memory repository by akitaonrails implements a knowledge management system for AI agents using a multi-crate Rust workspace. Understanding the specific crate responsibilities within the ai-memory workspace is essential for navigating the codebase, as each crate maintains isolated concerns—from privacy-strip sanitization to single-writer SQLite actors—ensuring the system remains testable, modular, and free of circular dependencies.

## Core Domain and Shared Types

The **ai-memory-core** crate defines the fundamental domain model used throughout the system. It contains pure-logic structs such as `Page`, `Observation`, and `Handoff`, alongside identifier types, error enums, and privacy-strip sanitisation logic in [`src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/sanitize.rs).

This crate performs no I/O operations, making it trivially unit-testable. All other crates depend on the types defined in [`crates/ai-memory-core/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/lib.rs) and its submodules ([`src/page.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/page.rs), [`src/observation.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/observation.rs)), establishing a shared vocabulary for the entire workspace.

## SQLite Persistence and Storage

The **ai-memory-store** crate provides the SQLite persistence layer with a **single-writer actor** pattern. All database writes are funneled through a dedicated thread to guarantee consistency, while connection pooling supports concurrent reads.

Key implementation files include [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) and [`src/writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/writer.rs). The crate implements FTS5 indexing for full-text search and exposes the `ScopeResolver` helper used by the MCP server for scoped data access.

## Atomic Wiki Operations

The **ai-memory-wiki** crate handles atomic markdown writes and file-system watching. It serves as the source-of-truth for page content, implementing durability through the `Wiki::write_page` method—which performs atomic updates using a temporary-file-plus-rename-plus-fsync pattern—and a filesystem watcher that ignores its own writes.

Critical source files include [`crates/ai-memory-wiki/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/lib.rs) and [`src/write.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/write.rs), which ensure that even if the process crashes during a write, the wiki remains in a consistent state.

## MCP Transport and HTTP API

The **ai-memory-mcp** crate implements the **Message-Control-Protocol (MCP)** transport and HTTP server using `axum`. It exposes admin routes, authentication middleware ([`src/auth.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/auth.rs)), and the public API at `/api/v1` defined in [`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs).

All route handlers integrate with the storage layer through `ai_memory_store::ScopeResolver`, ensuring that HTTP requests operate within properly scoped database contexts.

## Hook Processing and Sanitization

The **ai-memory-hooks** crate contains **hook payload schemas** and the sanitiser used by client-side hook bundles. The hook router receives fire-and-forget events, applies sanitization rules defined in [`src/sanitizer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/sanitizer.rs), and forwards clean payloads to the store.

This crate lives in [`crates/ai-memory-hooks/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/lib.rs) and acts as a defensive boundary against malformed or sensitive inbound data.

## LLM Provider Abstractions

The **ai-memory-llm** crate supplies provider-agnostic abstractions for Large Language Model interactions. It defines the `LlmProvider` and `Embedder` traits in [`src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/provider.rs), with concrete implementations for Anthropic, OpenAI, and Gemini.

Credential handling is isolated within this crate via the `figment` configuration library, ensuring that API keys never leak into the domain or persistence layers. The entry point is [`crates/ai-memory-llm/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/lib.rs).

## Auto-Improvement Pipeline

The **ai-memory-consolidate** crate runs the **auto-improvement pipeline**: ingest → lint → sweep → curator → consolidator. This orchestrates nightly or triggered LLM-driven processing that transforms raw observations into curated wiki pages.

The main orchestration logic resides in [`src/consolidator.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/consolidator.rs), while [`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs) provides the public `Consolidator` struct with methods like `new()` and `run_once()`.

## Native Workstream Handling

The **ai-memory-workstream** crate provides native support for the `ai-memory run` command. It manages workstream packets, maintains checkpoints, and offers an in-process transcript API for agents via the `Transcript` struct.

Key files include [`crates/ai-memory-workstream/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-workstream/src/lib.rs) and [`src/transcript.rs`](https://github.com/akitaonrails/ai-memory/blob/main/src/transcript.rs), which implement the `WorkstreamRepository` type for database interactions.

## Command-Line Interface and Tooling

The **ai-memory-cli** crate implements the user-facing command-line interface using `clap` for argument parsing. It loads configuration and dispatches to underlying crates, implementing commands such as `write-page`, `search`, `serve`, `reset`, and `purge-project`.

The main entry point is [`crates/ai-memory-cli/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-cli/src/main.rs), with individual command implementations in `src/commands/*.rs`.

## Testing and Import Utilities

The **ai-memory-test-support** crate supplies test-only utilities including temporary repository fixtures and mock credentials. This crate is marked as a dev-dependency and is excluded from production builds.

Additionally, the **ai-memory-importer** (located in `companions/ai-memory-importer/` rather than the main workspace) functions as a stand-alone tool for importing external conversation data into the ai-memory format.

## Practical Integration Examples

Below are concrete examples demonstrating how these crates interact through public APIs.

### Writing a Page via CLI

This example flows through `ai-memory-cli` → `ai-memory-wiki` → `ai-memory-store`:

```bash
ai-memory write-page \
  --path notes/today.md \
  --content "$(cat <<'EOF'
---
title: "Today's Summary"
---
We processed 42 observations today.
EOF
)"

```

### Programmatic Observation Ingestion

Using `ai-memory-core` for domain types and `ai-memory-store` for persistence:

```rust
use ai_memory_core::{NewObservation, ObservationKind, Observation};
use ai_memory_store::Store;

let obs = NewObservation {
    kind: ObservationKind::Log,
    body: "User clicked button".into(),
    // … other fields …
};

let store = Store::open("data/ai_memory.db")?;
store.insert_observation(obs)?;

```

### Running the Consolidation Pipeline

Using `ai-memory-consolidate` to trigger knowledge-base refinement:

```rust
use ai_memory_consolidate::Consolidator;

let consolidator = Consolidator::new(&config)?;
consolidator.run_once()?;   // pulls unsanitized observations, LLM-processes them, writes pages

```

### Accessing Workstream Transcripts

Using `ai-memory-workstream` for agent session management:

```rust
use ai_memory_workstream::{WorkstreamRepository, Transcript};

let repo = WorkstreamRepository::open("workstreams.db")?;
let transcript = Transcript::load(&repo, workstream_id)?;
println!("{}", transcript.pretty_print());

```

## Summary

- **ai-memory-core** defines the domain model and sanitization logic without I/O dependencies.
- **ai-memory-store** implements single-writer SQLite persistence with FTS5 indexing and connection pooling.
- **ai-memory-wiki** guarantees atomic markdown writes and filesystem watching for the knowledge base.
- **ai-memory-mcp** exposes HTTP APIs and MCP transport using `axum` and scoped resolvers.
- **ai-memory-hooks** sanitizes inbound webhook payloads before storage.
- **ai-memory-llm** abstracts LLM providers and isolates credential handling via `figment`.
- **ai-memory-consolidate** orchestrates the nightly LLM pipeline from raw observations to curated pages.
- **ai-memory-workstream** manages native workstreams and transcript APIs for the `run` command.
- **ai-memory-cli** provides the user interface and command dispatch.
- **ai-memory-test-support** and **ai-memory-importer** handle testing fixtures and data migration respectively.

## Frequently Asked Questions

### What is the responsibility of ai-memory-core within the workspace?

The ai-memory-core crate owns the **domain model**, defining structs like `Page` and `Observation`, error types, and privacy-stripping sanitization logic. It contains no I/O code, making it the foundation upon which all other crates depend for type safety and business logic validation.

### How does ai-memory-store ensure database consistency?

The crate implements a **single-writer actor** pattern where all write operations are funneled through a dedicated thread. This design prevents SQLite write conflicts while allowing concurrent read access through connection pooling, as implemented in [`crates/ai-memory-store/src/writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/writer.rs).

### Which crate handles LLM provider credentials and why?

The **ai-memory-llm** crate isolates credential handling for providers like OpenAI, Anthropic, and Gemini. It uses the `figment` configuration library to load API keys, ensuring that sensitive authentication details remain encapsulated and never leak into the core domain or storage layers.

### How can I programmatically access workstream transcripts?

Use the **ai-memory-workstream** crate, specifically the `WorkstreamRepository` and `Transcript` types. The `Transcript::load()` method retrieves session history from the workstream database, providing an in-process API for agents to inspect their own execution traces.