ai-memory Workspace Crate Responsibilities: A Modular Rust Architecture

The ai-memory workspace partitions functionality across eleven specialized crates, enforcing strict separation between pure domain logic (core), SQLite persistence (store), atomic file operations (wiki), network protocols (mcp), LLM integrations (llm), automation pipelines (consolidate), and user interfaces (cli).

The ai-memory repository by akitaonrails implements a knowledge management system for AI agents using a multi-crate Rust workspace. Understanding the specific crate responsibilities within the ai-memory workspace is essential for navigating the codebase, as each crate maintains isolated concerns—from privacy-strip sanitization to single-writer SQLite actors—ensuring the system remains testable, modular, and free of circular dependencies.

Core Domain and Shared Types

The ai-memory-core crate defines the fundamental domain model used throughout the system. It contains pure-logic structs such as Page, Observation, and Handoff, alongside identifier types, error enums, and privacy-strip sanitisation logic in src/sanitize.rs.

This crate performs no I/O operations, making it trivially unit-testable. All other crates depend on the types defined in crates/ai-memory-core/src/lib.rs and its submodules (src/page.rs, src/observation.rs), establishing a shared vocabulary for the entire workspace.

SQLite Persistence and Storage

The ai-memory-store crate provides the SQLite persistence layer with a single-writer actor pattern. All database writes are funneled through a dedicated thread to guarantee consistency, while connection pooling supports concurrent reads.

Key implementation files include crates/ai-memory-store/src/lib.rs and src/writer.rs. The crate implements FTS5 indexing for full-text search and exposes the ScopeResolver helper used by the MCP server for scoped data access.

Atomic Wiki Operations

The ai-memory-wiki crate handles atomic markdown writes and file-system watching. It serves as the source-of-truth for page content, implementing durability through the Wiki::write_page method—which performs atomic updates using a temporary-file-plus-rename-plus-fsync pattern—and a filesystem watcher that ignores its own writes.

Critical source files include crates/ai-memory-wiki/src/lib.rs and src/write.rs, which ensure that even if the process crashes during a write, the wiki remains in a consistent state.

MCP Transport and HTTP API

The ai-memory-mcp crate implements the Message-Control-Protocol (MCP) transport and HTTP server using axum. It exposes admin routes, authentication middleware (src/auth.rs), and the public API at /api/v1 defined in crates/ai-memory-mcp/src/server.rs.

All route handlers integrate with the storage layer through ai_memory_store::ScopeResolver, ensuring that HTTP requests operate within properly scoped database contexts.

Hook Processing and Sanitization

The ai-memory-hooks crate contains hook payload schemas and the sanitiser used by client-side hook bundles. The hook router receives fire-and-forget events, applies sanitization rules defined in src/sanitizer.rs, and forwards clean payloads to the store.

This crate lives in crates/ai-memory-hooks/src/lib.rs and acts as a defensive boundary against malformed or sensitive inbound data.

LLM Provider Abstractions

The ai-memory-llm crate supplies provider-agnostic abstractions for Large Language Model interactions. It defines the LlmProvider and Embedder traits in src/provider.rs, with concrete implementations for Anthropic, OpenAI, and Gemini.

Credential handling is isolated within this crate via the figment configuration library, ensuring that API keys never leak into the domain or persistence layers. The entry point is crates/ai-memory-llm/src/lib.rs.

Auto-Improvement Pipeline

The ai-memory-consolidate crate runs the auto-improvement pipeline: ingest → lint → sweep → curator → consolidator. This orchestrates nightly or triggered LLM-driven processing that transforms raw observations into curated wiki pages.

The main orchestration logic resides in src/consolidator.rs, while crates/ai-memory-consolidate/src/lib.rs provides the public Consolidator struct with methods like new() and run_once().

Native Workstream Handling

The ai-memory-workstream crate provides native support for the ai-memory run command. It manages workstream packets, maintains checkpoints, and offers an in-process transcript API for agents via the Transcript struct.

Key files include crates/ai-memory-workstream/src/lib.rs and src/transcript.rs, which implement the WorkstreamRepository type for database interactions.

Command-Line Interface and Tooling

The ai-memory-cli crate implements the user-facing command-line interface using clap for argument parsing. It loads configuration and dispatches to underlying crates, implementing commands such as write-page, search, serve, reset, and purge-project.

The main entry point is crates/ai-memory-cli/src/main.rs, with individual command implementations in src/commands/*.rs.

Testing and Import Utilities

The ai-memory-test-support crate supplies test-only utilities including temporary repository fixtures and mock credentials. This crate is marked as a dev-dependency and is excluded from production builds.

Additionally, the ai-memory-importer (located in companions/ai-memory-importer/ rather than the main workspace) functions as a stand-alone tool for importing external conversation data into the ai-memory format.

Practical Integration Examples

Below are concrete examples demonstrating how these crates interact through public APIs.

Writing a Page via CLI

This example flows through ai-memory-cli → ai-memory-wiki → ai-memory-store:

ai-memory write-page \
  --path notes/today.md \
  --content "$(cat <<'EOF'
---
title: "Today's Summary"
---
We processed 42 observations today.
EOF
)"

Programmatic Observation Ingestion

Using ai-memory-core for domain types and ai-memory-store for persistence:

use ai_memory_core::{NewObservation, ObservationKind, Observation};
use ai_memory_store::Store;

let obs = NewObservation {
    kind: ObservationKind::Log,
    body: "User clicked button".into(),
    // … other fields …
};

let store = Store::open("data/ai_memory.db")?;
store.insert_observation(obs)?;

Running the Consolidation Pipeline

Using ai-memory-consolidate to trigger knowledge-base refinement:

use ai_memory_consolidate::Consolidator;

let consolidator = Consolidator::new(&config)?;
consolidator.run_once()?;   // pulls unsanitized observations, LLM-processes them, writes pages

Accessing Workstream Transcripts

Using ai-memory-workstream for agent session management:

use ai_memory_workstream::{WorkstreamRepository, Transcript};

let repo = WorkstreamRepository::open("workstreams.db")?;
let transcript = Transcript::load(&repo, workstream_id)?;
println!("{}", transcript.pretty_print());

Summary

• ai-memory-core defines the domain model and sanitization logic without I/O dependencies.
• ai-memory-store implements single-writer SQLite persistence with FTS5 indexing and connection pooling.
• ai-memory-wiki guarantees atomic markdown writes and filesystem watching for the knowledge base.
• ai-memory-mcp exposes HTTP APIs and MCP transport using axum and scoped resolvers.
• ai-memory-hooks sanitizes inbound webhook payloads before storage.
• ai-memory-llm abstracts LLM providers and isolates credential handling via figment.
• ai-memory-consolidate orchestrates the nightly LLM pipeline from raw observations to curated pages.
• ai-memory-workstream manages native workstreams and transcript APIs for the run command.
• ai-memory-cli provides the user interface and command dispatch.
• ai-memory-test-support and ai-memory-importer handle testing fixtures and data migration respectively.

Frequently Asked Questions

What is the responsibility of ai-memory-core within the workspace?

The ai-memory-core crate owns the domain model, defining structs like Page and Observation, error types, and privacy-stripping sanitization logic. It contains no I/O code, making it the foundation upon which all other crates depend for type safety and business logic validation.

How does ai-memory-store ensure database consistency?

The crate implements a single-writer actor pattern where all write operations are funneled through a dedicated thread. This design prevents SQLite write conflicts while allowing concurrent read access through connection pooling, as implemented in crates/ai-memory-store/src/writer.rs.

Which crate handles LLM provider credentials and why?

The ai-memory-llm crate isolates credential handling for providers like OpenAI, Anthropic, and Gemini. It uses the figment configuration library to load API keys, ensuring that sensitive authentication details remain encapsulated and never leak into the core domain or storage layers.

How can I programmatically access workstream transcripts?

Use the ai-memory-workstream crate, specifically the WorkstreamRepository and Transcript types. The Transcript::load() method retrieves session history from the workstream database, providing an in-process API for agents to inspect their own execution traces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}