How the Capture Process and Lifecycle Hooks Work in ai-memory

The ai-memory system automatically records tool-lifecycle events through a seven-stage capture pipeline that sanitizes, persists, and indexes observations via SQLite and markdown-backed storage.

The akitaonrails/ai-memory repository implements an automatic observation recorder that intercepts agent-tool interactions through MCP-compatible lifecycle hooks. Every supported event—from SessionStart to PostToolUse—flows through a centralized pipeline that guarantees atomic storage, privacy-aware sanitization, and full-text searchability. Understanding this architecture is essential for developers who need to debug agent behavior, audit tool usage, or implement custom capture exclusion rules.

The ai-memory Capture Pipeline Overview

The capture process transforms raw tool-lifecycle events into queryable, version-controlled observations. At its core, the system relies on a hook router (ai-memory-hooks crate) that processes incoming JSON payloads, applies safety constraints, and delegates persistence to a single-writer SQLite actor.

The pipeline executes six distinct phases: hook registration, event emission, payload sanitization, database persistence, wiki mutation, and handoff creation. Each phase is isolated to ensure that destructive operations cannot interleave with active captures.

Step 1: Hook Registration and Installation

Clients enable capture by installing MCP hooks into their agent configuration. The install-hooks CLI command writes hook definitions—including SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, PostToolUseFailure, and Stop—into the agent’s configuration files.

During installation, ai-memory automatically detects the nearest marker file and applies capture-exclusion rules defined in the [capture] section. These rules prevent sensitive files from being recorded in the observation stream.

# Install the default lifecycle hooks for the current agent

ai-memory install-hooks --agent $(ai-memory which-agent)

# Verify hook configuration (example for Claude Code)

cat ~/.claude-code/config.toml | grep '\[\[hooks\]\]' -A 6

Step 2: Event Emission and Payload Transmission

When a hooked event fires, the agent constructs a JSON payload containing the tool name, arguments, stdout/stderr streams, and timestamps. This payload is transmitted to the hook router defined in crates/ai-memory-hooks/src/router.rs.

The router serves as the entry point for all capture operations, receiving raw HookPayload structures and initiating the sanitization phase before any data reaches persistent storage.

Step 3: Payload Sanitization and Size Enforcement

Before persistence, the raw payload passes through the sanitiser implemented in crates/ai-memory-hooks/src/sanitizer.rs. The sanitize() function strips potentially unsafe data and enforces a strict size bound of approximately 2 KB per observation, producing a typed Sanitized<NewObservation> value.

This step ensures that the capture pipeline respects privacy constraints and prevents storage overflow from verbose tool outputs. The sanitization logic also applies capture-exclusion patterns defined in the marker file to filter sensitive content.

// Simplified flow inside ai-memory-hooks/src/router.rs
async fn handle_hook(payload: HookPayload) -> Result<()> {
    // 1️⃣ Sanitize the raw payload
    let sanitized = sanitizer::sanitize(payload)?;
    // 2️⃣ Send to the writer actor (single‑writer SQLite)
    writer::send(WriterCommand::InsertObservation(sanitized)).await?;
    // 3️⃣ The writer will also call wiki::write_page to persist the markdown
    Ok(())
}

Step 4: Atomic Persistence to SQLite

Sanitized observations are handed to the single-writer SQLite actor located in crates/ai-memory-store/src/writer.rs. This actor serializes all write operations through a single thread, ensuring race-free atomic transactions.

The writer executes a single INSERT transaction into the observations table, which triggers automatic updates to the FTS5 full-text search index. Because all database mutations flow through this actor, the system maintains consistency even under high-concurrency hook emissions.

Step 5: Wiki Mutation and Markdown Generation

Following the database write, the Wiki::write_page routine in crates/ai-memory-wiki/src/write.rs creates a markdown page under the project’s UUID-keyed directory structure: <wiki_root>/<workspace_id>/<project_id>/observations/....

This markdown file serves as the source of truth for the observation. The writer actor then emits a watcher event that triggers re-indexing for full-text search, ensuring that file-system state remains synchronized with the SQLite database.

Step 6: Session Handoff Creation

Certain lifecycle events—particularly SessionEnd—generate handoff payloads that enable context resumption across agent restarts. These handoffs are stored in the handoffs table and automatically injected into new sessions at SessionStart.

This mechanism allows the next session to resume with the previous context without manual state transfer, creating a continuous memory stream across distinct agent invocations.

Lifecycle Safety and Mutation Gates

All destructive commands—such as purge-project and move-project—must acquire the wiki mutation gate before modifying the file system or database. This coordination primitive guarantees that no concurrent hook-driven writes can interleave with destructive operations, preventing data corruption during project moves or deletions.

The safety matrix and isolation semantics are documented in docs/lifecycle-ops.md, which defines how the capture pipeline coordinates with administrative lifecycle operations.

Summary

• ai-memory captures tool-lifecycle events through MCP hooks installed via the install-hooks CLI.
• The hook router (ai-memory-hooks/src/router.rs) accepts payloads and delegates to the sanitiser (sanitizer.rs), which enforces a ~2 KB size limit and removes unsafe data.
• A single-writer SQLite actor (ai-memory-store/src/writer.rs) provides atomic, race-free persistence to the observations table with automatic FTS5 indexing.
• Markdown wiki pages (ai-memory-wiki/src/write.rs) serve as the durable source of truth under UUID-keyed directories.
• Session handoffs stored in the handoffs table enable automatic context resumption across agent restarts.
• Mutation gates prevent destructive operations from corrupting active capture streams.

Frequently Asked Questions

How do I exclude sensitive files from the ai-memory capture process?

Define exclusion patterns in the [capture] section of your nearest marker file (documented in docs/marker-file.md). When you run ai-memory install-hooks, the system automatically respects these rules during both hook registration and payload sanitization, ensuring that matching files never appear in observations.

What happens if an observation exceeds the 2 KB size limit?

The sanitize() function in ai-memory-hooks/src/sanitizer.rs truncates or redacts content to enforce the size bound, returning a Sanitized<NewObservation> value that fits within the constraint. This prevents storage bloat while preserving the essential context of the tool invocation.

Where are captured observations physically stored?

Observations persist in two locations: the observations table in data/db/memory.sqlite (including FTS5 indexes for full-text search), and as markdown files in <wiki_root>/<workspace_id>/<project_id>/observations/. The SQLite database provides fast querying, while the markdown wiki serves as the human-readable, version-controllable source of truth.

How does ai-memory prevent data loss during project moves or purges?

All destructive operations acquire the wiki mutation gate before executing. This lock ensures that the single-writer SQLite actor cannot process new observations while a move-project or purge-project command is mutating the file system or database schema, guaranteeing atomicity and preventing partial writes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}