How ai-memory Captures Agent Lifecycle Events: The Complete Hook Pipeline

ai-memory captures agent lifecycle events through a tightly-coupled hook pipeline that ingests HTTP JSON payloads, sanitizes sensitive data, and persists structured observations to SQLite while generating searchable wiki pages.

The akitaonrails/ai-memory repository implements a comprehensive event capture system that transforms fleeting agent interactions into persistent knowledge. Understanding how agent lifecycle events flow from runtime hooks to database storage requires examining the multi-stage pipeline spanning shell scripts, Axum routers, and storage actors.

The Hook Pipeline Architecture

The system processes agent lifecycle events—including SessionStart, PostToolUse, UserPromptSubmit, SessionEnd, and Stop—through a sequence of seven distinct stages. Each stage ensures data privacy, bounded latency, and eventual consistency while maintaining non-blocking behavior for agent clients.

Agent-Side Hook Scripts

Every supported agent ships with lightweight shell shims located in hooks/pool/ (e.g., hooks/pool/session-start.sh, hooks/pool/post-tool-use.sh). These scripts execute synchronously whenever the agent reaches a lifecycle milestone.

The shim constructs a JSON payload containing the raw event data, agent identifier, current working directory, and optional session ID. It then transmits this data via HTTP POST to the server's /hook endpoint, effectively bridging the agent process with the ai-memory service.

HTTP Ingestion and Routing

The server exposes an Axum router defined in crates/ai-memory-hooks/src/router.rs that handles both single-event POST /hook and bulk POST /hook/batch endpoints.

The router parses the query string into a HookQuery struct (defined in crates/ai-memory-hooks/src/payload.rs) and deserializes the JSON body into a HookEnvelope. Critical query parameters include event type, agent identifier, ingest_key (for deduplication), and capture_assistant flags that request assistant content inclusion.

Sanitization and Capture Policy

Before persistence, every payload passes through ai_memory_core::Sanitizer and the CapturePolicy logic in crates/ai-memory-hooks/src/capture_policy.rs.

The sanitiser strips private data, truncates large bodies according to the global OBSERVATION_BODY_MAX_BYTES limit, and applies privacy rules. The capture policy then determines storage behavior: full capture, metadata-only reduction, or complete dropping—particularly for sub-agent events when the drop_subagent flag is enabled.

Observation Creation

Sanitized events become NewObservation instances defined in crates/ai-memory-core/src/observation.rs. The system maps HookEvent variants to ObservationKind enums—for example, SessionStart maps to ObservationKind::SessionStart, while PostToolUse becomes ObservationKind::ToolUse.

This type transformation creates a strongly-typed internal representation that the storage layer consumes, ensuring schema consistency across different agent implementations.

Persistence and Wiki Integration

The router hands observations to the writer actor (ai_memory_store::WriterHandle) implemented in crates/ai-memory-store/src/lib.rs. This actor performs two atomic operations:

1. Writes a row into the observations SQLite table.
2. Creates a handoff for SessionStart events that generates an initial wiki page via synthesize_session_page.

The synthesize_session_page function in crates/ai-memory-hooks/src/synth.rs assembles markdown from session metadata, persisting it atomically via ai_memory_wiki::Wiki::write_page. This page serves as the durable source-of-truth and enables full-text search indexing.

Notably, the router returns HTTP 202 immediately upon accepting the event, while the actual database write and wiki indexing occur asynchronously afterward. This guarantees non-blocking client behavior while preserving transactional consistency between the observation store and search indexes.

Performance and Safety Mechanisms

Several mechanisms ensure reliable ingestion without blocking agents.

Bounded ingestion limits concurrent processing via DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT. Excess requests receive HTTP 429 responses, allowing clients to implement backoff or drop semantics.

Batch ingestion via POST /hook/batch accepts up to MAX_HOOK_BATCH_ITEMS (256) events, enabling efficient queue draining for high-throughput scenarios.

Idempotency is guaranteed through the ingest_key query parameter, preventing duplicate processing during network retries.

Sub-agent filtering recognizes markers defined in SUBAGENT_MARKER_KEYS within crates/ai-memory-hooks/src/payload.rs. When drop_subagent is configured, the system filters these events to prevent noise from nested agent processes while preserving parent session continuity.

Practical Code Examples

Sending a single lifecycle event from an agent:

curl -X POST "http://localhost:49374/hook?event=session-start&agent=claude-code" \
     -H "Content-Type: application/json" \
     -d '{
           "session_id":"0199f3d2-1c4e-7a10-9f3b-2b0c5d8e7a11",
           "cwd":"/home/user/project",
           "prompt":"You are a helpful assistant."
         }'

Batch ingestion using a Rust client:

use reqwest::Client;
use serde_json::json;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let client = Client::new();
    let payload = json!([{
        "event": "post-tool-use",
        "session_id": "0199f3d2-…",
        "tool_name": "search",
        "output": "found 3 results"
    }]);
    let resp = client
        .post("http://localhost:49374/hook/batch?event=post-tool-use&agent=claude-code")
        .json(&payload)
        .send()
        .await?;
    println!("status: {}", resp.status());
    Ok(())
}

Querying captured events via MCP:

ai-memory memory-recent --agent claude-code --workspace default --project my-app

Summary

• Hook scripts in hooks/pool/ capture events at the agent level and POST to /hook with session metadata and working directory context.
• Axum routers in router.rs ingest events as HookEnvelope structures with HookQuery metadata, supporting both single and batch (/hook/batch) ingestion.
• Sanitization in capture_policy.rs enforces privacy limits, truncates to OBSERVATION_BODY_MAX_BYTES, and applies CapturePolicy rules for sub-agent filtering.
• Observations in observation.rs transform raw hooks into typed NewObservation instances with corresponding ObservationKind variants.
• Writer actors persist data to SQLite and generate wiki pages via synthesize_session_page for session visibility and search indexing.
• Bounded ingestion via DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT and batch support for 256 items ensures predictable memory usage and latency.
• Idempotency keys and sub-agent markers provide operational safety for distributed agent deployments.

Frequently Asked Questions

What types of agent lifecycle events does ai-memory capture?

ai-memory captures SessionStart, PostToolUse, UserPromptSubmit, SessionEnd, and Stop events. Each event type maps to a specific ObservationKind variant in crates/ai-memory-core/src/observation.rs, enabling differentiated storage strategies for different interaction phases.

How does ai-memory prevent duplicate event processing?

The system implements idempotency through the ingest_key query parameter defined in HookQuery. When retrying failed requests, agents supply the same key, allowing the server to deduplicate identical payloads in the ingestion pipeline and prevent double-counting of lifecycle milestones.

What happens when the ingestion queue is saturated?

When concurrent requests exceed DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT as defined in router.rs, the server responds with HTTP 429 (Too Many Requests). This non-blocking behavior allows agents to continue operating while choosing to either retry with exponential backoff or drop the event, maintaining system stability under load.

How are sub-agent events handled differently from parent agent events?

The router recognizes sub-agent markers defined in SUBAGENT_MARKER_KEYS within payload.rs. Projects can configure drop_subagent policies to filter these events entirely, reducing noise from nested agent processes while preserving parent agent session continuity and observability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}