How LLM Consolidations Are Handled in ai-memory: Complete Technical Guide

LLM consolidations in ai-memory are handled by a dedicated Consolidator component that batches session observations, sends them to a configured LLM provider, and atomically writes the results to the wiki.

The ai-memory project implements a sophisticated consolidation pipeline that transforms ephemeral session data into durable, searchable knowledge. This article examines the complete flow—from trigger mechanisms through LLM invocation to atomic wiki commits—based on the actual source code in the akitaonrails/ai-memory repository.

Consolidator Architecture and Core Struct

At the heart of ai-memory's LLM consolidation system is the Consolidator struct, defined in crates/ai-memory-consolidate/src/consolidator.rs at line 67. This component provides a single, typed entry point that isolates LLM interaction from storage concerns.

The Consolidator is constructed with three dependencies:

let consolidator = Arc::new(Consolidator::new(
    store.clone(),
    wiki.clone(),
    llm_provider.clone(),
));

• store — Persists session observations and slots
• wiki — Handles atomic page writes via wiki::write_page
• llm_provider — Dispatches prompts to configured LLM backends

Error handling is centralized through the ConsolidatorError enum (line 25 in the same file), covering cases like empty sessions, missing sessions, and LLM failures.

Three Ways to Trigger Consolidation

The ai-memory system supports multiple entry points for LLM consolidation, ensuring flexibility across manual, automatic, and lifecycle-driven workflows.

1. Manual Invocation via MCP Tool or CLI

The primary manual interface is the memory_consolidate MCP tool, implemented in crates/ai-memory-mcp/src/server.rs at line 373. Users can also invoke consolidation from the command line:

# Consolidate the most recent session of the current project

ai-memory consolidate --project my-project

The MCP tool accepts structured arguments:

{
  "tool": "memory_consolidate",
  "args": {
    "session_id": "0189f2a5-...",
    "project": "my-project",
    "instructions": "Summarize the session as a decision page."
  }
}

2. Automatic Session-End Consolidation

When the environment variable AI_MEMORY_CONSOLIDATE_ON_SESSION_END is set, consolidation runs automatically at session termination. This integration point in crates/ai-memory-hooks/src/router.rs (line 433) connects the consolidator to the hook lifecycle.

3. PreCompact Operations

Consolidation can be triggered during compaction workflows—specifically as a PreCompact step that prepares data before storage optimization runs.

The Consolidation Pipeline: Step by Step

Once triggered, the LLM consolidation pipeline executes eight distinct phases. Each phase is implemented with explicit error boundaries and retry semantics.

Phase 1: Session Resolution

The consolidator first resolves the session ID, workspace, and project context. It verifies the caller's agent kind through resolve_agent_origin. Invalid or missing sessions immediately return ConsolidatorError variants.

Phase 2: Observation Collection

The system gathers session observations and slots, building a ConsolidatedBatch of prompt-ready data. This batch construction is handled by build_batch_request, demonstrated in the A/B evaluation harness at evals/src/ab.rs:

use ai_memory_consolidate::{build_batch_request, ConsolidatedBatch};

let batch: ConsolidatedBatch = build_batch_request(
    &store,
    &wiki,
    session_id,
    /*options=*/ Default::default(),
)?;

Phase 3: System Prompt Assembly

A system-level prompt—BATCH_SYSTEM_PROMPT—is prepended to the batch. Projects can customize this behavior.

Phase 4: LLM Provider Selection

The LLM provider is selected via the factory pattern in crates/ai-memory-llm/src/factory.rs. Supported providers include Anthropic, OpenAI, and others; configuration details are documented in docs/llm-providers.md.

Phase 5: LLM Invocation

The assembled prompt is dispatched to the configured provider. Network errors, rate limits, and malformed responses are captured as ConsolidatorError::Llm.

Phase 6: Response Parsing

The JSON-structured LLM response is parsed into ConsolidationOutcome objects. Each outcome represents a candidate wiki page with metadata.

Phase 7: Atomic Wiki Write

For each outcome, the consolidator creates a WritePageRequest committed atomically via wiki::write_page. This guarantees that partial failures don't leave the wiki in an inconsistent state.

Phase 8: Handoff Emission

A memory_consolidate handoff is emitted, enabling downstream agents to react to new wiki pages immediately.

Durability and Retry Mechanics

When consolidation is opted-in for session-end, the work is queued in a durable retry queue outside the hook response. As documented in docs/usage.md, this ensures consolidation survives process restarts and respects at-least-once delivery semantics.

Customization: Project-Level Prompts

Teams can tailor consolidation behavior without code changes:

Method	Location	Override Capability
Project default	_prompts/consolidation.md	System prompt template
Per-call	instructions argument	Task-specific guidance

The instructions parameter in memory_consolidate takes precedence over project defaults, enabling dynamic context injection.

Key Source Files Reference

Understanding LLM consolidation in ai-memory requires familiarity with these components:

• crates/ai-memory-consolidate/src/consolidator.rs — Core Consolidator struct, error types, and main algorithm
• crates/ai-memory-mcp/src/server.rs (line 373) — MCP server wiring and memory_consolidate endpoint
• crates/ai-memory-hooks/src/router.rs (line 433) — Hook lifecycle integration
• crates/ai-memory-llm/src/factory.rs — LLM provider selection
• docs/usage.md — CLI usage and environment variables
• docs/llm-providers.md — Provider configuration
• evals/src/ab.rs — Batch request construction examples

Summary

• Consolidator is the central abstraction for LLM-driven knowledge extraction in ai-memory
• Three triggers exist: MCP tool/CLI, session-end automation, and PreCompact hooks
• The pipeline guarantees atomic wiki writes and durable retry for session-end jobs
• Project-level prompts (_prompts/consolidation.md) and per-call instructions enable customization without code changes
• All LLM provider configuration is factory-based and documented in docs/llm-providers.md

Frequently Asked Questions

What happens if the LLM fails during consolidation?

The consolidator wraps LLM errors as ConsolidatorError::Llm and propagates them to the caller. For session-end consolidations, the retry queue ensures eventual execution; for manual calls, the error surfaces through the MCP tool response or CLI exit code.

How do I configure which LLM provider handles consolidation?

LLM providers are configured in docs/llm-providers.md. The factory in crates/ai-memory-llm/src/factory.rs instantiates the appropriate client based on configuration flags—supporting Anthropic, OpenAI, and extensible to additional backends.

Can I run consolidation without the MCP server?

Yes. The CLI command ai-memory consolidate provides direct access to the same Consolidator logic used by the MCP server, accepting --project and other flags documented in docs/usage.md.

What's the difference between consolidation and embedding generation?

Consolidation produces structured wiki pages from session observations using LLM reasoning. Embeddings (handled by separate components) create vector representations for semantic search. Both can use the same LLM provider but serve distinct purposes in the knowledge pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}