How ai-memory Implements the Karpathy-Style Consolidation Pipeline for Observations to Wiki Pages

ai-memory implements a three-stage consolidation pipeline that transforms raw session observations into durable, version-controlled markdown wiki pages by collecting observations into bounded prompts, using structured LLM completion, and atomically writing results with Git-based supersession.

The akitaonrails/ai-memory project provides a Rust-based memory system inspired by Andrej Karpathy's LLM wiki concept: ephemeral observations accumulate during agent sessions, then get periodically distilled into clean, enduring knowledge pages. This article breaks down exactly how the ai-memory-consolidate crate implements this pipeline.

What Is the Karpathy-Style Consolidation Pattern?

Karpathy's original vision treats LLM sessions as producing raw, stream-of-consciousness observations that must be periodically rewritten into structured, persistent documentation. The key characteristics are:

• Immutable raw data: observations are append-only and never modified
• LLM-powered rewrite: a capable model condenses noise into signal
• Versioned supersession: new pages replace old ones while preserving history
• Atomic commits: the entire operation succeeds or rolls back cleanly

ai-memory realizes this through a dedicated consolidation crate that bridges the observation store, LLM provider, and wiki subsystem.

Stage 1: Collecting and Projecting Observations

The pipeline begins in projection.rs with the Consolidator fetching session data via ReaderPool::observations_for_session.

The project_observations function transforms raw observations into a compact text representation that respects configurable character budgets. Each observation body gets trimmed to safe size, ensuring the final prompt fits within LLM context windows.

Key implementation in [crates/ai-memory-consolidate/src/projection.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/projection.rs):

// Observations are chronologically ordered and size-trimmed
let projected = project_observations(observations, budget);
// Result: "2024-01-15T09:23:00Z [user]: Started debugging the auth flow..."

This projection step is critical for the Karpathy pattern—it prevents prompt overflow while preserving temporal structure and attribution.

Stage 2: Building the LLM Request

The prompt assembly logic lives in [consolidator.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/consolidator.rs). Two system prompt templates ship with the crate:

• single_consolidate_system.md: for consolidating one page at a time
• batch_consolidate_system.md: for multi-page updates up to five slots

These compile-time embedded prompts define the exact contract the LLM must follow.

Request Structure

The build_request and build_batch_request_with_slots functions assemble:

1. Prefix: session ID and metadata
2. Observation dump: the projected content from Stage 1
3. Context slot: current page body (single) or slot snapshots (batch)
4. Custom instructions: optional _prompts/consolidation.md per project
5. JSON schema suffix: mandatory output format specification

Prompt budgeting via PromptBudgets (see line 70 of consolidator.rs) ensures the total payload stays within provider limits.

let request = build_request(
    session_id,
    projected_observations,
    current_body,           // None for new pages
    project_instructions,   // Optional custom guidance
    budgets,                // PromptBudgets { total: 100_000, response: 32_000 }
);

Stage 3: LLM Completion and Wiki Write

The structured request flows through ai_memory_llm::complete_structured, which enforces response schema compliance. The LLM must return:

• ConsolidatedPage (single): title, body, tags, description
• ConsolidatedBatch (multi): vector of ConsolidatedPageUpdate objects

Front-Matter Construction and Atomic Commit

Before writing, build_frontmatter stamps each page with:

fn build_frontmatter(page: &ConsolidatedPage, session_id: Uuid, agent_kind: &str) -> BTreeMap<String, FrontMatterValue> {
    let mut fm = BTreeMap::new();
    fm.insert("session_id".into(), session_id.to_string().into());
    fm.insert("consolidated".into(), true.into());
    fm.insert("agent_kind".into(), agent_kind.into());
    fm.extend(page.tags.iter().map(|t| (format!("tag:{}", t), true.into())));
    fm
}

The Wiki component then handles atomic storage:

• Wiki::write_page for single-page consolidation
• Wiki::apply_batch for multi-page atomic updates

Both trigger wiki.commit_all("consolidate(session ...)"), creating a new Git revision that preserves the supersession chain.

Single-Page Consolidation Flow

The consolidate_session method orchestrates the complete pipeline in [consolidator.rs lines 31–225](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/consolidator.rs):

// 1. Fetch observations
let observations = reader.observations_for_session(session_id).await?;

// 2. Resolve workspace/project
let (ws, proj) = resolve_target(&writer, workspace_id, project_id).await?;

// 3. Preflight admission check
wiki.preflight_admission(&ws, &proj, &path).await?;

// 4. Read existing page body if present
let current_body = wiki.read_page(&ws, &proj, &path).await.ok().map(|p| p.body);

// 5. Build and send LLM request
let request = build_request(session_id, observations, current_body, instructions, budgets);
let page: ConsolidatedPage = llm.complete_structured(request).await?;

// 6. Assemble and write
let frontmatter = build_frontmatter(&page, session_id, agent_kind);
let id = wiki.write_page(WritePageRequest {
    workspace_id: ws.id,
    project_id: proj.id,
    path: page.title.to_slug(),
    body: page.body,
    frontmatter,
}).await?;

// 7. Atomic commit
wiki.commit_all(&format!("consolidate({})", session_id)).await?;

Lines 62–90, 119–130, 140–158, and 176–194 contain the specific implementation segments.

Multi-Page Batch Consolidation

For long sessions producing disparate knowledge, consolidate_session_multi extends the pipeline with slot-aware batching. Key additions in [consolidator.rs lines 218–380](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/consolidator.rs):

1. Slot snapshot retrieval: fetches pinned memory contexts via curator.rs integration
2. Batch prompt construction: build_batch_request_with_slots prepares a single prompt returning up to five page updates
3. Atomic batch write: Wiki::apply_batch commits all updates or none

The slot mechanism lets operators pin critical facts that should persist across consolidations—essential for maintaining continuity in long-running agent workflows.

Practical Usage Example

use ai_memory_consolidate::Consolidator;
use ai_memory_store::{ReaderPool, WriterHandle};
use ai_memory_wiki::Wiki;
use std::sync::Arc;
use ai_memory_llm::providers::OpenAiProvider;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Initialize core components
    let reader: ReaderPool = /* configured elsewhere */;
    let writer: WriterHandle = /* configured elsewhere */;
    let wiki = Wiki::new(/* config */);
    let llm = Arc::new(OpenAiProvider::new("gpt-4o-mini")?);

    // Create consolidator with optional limits
    let consolidator = Consolidator::new(
        reader, writer, wiki, llm,
        workspace_id, project_id
    ).with_prompt_limits(100_000, 32_000);

    // Single-page consolidation
    let outcome = consolidator
        .consolidate_session(session_id, false, actor_ctx, None, None)
        .await?;
    println!("Page written: {}", outcome.path);

    // Multi-page batch for complex sessions
    let batch = consolidator
        .consolidate_session_multi(session_id, false, actor_ctx, None, None)
        .await?;
    for update in batch {
        println!("Updated {} ({:?})", update.path, update.page_id);
    }

    Ok(())
}

Error Handling and Observability

All failure modes wrap in ConsolidatorError:

• Store errors: observation fetch or metadata resolution failures
• Wiki errors: preflight rejection, write conflicts, or commit failures
• LLM errors: structured completion parse failures or schema violations

The dry-run parameter (false in examples above) previews the operation without persisting changes—essential for testing prompt changes or custom instructions.

Summary

• ai-memory's consolidation pipeline implements Karpathy's vision through three tightly coupled stages: observation projection, structured LLM prompting, and atomic wiki writes
• The ai-memory-consolidate crate provides both single-page (consolidate_session) and multi-page batch (consolidate_session_multi) variants
• Prompt budgeting and compile-time templates ensure reliable operation across different LLM providers and context window sizes
• Git-based atomic commits preserve complete version history while enabling clean supersession of outdated knowledge
• Slot snapshots in multi-page mode support operator-pinned memory for continuity across long agent sessions

Frequently Asked Questions

What makes this "Karpathy-style" specifically?

The term refers to Andrej Karpathy's described pattern of treating LLM session outputs as raw observations that get periodically rewritten into clean, durable wiki pages. ai-memory matches this with immutable observation logs, LLM-powered condensation, and versioned markdown storage where new pages atomically replace old ones while preserving Git history.

How does prompt budgeting prevent context overflow?

The PromptBudgets struct (defined in consolidator.rs line 70) enforces hard limits on total input tokens and reserved response capacity. The project_observations function aggressively trims observation bodies until the projected content fits within budget.total - budget.response, ensuring the LLM request always succeeds.

Can I customize the consolidation behavior per project?

Yes. The pipeline checks for _prompts/consolidation.md in each project root and appends its contents to the system prompt. This lets project owners inject domain-specific guidance (e.g., "prefer bullet points" or "always include code examples") without modifying the core templates.

What happens if the LLM returns malformed JSON?

complete_structured validates responses against the ConsolidatedPage or ConsolidatedBatch schema defined in types.rs. Parse failures propagate as ConsolidatorError::Llm variants, allowing callers to retry with exponential backoff or escalate to manual review.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}