# How the Session Consolidation Process Compiles Observations into Wiki Pages in ai-memory

> Understand ai-memory's session consolidation process. Learn how observations are compiled into wiki pages via capture, rule-based summarization, and LLM-driven consolidation.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: internals
- Published: 2026-08-19

---

**The session consolidation process in ai-memory transforms transient agent observations into durable wiki pages through a three-stage pipeline: capture via lifecycle hooks, rule-based session summarization, and LLM-driven consolidation into semantic knowledge families.**

The session consolidation process is the core mechanism by which the akitaonrails/ai-memory repository turns fleeting coding session data into permanent, searchable knowledge. Every agent interaction feeds a steady-state loop that culminates in structured markdown output under the project's wiki directory. This pipeline lets operators maintain an accurate, versioned knowledge base without manual documentation overhead.

## Three Stages of the Session Consolidation Process

### Capture via Lifecycle Hooks

Agent CLIs emit short-lived HTTP POST requests to the `/hook` endpoint throughout a coding session. Each payload carries sanitized observations such as *session-start*, *user-prompt*, *post-tool-use*, and *session-end*. The server's hook router sanitizes the incoming payload, assigns an `ObservationKind`, and pushes a `WriteCmd` to the single-writer SQLite actor for durable storage.

This steady-state loop is documented in [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), which defines how raw observations enter the system before any summarization occurs.

### Rule-Based Session Summary Generation

When the server processes a `session-end` event, it synthesizes a canonical session record at `sessions/<id>.md`. This summary page is generated entirely from stored observations using a deterministic, rule-based template. No LLM participates in this stage; the output is predictable and reproducible from the SQLite observation log.

The trigger for this step resides in [`hooks/opencode/session-end.sh`](https://github.com/akitaonrails/ai-memory/blob/main/hooks/opencode/session-end.sh). The resulting markdown file becomes the canonical record of the session and seeds the context for subsequent hand-offs.

### LLM-Driven Consolidation

If an LLM provider is configured via the `AI_MEMORY_LLM_PROVIDER` environment variable, the system invokes the `memory_consolidate` tool automatically or on demand. The **Consolidator** struct, defined in [`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs), reads the session summary, assembles a prompt containing the source material and related wiki pages, and delegates rewriting to the configured LLM.

The consolidation prompt uses `ProjectNameStrategy` and enforces chunk budgeting to stay within token limits. Output is written to semantic families under the wiki directory, such as `concepts/`, `decisions/`, and `gotchas/`, with an optional consolidated page placed in the original session location.

The MCP server registers this tool in [`crates/ai-memory-mcp/src/routes/api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/routes/api.rs), which also enforces the LLM-provider gate before allowing execution.

## End-to-End Data Flow

The complete session consolidation process follows a strict sequence:

1. The agent CLI fires `/hook` requests until the session ends.
2. The `session-end` handler writes a rule-based summary to `sessions/<id>.md`.
3. If `AI_MEMORY_LLM_PROVIDER` is set, the `memory_consolidate` tool rewrites the content into richer wiki pages.
4. Every consolidation pass creates a new Git commit in `<data_dir>/wiki/`, making the compiled knowledge immutable and versioned.
5. SQLite triggers keep the search index synchronized with the latest wiki state.

Because the wiki directory serves as the single source of truth, previous versions remain reachable through Git history while the active index always reflects the newest consolidation output.

## Running Consolidation via CLI and MCP

You can interact with the session consolidation process through the CLI, the MCP tool surface, or programmatically in Rust.

### Command-Line Invocation

First, ensure the `session-end` hook has produced a summary page. Then run the LLM-driven consolidation:

```bash

# Finalize the session to guarantee the summary exists

ai-memory finalize-session --project myproject

# Run LLM-driven consolidation (requires AI_MEMORY_LLM_PROVIDER)

ai-memory memory_consolidate \
  --project myproject \
  --multi_page=true \
  --max_input_tokens=100000 \
  --max_output_tokens=32000

```

### MCP Tool Call

MCP clients can invoke consolidation directly via the registered tool:

```json
{
  "tool": "memory_consolidate",
  "arguments": {
    "project": "myproject",
    "targets": ["concepts/", "decisions/", "gotchas/"],
    "multi_page": true,
    "max_input_tokens": 100000,
    "max_output_tokens": 32000
  }
}

```

### Programmatic Rust Usage

Internally, the server and CLI delegate to the `Consolidator` struct exposed by [`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs):

```rust
use ai_memory_consolidate::{Consolidator, ConsolidatorError};

async fn run_consolidation(db: &Db, project_id: ProjectId) -> Result<(), ConsolidatorError> {
    let consolidator = Consolidator::new(db.clone());
    consolidator
        .consolidate_project(project_id, /*auto_approve=*/ true)
        .await
}

```

## Key Files in the Consolidation Architecture

Several source files define the session consolidation process end to end:

- **[`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md)** — Documents the steady-state observation loop and the consolidation step.
- **[`hooks/opencode/session-end.sh`](https://github.com/akitaonrails/ai-memory/blob/main/hooks/opencode/session-end.sh)** — Triggers creation of the rule-based `sessions/<id>.md` summary.
- **[`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs)** — Implements the `Consolidator` struct, prompt construction, chunk budgeting, and page write logic.
- **[`crates/ai-memory-mcp/src/routes/api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/routes/api.rs)** — Exposes the `memory_consolidate` tool and enforces the LLM-provider configuration gate.
- **[`crates/ai-memory-cli/src/main.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-cli/src/main.rs)** — Parses the `memory_consolidate` subcommand and forwards arguments to the MCP layer.

## Summary

- The session consolidation process begins when agent lifecycle hooks POST observations to `/hook`, which are stored as `ObservationKind` records via a single-writer SQLite actor.
- A deterministic, rule-based template in [`hooks/opencode/session-end.sh`](https://github.com/akitaonrails/ai-memory/blob/main/hooks/opencode/session-end.sh) generates `sessions/<id>.md` without LLM involvement.
- The `memory_consolidate` tool, gated by `AI_MEMORY_LLM_PROVIDER`, uses the `Consolidator` struct to rewrite session summaries into semantic wiki families.
- Every consolidation pass commits immutable output to `<data_dir>/wiki/` and syncs the SQLite index through triggers.

## Frequently Asked Questions

### What triggers the session consolidation process?

The process starts automatically when an agent CLI sends a `session-end` observation to the `/hook` endpoint. This event causes the server to write a rule-based summary page. If an LLM provider is configured, the `memory_consolidate` tool then triggers the LLM-driven rewrite.

### Does session consolidation require an LLM provider?

No. The initial stage that compiles observations into `sessions/<id>.md` is fully deterministic and requires no LLM. However, the enrichment pass that generates multi-page semantic wiki output under `concepts/`, `decisions/`, and `gotchas/` requires `AI_MEMORY_LLM_PROVIDER` to be set.

### Where are consolidated wiki pages stored?

All consolidated output lives in `<data_dir>/wiki/`. Each consolidation pass creates a new Git commit, so the wiki remains versioned and immutable. The SQLite search index stays synchronized with these files via database triggers.

### How does ai-memory prevent data loss during consolidation?

The system treats the wiki directory as the single source of truth and uses Git for versioning. Because the `Consolidator` writes through Git commits, prior states are preserved. Meanwhile, the SQLite actor processes `WriteCmd` operations sequentially, ensuring the observation log remains consistent even during concurrent hook traffic.