How Session Summaries Are Generated in ai-memory Without an LLM

ai-memory creates deterministic, machine-readable session summaries as JSON files by aggregating pre-sanitized observation records from SQLite, eliminating the need for LLM inference during the core summary generation pipeline.

The akitaonrails/ai-memory repository implements a lightweight, deterministic approach to session summarization that avoids the latency and cost of large language model calls. By leveraging structured storage flags and targeted database queries, the system produces summary.json artifacts through pure data transformation rather than generative AI.

The Deterministic Summary Pipeline

The session summary generation follows a strict four-stage pipeline that operates entirely within the Rust-based storage and workstream layers. Each stage is designed to work without neural network inference, ensuring predictable performance and reproducible outputs.

Step 1: Capturing Summary Events via the /hook Endpoint

Every event sent to the /hook endpoint undergoes sanitization before storage in the SQLite-backed observation store. When the event payload carries a summary type—such as records with "type": "summary" or "type": "compact"—the storage layer marks these rows with is_summary_message = 1.

In crates/ai-memory-store/src/ops.rs, the schema definition around line 3460 establishes this boolean flag within the observations table. This flag serves as the primary filter mechanism for later retrieval, ensuring that only relevant summary content enters the aggregation pipeline.

Step 2: Triggering Session-End Processing

When a session terminates, the hook router in crates/ai-memory-hooks/src/router.rs activates the session-end handling path. Even for boundary-only sessions that produce no other artifacts, the system forces a write of the session-summary page. This guarantees that every session—regardless of complexity—outputs a baton file for subsequent sessions to discover.

This design ensures continuity across session boundaries without requiring semantic analysis of the session content.

Step 3: Querying Summary Records from the Store

The retrieval logic resides in crates/ai-memory-store/src/reader.rs within the ReaderPool::session_summary_scoped method. This function queries the datastore for all rows belonging to a specific session, workstream, and project combination where is_summary_message != 0.

The query extracts only the parts column containing the raw text payload. No tokenization, embedding generation, or LLM-based compression occurs during this phase. The method returns a scoped collection of summary records ready for JSON serialization.

Step 4: Assembling the summary.json Output

The final assembly occurs in crates/ai-memory-workstream/src/transcript.rs. This module reads the collected rows and extracts the first available field from each payload—checking for "summary", "content", or "text" keys in priority order.

The resulting structure follows this exact schema:

{
  "info": { 
    "session_id": "...",
    "timestamp": "..."
  },
  "summary": [
    { "timestamp": "2026-08-21T12:00:00Z", "summary": "User completed the onboarding flow." },
    { "timestamp": "2026-08-21T12:05:00Z", "summary": "System performed cleanup operations." }
  ]
}

The file is written atomically to <session-dir>/summary.json using fs::write, ensuring that partial writes never corrupt the session state. The grok_discovery_matches_checkout_via_summary_json_not_bucket_name test in the same file validates that this JSON artifact alone enables session discovery without scanning the full markdown journal.

Key Implementation Files

The LLM-free summary generation spans four critical source files:

• crates/ai-memory-store/src/ops.rs (line ~3460) — Defines the is_summary_message column schema for flagging summary records
• crates/ai-memory-store/src/reader.rs — Implements session_summary_scoped for targeted summary retrieval
• crates/ai-memory-hooks/src/router.rs — Handles session-end triggers and boundary-only session processing
• crates/ai-memory-workstream/src/transcript.rs — Assembles and writes the final summary.json with atomic file operations

The ai-memory-llm crate exists within the repository, but according to the source code, it is reserved exclusively for auto-improve and embedding pipelines. Basic session-summary generation never imports or calls this module.

Practical Code Examples

Querying Summary Records Programmatically

let summary = store
    .reader()
    .await?
    .session_summary_scoped(ws, proj, session_id, OwnerFilter::Any)
    .await?;
println!("Session {} has {} summary entries", session_id, summary.len());

This pattern retrieves all flagged summary messages for a specific session scope without loading the full observation history.

Reading the Generated Summary JSON

use std::fs;
let path = session_dir.join("summary.json");
let raw = fs::read_to_string(path)?;
let json: serde_json::Value = serde_json::from_str(&raw)?;
println!("First summary line: {}", json["summary"][0]["summary"]);

Injecting Custom Summary Events

Clients can populate the summary pipeline by sending structured payloads to the hook endpoint:

{
  "type": "summary",
  "timestamp": "2026-08-21T12:00:00Z",
  "message": "User completed the onboarding flow."
}

The storage layer automatically flags these events with is_summary_message = 1, ensuring they appear in the final summary.json without additional processing.

Summary

• ai-memory generates session summaries through deterministic JSON assembly rather than LLM inference
• The is_summary_message flag in the SQLite schema filters relevant records during storage in crates/ai-memory-store/src/ops.rs
• The ReaderPool::session_summary_scoped method in crates/ai-memory-store/src/reader.rs retrieves only summary-flagged rows
• Assembly logic in crates/ai-memory-workstream/src/transcript.rs extracts text fields and writes atomic summary.json files
• The ai-memory-llm crate handles embeddings and improvements, but never participates in basic summary generation
• Session discovery can occur via summary.json alone, as verified by the grok_discovery_matches_checkout_via_summary_json_not_bucket_name test

Frequently Asked Questions

Why does ai-memory avoid using an LLM for session summaries?

Large language models introduce non-deterministic latency, token costs, and potential hallucinations. By contrast, the deterministic pipeline using is_summary_message flags and JSON aggregation in crates/ai-memory-workstream/src/transcript.rs produces instant, reproducible summaries suitable for machine-readable baton passing between sessions.

Can the summary.json format be customized?

The current implementation in crates/ai-memory-workstream/src/transcript.rs follows a fixed schema extracting "summary", "content", or "text" fields. While the core structure is standardized for discovery compatibility, the system accepts arbitrary payload fields in the input events sent to the /hook endpoint. Custom fields beyond the standard keys will be preserved in the SQLite store but may not appear in the automatically extracted summary array.

How does session discovery work without reading the full journal?

The grok_discovery_matches_checkout_via_summary_json_not_bucket_name test demonstrates that the presence of summary.json in the session directory provides sufficient metadata for session identification. Workstream tools can locate and validate sessions by reading this single JSON file rather than parsing potentially large markdown journals, significantly reducing I/O overhead.

What happens if no summary events are captured during a session?

Even boundary-only sessions that generate no explicit summary events trigger a session-end write through crates/ai-memory-hooks/src/router.rs. The resulting summary.json will contain an empty "summary": [] array, ensuring the session remains discoverable and the baton-handling mechanism functions correctly for workflow continuity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}