How ai-memory Handles Session Finalization Without a Reliable Session-End Hook

ai-memory uses a durable SQLite-backed job queue to defer session finalization, allowing manual CLI triggers and recovery from missed hooks without data loss.

The akitaonrails/ai-memory project addresses session finalization without a reliable session-end hook by persisting consolidation jobs to a database rather than processing them immediately. Instead of depending on ephemeral hook execution, the system stores work items in a session_consolidation_jobs table, enabling the application to survive crashes, missed events, and environments where hooks cannot be trusted. This architecture guarantees that session observations are eventually consolidated regardless of whether the session-end hook fires.

The Database-Backed Queue Architecture

At the heart of the system is the session_consolidation_jobs table, which acts as a durable work queue. This design decouples the detection of session termination from the computationally expensive LLM-based consolidation process.

Enqueueing Logic in session_consolidation.rs

The enqueue function in crates/ai-memory-store/src/session_consolidation.rs inserts a new job only when a session meets specific criteria: it must have a non-empty observation set and an ended_at timestamp. This validation prevents incomplete or empty sessions from entering the queue.

The function also implements intelligent deduplication. If a newer observation generation appears before the worker processes the job, older pending entries are marked superseded via an UPDATE … SET state = 'superseded' clause. This ensures that workers never waste cycles on stale data when newer observations exist.

// Queue a job when a session ends (called by hook or CLI)
let inserted = session_consolidation::enqueue(
    &mut conn,
    workspace_id,
    project_id,
    session_id,
)?;
// `inserted == true` ⇢ new job created

Manual and Automated Triggers

Because the queue persists in SQLite, finalization does not require an immediate hook invocation. The system provides multiple pathways to enqueue a job.

The CLI Fallback Command

When the session-end hook fails or is unavailable, users can manually trigger finalization using the ai-memory finalize-session command. Implemented in crates/ai-memory-cli/src/commands/finalize_session.rs, this command resolves the session identifiers and invokes the same enqueue routine used by automated hooks.

// crates/ai-memory-cli/src/commands/finalize_session.rs
fn run(opts: FinalizeOpts) -> Result<()> {
    // Resolve IDs, open DB...
    session_consolidation::enqueue(&mut conn, ws_id, proj_id, sess_id)?;
    Ok(())
}

This CLI mechanism guarantees that session data can be consolidated even in headless environments or when IDE integrations fail to trigger the hook.

Optional Hook Integration

While not required, ai-memory supports optional hook scripts (session-end.sh and session-end.ps1 in the hooks/ directory). When these scripts execute successfully, they trigger the same enqueue logic. However, their failure or absence never blocks finalization, as the CLI command and automatic retry mechanisms provide alternative entry points to the queue.

Asynchronous Worker Processing

The consolidation work happens asynchronously through a background worker, ensuring that session-end events return immediately without waiting for LLM processing.

Claiming and Executing Jobs

The worker logic resides in crates/ai-memory-store/src/writer.rs (lines 811-830). The session-consolidation worker repeatedly calls claim_next to lock the oldest pending job, preventing race conditions in multi-process scenarios. After running the LLM-based consolidation, the worker marks the job complete using the complete function, or transitions it to fail or release states on error.

// Worker claims the oldest due job
if let Some(job) = session_consolidation::claim_next(&mut conn, now, stale_before)? {
    // ... run LLM consolidation ...
    session_consolidation::complete(&mut conn, &job)?;
}

Because claim_next operates against the durable SQLite store, the system survives process restarts and crashes. Jobs remain in the session_consolidation_jobs table until explicitly finalized, providing at-least-once execution semantics with automatic recovery.

Summary

• Durable queue: The session_consolidation_jobs table persists finalization work across crashes and restarts.
• Conditional enqueueing: The enqueue function in session_consolidation.rs validates sessions and marks stale jobs as superseded to prevent duplicate work.
• CLI fallback: The ai-memory finalize-session command in finalize_session.rs provides a manual trigger independent of hooks.
• Asynchronous processing: The worker in writer.rs uses claim_next and complete to process jobs without blocking session termination.

Frequently Asked Questions

What happens if the session-end hook never fires?

The system does not require the hook to function. Users can manually enqueue consolidation jobs using the ai-memory finalize-session CLI command, which calls the same enqueue function used by hooks. Because jobs persist in SQLite, they remain queued until the background worker processes them, regardless of how they were enqueued.

How does ai-memory prevent duplicate consolidation jobs?

The enqueue function automatically marks older pending jobs as superseded when a newer observation generation exists for the same session. This UPDATE … SET state = 'superseded' logic ensures that workers only process the most current session state, eliminating redundant LLM calls.

Which database table stores the consolidation jobs?

Jobs are stored in the session_consolidation_jobs table within the SQLite database. This table tracks job state—including pending, superseded, completed, and failed statuses—providing durability and visibility into the finalization pipeline.

Where is the worker logic that processes these jobs?

The session-consolidation worker logic resides in crates/ai-memory-store/src/writer.rs (lines 811-830). This module invokes claim_next to acquire exclusive locks on pending jobs and complete to finalize them after LLM-based consolidation succeeds.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}