How Claude-Mem's Lifecycle Hooks Capture and Store Observations: A Deep Dive

Claude-Mem uses Claude Code hooks to initialize sessions and forward prompts, while an event-driven SDK agent captures memory session IDs and persists observations through an atomic SQLite transaction pipeline.

Claude-Mem, an open-source memory layer for Claude Code, implements a sophisticated observation pipeline that separates session initialization from data persistence. Understanding how Claude-Mem's lifecycle hooks capture and store observations requires examining the interaction between hook handlers, the SDK agent, and the SQLite storage layer.

Understanding the Hook Architecture

The lifecycle hooks act as entry points that prepare the environment but do not directly write to the database. They trigger the worker service that handles the actual observation capture.

SessionStart Hook: Initializing Context

Located in src/cli/handlers/context.ts, the SessionStart hook ensures the worker service is running and injects historical context into the conversation. It calls the worker's /api/context/inject endpoint to fetch a markdown timeline that becomes the initial system message.

// src/cli/handlers/context.ts
export const contextHandler: EventHandler = {
  async execute(input) {
    const workerReady = await ensureWorkerRunning();
    if (!workerReady) return { hookSpecificOutput: { hookEventName: 'SessionStart', additionalContext: '' } };

    const cwd = input.cwd ?? process.cwd();
    const context = getProjectContext(cwd);
    const port = getWorkerPort();

    const projects = context.allProjects.join(',');
    const url = `http://127.0.0.1:${port}/api/context/inject?projects=${encodeURIComponent(projects)}`;

    const response = await fetch(url);
    const additionalContext = (await response.text()).trim();

    return {
      hookSpecificOutput: { hookEventName: 'SessionStart', additionalContext },
    };
  },
};

This handler captures the contentSessionId from Claude Code, which links the CLI session to the database records.

UserPromptSubmit Hook: Forwarding Input

The UserPromptSubmit hook in src/cli/handlers/user-message.ts captures the user's prompt and forwards it to the worker via the CLI command. This triggers the SDK agent to continue the conversation without the hook itself performing any database operations.

Capturing the Memory Session ID in the SDK Agent

The actual observation capture begins in src/services/worker/SDKAgent.ts. This event-driven agent creates an async generator that yields prompts and processes assistant responses. When the Agent SDK returns an assistant message, the agent extracts the text and captures the critical memory_session_id that the SDK returns.

// src/services/worker/SDKAgent.ts (excerpt)
for await (const message of queryResult) {
  // SDK may return a new memory_session_id on the first assistant message
  if (message.session_id && message.session_id !== session.memorySessionId) {
    const previousId = session.memorySessionId;
    session.memorySessionId = message.session_id;

    // Persist immediately so FK constraints succeed later
    this.dbManager.getSessionStore().ensureMemorySessionIdRegistered(
      session.sessionDbId,
      message.session_id,
    );
    logger.info('SESSION', `MEMORY_ID_${previousId ? 'UPDATED' : 'CAPTURED'} ${message.session_id}`);
  }

  if (message.type === 'assistant') {
    const text = extractText(message);
    await processAgentResponse(
      text,
      session,
      this.dbManager,
      this.sessionManager,
      worker,
      discoveryTokens,
      originalTimestamp,
      'SDK',
      cwdTracker.lastCwd,
    );
  }
}

The ensureMemorySessionIdRegistered call acts as a safety net, ensuring the foreign key constraint is satisfied before any observation is saved.

Parsing and Storing Observations Atomically

Once the SDK agent captures the response, src/services/worker/agents/ResponseProcessor.ts handles parsing and persistence.

Response Parsing and Validation

The processAgentResponse function receives the assistant text and parses structured observations using parseObservations, along with an optional summary via parseSummary. It verifies that session.memorySessionId exists before proceeding, throwing an error if the ID is missing to prevent orphaned records.

Foreign Key Safety Net

Before storage, the processor calls sessionStore.ensureMemorySessionIdRegistered as an idempotent safety check. This function, implemented in src/services/sqlite/SessionStore.ts, updates the sdk_sessions.memory_session_id row if it is still NULL or stale:

// src/services/sqlite/SessionStore.ts (excerpt)
ensureMemorySessionIdRegistered(sessionDbId: number, memorySessionId: string): void {
  const session = this.db.prepare(`
    SELECT id, memory_session_id FROM sdk_sessions WHERE id = ?
  `).get(sessionDbId) as { id: number; memory_session_id: string | null } | undefined;

  if (!session) throw new Error(`Session ${sessionDbId} not found`);

  if (session.memory_session_id !== memorySessionId) {
    this.db.prepare(`
      UPDATE sdk_sessions SET memory_session_id = ? WHERE id = ?
    `).run(memorySessionId, sessionDbId);
    logger.info('DB', 'Registered memory_session_id before storage (FK fix)', {
      sessionDbId,
      oldId: session.memory_session_id,
      newId: memorySessionId,
    });
  }
}

Atomic SQLite Transactions

The actual storage occurs in src/services/sqlite/transactions.ts via storeObservations. This function executes a single atomic transaction that inserts each observation into the observations table, including hierarchical fields like title, subtitle, facts, concepts, files_read, and files_modified, along with the prompt_number that ties the observation to the specific user prompt.

// src/services/worker/agents/ResponseProcessor.ts (excerpt)
export async function processAgentResponse(
  text,
  session,
  dbManager,
  sessionManager,
  worker,
  discoveryTokens,
  originalTimestamp,
  agentName,
  projectRoot,
) {
  const observations = parseObservations(text, session.contentSessionId);
  const summary = parseSummary(text, session.sessionDbId);

  if (!session.memorySessionId) {
    throw new Error('Cannot store observations: memorySessionId not yet captured');
  }

  // Ensure FK safety net
  sessionStore.ensureMemorySessionIdRegistered(session.sessionDbId, session.memorySessionId);

  const result = sessionStore.storeObservations(
    session.memorySessionId,
    session.project,
    observations,
    summaryForStore,
    session.lastPromptNumber,
    discoveryTokens,
    originalTimestamp ?? undefined,
  );

  // … broadcast & Chroma sync omitted for brevity …
}

Broadcasting and Vector Sync

After the transaction commits, ResponseProcessor broadcasts the new observations via Server-Sent Events (SSE) to the web UI and syncs the data to Chroma for vector search capabilities. This ensures real-time visibility and semantic searchability of captured observations.

Summary

• Lifecycle hooks (SessionStart and UserPromptSubmit) initialize the session and forward user input without directly touching the database.
• SDKAgent captures the critical memory_session_id from the Agent SDK and ensures it is registered before any storage occurs.
• ResponseProcessor parses structured observations and summaries from assistant messages, verifying foreign key constraints.
• SessionStore provides idempotent FK safety via ensureMemorySessionIdRegistered and atomic transactions via storeObservations.
• Broadcast and sync operations push observations to the web UI and Chroma vector store immediately after persistence.

Frequently Asked Questions

What triggers observation storage in Claude-Mem?

Observation storage triggers when the SDK Agent receives an assistant message from the Claude Code SDK. The UserPromptSubmit hook forwards the user prompt to the worker, which initiates a conversation via SDKAgent.ts. When the SDK yields an assistant response, the processAgentResponse function in ResponseProcessor.ts parses and stores the observations.

How does Claude-Mem handle foreign key constraints during storage?

Claude-Mem implements a defensive foreign key safety net through ensureMemorySessionIdRegistered in src/services/sqlite/SessionStore.ts. Before any observation is inserted, the system verifies that the sdk_sessions table contains the memory_session_id. If the ID is missing or stale, the function updates the row immediately, ensuring the foreign key constraint to the observations table will succeed during the atomic transaction.

What is the role of the ResponseProcessor in the observation pipeline?

The ResponseProcessor in src/services/worker/agents/ResponseProcessor.ts serves as the central parsing and persistence coordinator. It receives raw assistant text, extracts structured observations using parseObservations, and optional summaries via parseSummary. It validates that the memorySessionId exists, ensures foreign key integrity, and executes the atomic storeObservations transaction. After storage, it handles broadcasting to the UI and syncing to Chroma.

How does the SessionStart hook prepare the environment for observation capture?

The SessionStart hook in src/cli/handlers/context.ts prepares the observation pipeline by ensuring the worker service is running and injecting historical context into the conversation. It fetches a markdown timeline from the worker's /api/context/inject endpoint, which becomes the initial system message. This establishes the contentSessionId in the ActiveSession object, linking the Claude Code CLI session to the database records that will store future observations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}