How the Three-Tier Memory System (Hot/Warm/Cold) Captures and Retrieves Learning Signals in PAI

The Personal AI Infrastructure implements a three-tier memory system that splits persistent state into hot (fast runtime JSON), warm (graded learning records), and cold (immutable work archives) tiers to optimize for speed, signal richness, and long-term durability.

The danielmiessler/Personal_AI_Infrastructure repository implements a sophisticated three-tier memory system that enables continuous learning across sessions. By segregating data into hot, warm, and cold storage layers, PAI balances immediate responsiveness with deep retrospective analysis, ensuring every interaction generates durable learning signals.

Hot Tier: Immediate Runtime State

The hot tier stores ephemeral, high-speed runtime data in ~/.claude/MEMORY/STATE/. These JSON blobs serve as the system's working memory, enabling instantaneous context retrieval during active sessions.

Capture Mechanism

Hooks write directly to the hot tier for instant feedback. The AutoWorkCreation.hook.ts updates STATE/current-work.json to track the active work pointer, while session-specific caches and tab titles populate STATE/ subdirectories. These files are deliberately ephemeral—they can be rebuilt from raw transcripts if deleted, prioritizing write speed over durability.

Retrieval Pattern

The Algorithm reads hot tier files at the beginning of each phase to determine current work and session context. Simple filesystem commands provide immediate access:

# Show active work pointer

cat ~/.claude/MEMORY/STATE/current-work.json | jq .

# List recent UI state

ls ~/.claude/MEMORY/STATE/tab-titles/

Warm Tier: Graded Learning Signals

The warm tier in ~/.claude/MEMORY/LEARNING/ captures graded learning records including ratings, sentiment analysis, failures, and synthesized patterns. This tier transforms raw interactions into structured intelligence.

Capture Pipeline

Multiple hooks and harvesters populate the warm tier with increasing levels of abstraction:

Explicit Ratings: RatingCapture.hook.ts triggers on UserPromptSubmit, storing ratings to LEARNING/SIGNALS/ratings.jsonl. Low ratings (1-3) additionally invoke FailureCapture.ts to write full-context dumps to LEARNING/FAILURES/YYYY-MM/DD_HHMMSS_<description>/.

Harvesting: SessionHarvester.ts scans Claude Code projects/ directories, converting raw JSONL transcripts into structured learning entries. LearningPatternSynthesis.ts aggregates these into LEARNING/SYNTHESIS/weekly-patterns.md, summarizing rating trends and recurring pain points.

Retrieval During Learning

The Algorithm's LEARN phase consumes warm tier signals to update Ideal State Criteria and suggest improvements. Developers can surface recent signals directly:

# View last 20 ratings with sentiment

tail ~/.claude/MEMORY/LEARNING/SIGNALS/ratings.jsonl | jq .

# List recent failure investigations

ls -lt ~/.claude/MEMORY/LEARNING/FAILURES/$(date +%Y-%m) | head -5

# Read synthesized weekly patterns

cat ~/.claude/MEMORY/LEARNING/SYNTHESIS/weekly-patterns.md

Cold Tier: Persistent Work Archives

The cold tier provides immutable, long-term storage in ~/.claude/MEMORY/WORK/ and top-level archives (PAISYSTEMUPDATES/, RESEARCH/, SECURITY/). These durable records enable cross-session reasoning and audit trails.

Capture of Permanent Records

Work Tracking: Every discrete work item creates a directory under WORK/{work_id}/ containing:

• META.yaml with status, timestamps, and lineage information
• ISC.json capturing the Ideal State Criteria at session end
• items/, verification/, research/, scratch/ subdirectories for artefacts and proofs

System Archives: Security events log to SECURITY/ via SecurityValidator.hook.ts. Architectural decisions and migrations record to PAISYSTEMUPDATES/ through CreateUpdate.ts. Research agent outputs populate RESEARCH/.

Retrieval for Historical Context

The cold tier supports browsing, searching, and replaying past work:

# Show 10 most recent completed work items

ls -lt ~/.claude/MEMORY/WORK/ | head -10

# Replay a specific work's Ideal State Criteria

cat ~/.claude/MEMORY/WORK/2026-01-18-abcdef/ISC.json | jq .

# Audit recent security events

cat ~/.claude/MEMORY/SECURITY/security-events.jsonl | jq .

Signal Flow Across the Three-Tier Memory System

The three tiers operate as a pipeline that transforms transient interactions into durable intelligence:

1. User Request triggers hooks that write immediately to the hot tier (STATE/) for runtime context
2. Session Completion activates harvesters (SessionHarvester.ts, LearningPatternSynthesis.ts) that promote signals to the warm tier (LEARNING/) as graded records and patterns
3. Work Finalization persists complete artefacts to the cold tier (WORK/, SECURITY/, PAISYSTEMUPDATES/) for long-term reference
4. Algorithm Execution reads across all tiers: hot for current context, warm for recent learnings, cold for historical constraints

This architecture ensures speed (hot tier JSON blobs), signal richness (warm tier graded analysis), and durability (cold tier immutable archives).

Summary

• Hot Tier (~/.claude/MEMORY/STATE/): Stores ephemeral runtime state including current work pointers and session caches via hooks like AutoWorkCreation.hook.ts, optimized for instantaneous read/write access.
• Warm Tier (~/.claude/MEMORY/LEARNING/): Captures graded learning signals through RatingCapture.hook.ts and harvesters like SessionHarvester.ts, storing ratings, failures, and synthesized patterns for algorithmic learning.
• Cold Tier (~/.claude/MEMORY/WORK/): Maintains immutable archives of complete work items with META.yaml and ISC.json files, plus security logs and system updates, enabling cross-session reasoning and audit trails.
• Retrieval Strategy: The Algorithm reads hot tier for immediate context, warm tier during the LEARN phase for recent signals, and cold tier for historical work replay and constraint checking.

Frequently Asked Questions

How does the hot tier differ from the warm tier in the PAI three-tier memory system?

The hot tier stores ephemeral, high-speed runtime data like current-work.json and session caches in ~/.claude/MEMORY/STATE/, designed for instantaneous access during active sessions. In contrast, the warm tier in ~/.claude/MEMORY/LEARNING/ holds graded, semi-processed learning signals such as ratings, sentiment analysis, and failure patterns that persist across sessions but require periodic harvesting and synthesis.

What triggers data to move from the warm tier to the cold tier?

Data transitions from warm to cold when work items reach completion and require permanent archival. Hooks like AutoWorkCreation.hook.ts and completion handlers create directories under ~/.claude/MEMORY/WORK/{work_id}/ containing META.yaml, ISC.json, and verification artefacts. Additionally, security events captured by SecurityValidator.hook.ts and system updates via CreateUpdate.ts write directly to cold storage archives.

How does the Algorithm retrieve learning signals during the LEARN phase?

During the LEARN phase, the Algorithm queries the warm tier by reading ~/.claude/MEMORY/LEARNING/SIGNALS/ratings.jsonl for recent ratings, scanning LEARNING/FAILURES/ for low-rated interactions, and consuming LEARNING/SYNTHESIS/weekly-patterns.md for aggregated trends. These signals update the Ideal State Criteria and inform future planning decisions, creating a feedback loop that improves performance across sessions.

Can the cold tier be used to replay previous work sessions?

Yes, the cold tier enables session replay by loading persisted work directories. Each completed work item in ~/.claude/MEMORY/WORK/{work_id}/ contains an ISC.json file capturing the Ideal State Criteria and associated artefacts in subdirectories like items/ and verification/. The Algorithm can reload these files to reconstruct context from previous sessions, enabling cross-session reasoning and continuity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}