# How to Implement Hierarchical Memory Systems: Episodic, Semantic, and Procedural Best Practices

> Master hierarchical memory systems with best practices for episodic, semantic, and procedural implementation. Learn to build adaptive, efficient memory architectures.

- Repository: [davidkimai/context-engineering](https://github.com/davidkimai/context-engineering)
- Tags: best-practices
- Published: 2026-02-28

---

**Implement hierarchical memory systems using a three-layer architecture (working, long-term, external) with explicit promotion/demotion logic, organize fragments into four-level hierarchies (raw → semantic → temporal → conceptual), and tag entries by type (episodic, semantic, procedural) to enable adaptive retrieval and compression.**

The `davidkimai/context-engineering` repository provides a production-ready framework for building hierarchical memory systems that mirror human cognitive architecture. This guide extracts implementation patterns from [`memory_management_lab.py`](https://github.com/davidkimai/context-engineering/blob/main/memory_management_lab.py) and the cognitive architecture specifications to show you how to implement episodic, semantic, and procedural memory layers with proper promotion, compression, and retrieval mechanisms.

## Architectural Overview of Hierarchical Memory Systems

### Three-Layer Storage Architecture

The `HierarchicalMemorySystem` class in [`00_COURSE/03_context_management/labs/memory_management_lab.py`](https://github.com/davidkimai/context-engineering/blob/main/00_COURSE/03_context_management/labs/memory_management_lab.py) implements a tiered storage model:

- **Working Memory**: A fast, token-limited cache (default 5,000 tokens) for immediate conversational context. Implemented as `WorkingMemory` inside `HierarchicalMemorySystem`.
- **Long-Term Memory**: A persistent store (10M+ bytes) holding **episodic**, **semantic**, and **procedural** fragments. Implemented as `LongTermMemory`.
- **External Retrieval**: An optional plug-in hook (`external_retriever`) for vector databases or web search.

### Memory Type Taxonomy

The repository distinguishes three fundamental memory types:

- **Episodic**: Time-stamped events (e.g., "User asked about hierarchical memory on 2024-02-28").
- **Semantic**: Extracted concepts and facts (e.g., "Hierarchical fragmentation uses four levels").
- **Procedural**: Learned procedures and policies (e.g., "If access_count > 3, promote to working memory").

## Implementing the Core Memory Layers

### Instantiating the Hierarchical Memory System

Create a system with explicit size constraints and optional external retrieval:

```python
from memory_management_lab import HierarchicalMemorySystem

# Create a system with a 5k token working buffer

mem_sys = HierarchicalMemorySystem(
    working_memory_size=5000,         # tokens

    long_term_memory_size=10_000_000 # bytes

)

# Optional: plug in an external retriever (e.g., a vector DB)

def external_search(query, limit):
    # Replace with real vector search implementation

    return [("ext_1", "External result 1"), ("ext_2", "External result 2")]

mem_sys.external_retriever = external_search

```

### Promotion and Demotion Logic

The repository implements explicit thresholds for moving data between layers:

- **Promotion to Working Memory**: Triggered when `access_count > 3` **or** `priority > 0.7`.
- **Demotion from Working Memory**: Occurs when `access_count < 2` **and** `last_accessed > 6` hours.
- **Long-Term Storage**: Entries with `priority > 0.5` or specific tags are written to `LongTermMemory` during the `store()` operation.

Periodically call `optimize()` to decay stale priorities (7-day half-life) and evict low-usage items.

## Organizing Memory Fragments Hierarchically

### The Four-Level Fragment Hierarchy

According to [`cognitive-tools/cognitive-architectures/reconstruction-memory-architecture.md`](https://github.com/davidkimai/context-engineering/blob/main/cognitive-tools/cognitive-architectures/reconstruction-memory-architecture.md), fragments organize into a four-level hierarchy:

1. **Level 0 (Raw Fragments)**: Individual episodic events or data points.
2. **Level 1 (Semantic Clusters)**: Grouped by conceptual similarity.
3. **Level 2 (Temporal Sequences)**: Ordered series of clusters representing processes.
4. **Level 3 (Conceptual Themes)**: High-level abstractions and schemas.

### HierarchicalFragmentOrganizer Implementation

Use the `HierarchicalFragmentOrganizer` class to manage this structure:

```python
from reconstruction_memory_architecture import HierarchicalFragmentOrganizer

organizer = HierarchicalFragmentOrganizer(max_levels=4)

# Assume `fragments` is a list of raw episodic snippets

fragments = [
    {"text": "User asked about hierarchical memory.", "timestamp": 1},
    {"text": "Explained three memory layers.", "timestamp": 2},
    # …

]

hierarchy = organizer.organize_fragments(fragments)

# Reconstruct using a cue (e.g., the word "memory")

cues = ["memory"]
reconstruction = organizer.reconstruct_with_hierarchy(cues, context={})
print(reconstruction)   # → high-level thematic summary + relevant details

```

Reconstruction proceeds top-down: activate themes → sequences → clusters → fragments.

## Storing Episodic, Semantic, and Procedural Memory

Tag entries explicitly to distinguish memory types during retrieval:

```python

# Episodic (event) entry

mem_sys.store(
    key="event_2024-02-28_meeting",
    content="Discussed hierarchical memory best practices with team.",
    tags=["episodic", "meeting"],
    priority=0.9
)

# Semantic (concept) entry

mem_sys.store(
    key="concept_hierarchical_fragmentation",
    content="Organize memory fragments into 4 hierarchical levels: raw → semantic → temporal → conceptual.",
    tags=["semantic", "memory"],
    priority=0.8
)

# Procedural (how-to) entry

mem_sys.store(
    key="procedure_promote_entry",
    content="If access_count > 3 or priority > 0.7, copy entry to working memory.",
    tags=["procedural", "policy"],
    priority=0.7
)

```

The `retrieve()` method automatically promotes frequently accessed entries to working memory based on the thresholds defined in the system configuration.

## Multi-Modal Integration and Compression

### Cross-Modal Memory Mapping

For systems handling diverse data types, implement multi-modal encoding as described in [`reconstruction-memory-architecture.md`](https://github.com/davidkimai/context-engineering/blob/main/reconstruction-memory-architecture.md):

- Assign modality identifiers (`['text']`, `['visual']`, `['auditory']`, `['spatial']`, `['temporal']`).
- Use `CrossModalMapper` to discover correspondences between modalities.
- Build unified representations that enable procedural memories (e.g., "how to assemble a device") to coexist with textual facts.

### Adaptive Compression Pipeline

The `HierarchicalMemory` class in [`00_COURSE/02_context_processing/labs/long_context_lab.py`](https://github.com/davidkimai/context-engineering/blob/main/00_COURSE/02_context_processing/labs/long_context_lab.py) implements a three-tier compression strategy:

```python
from long_context_lab import HierarchicalMemory
import numpy as np

# d_model = dimensionality of the transformer representation

hm = HierarchicalMemory(d_model=256)

# Add a new context vector (e.g., a hidden state from a language model)

vector = np.random.randn(256)
hm.add_context(vector)

# Query with another vector; get the most relevant compressed representation

query_vec = np.random.randn(256)
relevant = hm.retrieve_relevant(query_vec, max_tokens=256)
print("Retrieved shape:", relevant.shape)

```

- **Short-term**: Raw vectors stored in working buffer.
- **Medium-term**: Compressed summaries using `compress_medium` (4× compression ratio).
- **Long-term**: Highly compressed gists using `compress_long`.

Tune the learned compression matrices on downstream tasks to preserve relevance while minimizing footprint.

## Consolidated Best Practices Checklist

Follow these evidence-based guidelines derived from the Context-Engineering codebase:

- **Size your layers**: Configure working memory to your token budget (e.g., 5,000 tokens) and long-term memory to 10M+ bytes to keep inference cheap while preserving depth.
- **Priority decay**: Assign higher priority to frequently queried or critical facts, but decay priority after 7 days of inactivity to prevent stale entries from hogging space.
- **Promotion thresholds**: Promote entries to working memory when `access_count > 3` or `priority > 0.7`; demote when `access_count < 2` and `last_accessed > 6` hours.
- **Compression ratios**: Use a 4× compression ratio when moving from medium-term to long-term storage, tuning the `compress_medium` and `compress_long` matrices on downstream tasks.
- **Fragment indexing**: Store each of the four hierarchy levels (raw, semantic clusters, temporal sequences, conceptual themes) in separate indexes to enable efficient top-down reconstruction.
- **Modality tagging**: Tag all entries with modality identifiers (`text`, `visual`, `auditory`) to ensure correct encoder selection and easier cross-modal mapping.
- **Search order**: Always search working memory first, then long-term with a fallback limit, then external retrievers, deduplicating keys before merging results.
- **Monitoring**: Expose `get_statistics()` to track hits, promotions, demotions, and utilization ratios, logging trends to data-drive layer resizing.
- **Testing**: Unit test `store()`, `retrieve()`, and `optimize()` methods, verifying that promotion/demotion respects the defined thresholds.

## Summary

Implementing hierarchical memory systems requires balancing speed, capacity, and retrieval accuracy across three distinct layers. The key takeaways from the Context-Engineering repository include:

- Use a **three-tier architecture** (working, long-term, external) with explicit promotion thresholds (`access_count > 3` or `priority > 0.7`) and demotion rules.
- Organize content into a **four-level fragment hierarchy** (raw → semantic clusters → temporal sequences → conceptual themes) using `HierarchicalFragmentOrganizer`.
- Tag entries by type (**episodic**, **semantic**, **procedural**) and modality to enable precise retrieval and cross-modal integration.
- Apply **adaptive compression** (4× ratio) when moving data to long-term storage, and decay priorities after 7 days to maintain relevance.
- Monitor system statistics via `get_statistics()` to optimize layer sizing and retrieval performance.

## Frequently Asked Questions

### What is the difference between episodic, semantic, and procedural memory in hierarchical systems?

**Episodic memory** stores time-stamped events and experiences (e.g., "User asked about hierarchical memory on 2024-02-28"), **semantic memory** stores facts and concepts (e.g., "Hierarchical fragmentation uses four levels"), and **procedural memory** stores learned skills and policies (e.g., "If access_count > 3, promote to working memory"). In the Context-Engineering repository, these are distinguished by tags passed to the `store()` method in [`memory_management_lab.py`](https://github.com/davidkimai/context-engineering/blob/main/memory_management_lab.py), enabling type-specific retrieval and reconstruction strategies.

### How does promotion between working memory and long-term memory work?

Promotion occurs automatically when an entry's `access_count` exceeds 3 or its `priority` score exceeds 0.7, triggering a copy from `LongTermMemory` to `WorkingMemory`. Conversely, demotion occurs when `access_count` drops below 2 and `last_accessed` exceeds 6 hours. The `optimize()` method in `HierarchicalMemorySystem` handles these transitions while decaying priorities (7-day half-life) and evicting stale entries to maintain performance.

### What compression strategy should I use for long-term storage?

Use a **4× compression ratio** when transitioning from medium-term to long-term storage. The `HierarchicalMemory` class in [`long_context_lab.py`](https://github.com/davidkimai/context-engineering/blob/main/long_context_lab.py) implements learned compression matrices (`compress_medium` and `compress_long`) that should be fine-tuned on downstream tasks to preserve semantic relevance while minimizing footprint. This approach reduces memory usage without sacrificing the ability to reconstruct high-level themes during retrieval.

### How do I organize memory fragments for efficient retrieval?

Implement a **four-level hierarchy** using `HierarchicalFragmentOrganizer` from [`reconstruction-memory-architecture.md`](https://github.com/davidkimai/context-engineering/blob/main/reconstruction-memory-architecture.md): Level 0 (raw episodic fragments), Level 1 (semantic clusters), Level 2 (temporal sequences), and Level 3 (conceptual themes). Store each level in separate indexes and use top-down reconstruction (themes → sequences → clusters → fragments) when retrieving with cues. This structure enables efficient semantic lookup while preserving temporal and thematic context.