Short-Term, Working, and Long-Term Memory in AI Agents: Architecture and Implementation

AI agents manage context through three distinct memory layers: short-term memory for temporary context retention, working memory for active reasoning and manipulation, and long-term memory for persistent knowledge storage across sessions.

The davidkimai/context-engineering repository implements this hierarchy as a dual-layer system where working memory functions as the short-term buffer, enabling fast token-level access while long-term memory provides durable, searchable storage. Understanding how these layers interact is critical for building agents that maintain conversational coherence without exceeding model token limits.

Architectural Overview: The Three Memory Types

Short-Term Memory: The Transient Buffer

Short-term memory serves as the immediate context window, holding the most recent interactions and transient data. According to the source code in 00_COURSE/02_context_processing/labs/long_context_lab.py, this layer is implemented through the LongContext class and bounded by the short_term_size parameter (default 512 tokens) at line 306.

When the buffer exceeds this limit, the system triggers an eviction loop at line 324 that removes oldest entries first, ensuring the agent maintains only the most recent context for the current reasoning episode. This bounded approach prevents prompt overflow while keeping latency minimal.

Working Memory: Active Reasoning Space

While cognitive science distinguishes short-term storage from working memory (active manipulation), the repository implements working memory as the operational interface to the short-term buffer. The agent.working_memory object provides O(1) append and pop operations for the current turn’s data, as seen in the add short‑term entry implementation at line 324 of long_context_lab.py.

This layer handles active reasoning chains, immediate tool results, and the current user utterance—essentially everything the model is actively processing. The MemoryReasoningEngine in cognitive-tools/cognitive-programs/program-library.py (line 731) orchestrates how working memory feeds into the inference pipeline.

Long-Term Memory: Persistent Knowledge Storage

Long-term memory persists beyond individual reasoning episodes, storing compressed facts, summaries, and background knowledge. The LongTermMemory class in 00_COURSE/03_context_management/labs/memory_management_lab.py initializes with long_term_memory_size parameters (typically 10,000,000 bytes or larger) as shown at line 408.

Unlike the short-term buffer, long-term memory supports indexed lookup, similarity search, and disk persistence. The repository demonstrates this at line 444 with the store to disk functionality, enabling retrieval weeks or months after the initial storage.

Technical Implementation in Context-Engineering

Configuring Dual-Layer Memory Systems

The MemoryManagementSystem wrapper class provides a unified interface for both layers. You initialize it by specifying capacity limits for each memory type:

from memory_management_lab import MemoryManagementSystem

# Initialize with 512 tokens short-term, 10MB long-term storage

agent = MemoryManagementSystem(
    short_term_size=512,
    long_term_memory_size=10_000_000,
    persistence_file="ltm_store.pkl",
)

This configuration establishes the token budget for immediate reasoning (working memory) while allocating disk-backed storage for accumulated knowledge.

The Promotion Lifecycle: Short-Term to Long-Term

The repository demonstrates a standard pattern for moving information from working memory to long-term storage when the short-term buffer reaches capacity:

# Append raw interaction to working memory (short-term)

agent.working_memory.append(user_input)

# Check if we've hit the 512 token limit

if agent.working_memory.is_full():
    # Generate compressed summary before eviction

    summary = summarize(agent.working_memory.contents())
    
    # Promote to long-term memory with persistent key

    agent.long_term_memory.store(
        key="session_1_summary", 
        entry=summary
    )
    
    # Clear working memory for next reasoning episode

    agent.working_memory.clear()

This pattern prevents context loss while maintaining the strict size constraints required for low-latency inference.

Cross-Session Retrieval Patterns

Long-term memory retrieval involves indexed searches that return relevant context for injection into the current working buffer:

# Search persistent store (vector similarity or exact match)

results = agent.long_term_memory.search(
    query="project deadline", 
    limit=1
)

if results:
    # Inject retrieved fact into active working memory

    context = f"Context from previous session: {results[0]['content']}"
    agent.working_memory.append(context)

The search API at line 482 of memory_management_lab.py supports both semantic vector search and exact-match retrieval, depending on the indexing strategy configured.

Performance Characteristics and Trade-offs

Capacity and Lifetime Constraints

Memory Layer	Capacity	Lifetime	Access Latency
Working (Short-Term)	512 tokens (configurable)	Single reasoning episode	O(1) list operations
Long-Term	Megabytes to gigabytes	Persistent across sessions	Variable (I/O + search index)

The trim loop at line 328 of long_context_lab.py automatically evicts oldest entries when short_term_size is exceeded, while long-term entries persist until explicitly deleted or the persistence_file is invalidated.

Timeframe Configuration

The program-examples.py file explicitly labels these distinct horizons in its configuration schema:

demo_config = {
    "timeframes": ["short_term", "long_term"],  # Line 375

}

This configuration drives the MemoryReasoningEngine to treat working memory as the short-term horizon and the LongTermMemory instance as the persistent horizon.

Summary

• Working memory implements short-term functionality in the davidkimai/context-engineering repository, providing a 512-token buffer (configurable via short_term_size) for active reasoning with O(1) access patterns.
• Long-term memory offers persistent, disk-backed storage (megabyte scale via long_term_memory_size) with searchable indexing through the LongTermMemory.search() API.
• Information flows from working to long-term through explicit summarization when the short-term buffer fills, as implemented in memory_management_lab.py.
• The MemoryReasoningEngine in program-library.py orchestrates both layers, enabling agents to handle extended dialogues without exceeding token limits.

Frequently Asked Questions

What is the difference between short-term memory and working memory in AI agents?

In cognitive architecture, short-term memory refers to temporary storage duration while working memory refers to active manipulation of that information. In the context-engineering repository, these concepts are implemented as a single layer—the working_memory object functions as the short-term buffer, providing both temporary storage and immediate access for active reasoning. The codebase uses short_term_size to bound this unified layer at 512 tokens by default.

How does the repository handle memory eviction when limits are reached?

The LongContext class implements a FIFO (First-In-First-Out) eviction policy through the trim loop at line 328 of long_context_lab.py. When working_memory.is_full() detects that the total token count exceeds short_term_size, the system automatically discards oldest entries to maintain the capacity constraint. For long-term memory, no automatic eviction occurs; storage persists until the persistence_file is manually cleared or the long_term_memory_size byte limit is reached.

Can long-term memory perform semantic search, or only exact matching?

The LongTermMemory.search() method at line 482 of memory_management_lab.py supports flexible retrieval strategies including vector-based similarity search and exact-match queries. The implementation allows for compression and indexing before storage, enabling semantic retrieval of semantically related facts even when query keywords differ from stored content.

What file format does the long-term memory system use for disk persistence?

The repository demonstrates persistence using Python's pickle protocol (.pkl files) as shown in the initialization example where persistence_file="ltm_store.pkl". This format serializes the entire LongTermMemory dictionary structure to disk at line 444, preserving usage statistics and indexed entries across program restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}