How Short-Term Context Layering Works in TencentDB Agent Memory

TencentDB Agent Memory implements a four-tier context hierarchy (L0–L3) where L0 short-term memory captures real-time conversation, while background jobs progressively distill it into atomic facts, scenario blocks, and persistent persona knowledge to optimize retrieval and token usage.

Short-term context layering is the foundation of the TencentDB Agent Memory architecture, enabling AI agents to balance immediate conversational awareness with efficient long-term knowledge retrieval. This open-source system (TencentCloud/TencentDB-Agent-Memory) structures memory into four distinct tiers that automatically promote information from raw dialogue to stable, reusable knowledge. Understanding how these layers interact is essential for optimizing agent performance and managing the LLM context window effectively.

The Four-Tier Memory Hierarchy

As defined in the main README.md, the system organizes context into layers L0 through L3, each with distinct mutability, storage characteristics, and retrieval priority.

L0 – Short-Term Memory (Conversation)

L0 is the volatile, real-time layer that holds the newest turn-by-turn dialogue. According to the source architecture, this is the only layer that supports real-time write-back. When a user completes a turn, the Memory Proxy immediately posts the conversation slice to the Memory Core endpoint /v3/skill/conversation/add, storing it as short-term memory in MemoryCore/src/store/llm-binding-store.ts. This layer contains raw chat messages and recent user intents, making it available for immediate reasoning but subject to periodic compression.

L1 – Atomic Facts

L1 contains immutable facts extracted from L0 by background extraction workers. These are small, discrete knowledge units—such as "project X uses Node 22"—that can be rapidly retrieved via BM25 or vector search. The pipeline-worker.ts service processes L0 slices to generate these atoms, which serve as the first cache layer for structured knowledge.

L2 – Scenario Knowledge Blocks

L2 aggregates related L1 atoms into coherent scenario-level knowledge blocks (e.g., project specifications, product domains, or wiki pages). The context-injector.ts module treats L2 as the primary quick bootstrap layer, loading these blocks to provide agents with immediate situational awareness without parsing entire conversation histories.

L3 – Persona Profile

L3 is the most stable layer, containing the agent’s persona, high-level cognition, core skills, and company policies. Loaded once per session, L3 rarely changes and serves as the foundational context for consistent agent behavior across interactions.

Real-Time Write-Back via Memory Proxy

The short-term context pipeline begins with the Memory Proxy, which handles the immediate persistence of conversation turns to L0. This write-back mechanism ensures that recent dialogue is captured before background processing initiates.

import { MemoryClient } from '@tencentdb-agent-memory/memory-core';

// Create a client with isolation context (required by the core)
const client = new MemoryClient({ isolation: true });

// Send the latest user turn – this creates an L0 short-term entry
await client.skill.conversation.add({
  sessionId: 'sess-1234',
  content: [
    { role: 'user', text: 'How do I deploy the new version?' },
    { role: 'assistant', text: 'You need to run ./deploy.sh.' },
  ],
});

Background Extraction Pipeline (L0 to L1/L2)

Once L0 entries are persisted, background workers in MemoryCore/src/services/pipeline-worker.ts transform transient dialogue into structured knowledge. This extraction pipeline operates asynchronously to avoid blocking real-time interactions, progressively promoting data from L0 to L1 atoms and subsequently merging atoms into L2 scenarios.

import { extractAtoms } from './utils/atom-extractor';
import { storeAtom } from './store/atom-store';

async function processL0(sessionId: string) {
  const l0 = await getL0Conversation(sessionId);           // fetch recent slice
  const atoms = await extractAtoms(l0);                    // e.g. Regex / Entity extraction
  await Promise.all(atoms.map(a => storeAtom(sessionId, a)));
}

Retrieval Strategy and Context Injection

During generation, the retrieval system prioritizes higher layers to minimize token consumption. The buildSessionContextBlockWithToggles function in MemoryCore/src/session/context-injector.ts implements a tiered retrieval strategy: it first attempts to load L3 persona and L2 scenario blocks. Only if context is insufficient does it fall back to BM25 and vector search across L1 atoms and, finally, raw L0 conversation slices. Retrieval is governed by strict item counts, character budgets, and timeouts to enforce the model's context window limits.

import { buildSessionContextBlockWithToggles } from './session/context-injector';

// Assemble the final system prompt for the LLM
async function buildPrompt(sessionId: string) {
  // Load L3 persona + L2 scenario (fast, cached)
  const contextBlock = await buildSessionContextBlockWithToggles(sessionId, {
    includePersona: true,
    includeScenario: true,
    // fallback to L1/L0 only if needed
    fallback: true,
  });
  return [
    { role: 'system', content: contextBlock },
    // user messages follow...
  ];
}

Compression and Lifecycle Management

To prevent context window overflow, the L0 short-term memory layer undergoes periodic compression. Once background jobs confirm that L1 atoms and L2 scenarios have successfully ingested the information from llm-binding-store.ts, the original L0 conversation slices are summarized or discarded. This lifecycle ensures that only high-signal, condensed knowledge persists in active memory while raw dialogue is archived or removed.

Summary

• TencentDB Agent Memory uses a four-tier hierarchy (L0–L3) to separate volatile conversation from stable knowledge.
• L0 short-term memory is the only real-time writable layer, capturing immediate dialogue via the Memory Proxy's /v3/skill/conversation/add endpoint.
• Background workers in pipeline-worker.ts progressively extract L1 atoms and merge them into L2 scenario blocks.
• Retrieval prioritizes L2/L3 for efficient prompting, falling back to L1/L0 only when necessary.
• Compression removes or summarizes L0 entries after higher layers absorb their content, optimizing token usage.

Frequently Asked Questions

What is the difference between L0 and L1 in TencentDB Agent Memory?

L0 (Short-Term Memory) stores raw, real-time conversation slices that are immediately writable but temporary, while L1 (Atomic Facts) contains immutable, extracted facts generated by background workers processing L0 content. L1 serves as the first cache layer for structured retrieval, whereas L0 serves immediate contextual awareness.

How does the system prevent the LLM context window from overflowing?

The system enforces strict retrieval budgets—including item count limits, character budgets, and timeouts—and compresses L0 short-term memory once its information has been promoted to L1 atoms and L2 scenarios. This ensures only essential, condensed knowledge reaches the final prompt, keeping token usage within model limits.

Can developers write directly to the L2 or L3 layers?

No, according to the source architecture described in MemoryProxy/README.md, L0 is the only layer that accepts real-time writes. L1, L2, and L3 are populated asynchronously through background extraction pipelines that process and promote data from lower layers, ensuring data integrity and proper knowledge distillation.

Which source files control the context injection and retrieval logic?

The retrieval and injection logic is primarily implemented in MemoryCore/src/session/context-injector.ts, which assembles L2/L3 contexts and manages fallback to L1/L0. The background promotion from L0 to higher layers is handled by MemoryCore/src/services/pipeline-worker.ts, while MemoryCore/src/store/llm-binding-store.ts persists the short-term L0 chunks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}