How TencentDB Agent Memory Prevents Context Window Overflow with Its Layered Architecture

TencentDB Agent Memory prevents context window overflow by splitting conversation history into four hierarchical layers (L0-L3) and injecting only compressed, pre-summarized data into each LLM request.

TencentDB Agent Memory (TD-DB Agent Memory) solves the limited context window problem of large language models through a layered memory hierarchy that prioritizes information density over raw data volume. Instead of shipping full conversation history to the model, the system compresses older interactions into summaries and indexes, ensuring every prompt stays within token limits while preserving semantic relevance.

The Four-Layer Memory Hierarchy

The architecture organizes memory into four distinct layers, each with specific granularity, size constraints, and production mechanisms.

L0 – Raw Turn-by-Turn Storage

The L0 layer captures every inbound and outbound message at a one-to-one granularity with the conversation turn. This includes user queries, agent responses, and tool calls. In MemoryProxy/src/handler.ts, the proxy records these raw interactions through the write-l0 operation (referenced by the log entry tdai-recorder:write-l0).

This layer maintains the fidelity of the original conversation but is only retained for a configurable horizon before being collapsed into higher layers.

L1 – Turn Summaries

The L1 layer stores short summaries (approximately 200 tokens) of the most recent conversation turns. It maintains a sliding window of the last N turns, providing the LLM with immediate context without the token cost of full transcripts.

The L1 recall injector (tdai-l1-recall-injector.ts) generates these summaries by calling the memory-core summarization service and injecting the results into the session context. This compression happens at MemoryProxy/src/injection/injectors/tdai-l1-recall-injector.ts.

L2 – Scenario Index

The L2 layer functions as a compact index storing file paths, wiki IDs, and optional one-line summaries for relevant artifacts. Each entry consumes roughly 50 tokens, making it significantly more efficient than loading full document contents.

The L2 scene-navigation injector (tdai-profile-memory-injector.ts) builds this index at MemoryProxy/src/injection/injectors/tdai-profile-memory-injector.ts, appending only path + summary pairs without loading the full content. The actual data remains in the backend store, fetched on-demand via tool calls when the LLM references a specific path.

L3 – Global Knowledge

The L3 layer contains high-level project descriptions, static knowledge graphs, and skill definitions. This layer remains very small (≤100 tokens) and never grows with usage.

Static knowledge injectors (such as knowledge-tools-injector.ts) embed XML blocks describing the agent, task, and available tools. This functionality is implemented at MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts.

How the Session Context Injector Assembles the Prompt

When a request reaches the proxy, the Session-Context Injector (src/session/context-injector.ts) constructs the final <session_context> block by concatenating enabled layers:

// MemoryProxy/src/session/context-injector.ts
const CTX_OPEN = "<session_context>";
const CTX_CLOSE = "</session_context>";
// … build the block from L0-L3 layers according to toggle flags

Only layers enabled via request toggles or server defaults are appended to the first system-role message the LLM receives. This guarantees bounded prompt sizes because older turns collapse into L1 summaries, large artifacts reduce to L2 path indexes, and static L3 knowledge remains fixed.

Safety Mechanisms and Cost Guards

Beyond hierarchical compression, the system implements a cost-guard module that enforces hard limits on prompt size. Located in MemoryProxy/src/storage/factory.ts, this guard aborts injection if the assembled block would exceed the model's context window.

The layered approach transforms arbitrarily long conversations into a bounded structure:

[System prompt] + <session_context>
  ├─ L3 static knowledge
  ├─ L2 scenario index (paths + optional one-line summaries)
  ├─ L1 recent turn summaries
  └─ L0 most recent turn (if still within the window)
[user message] …

Because each layer is pre-summarized or indexed, the proxy continuously feeds the LLM without hitting the 4-KB (or model-specific) context limit, while preserving the ability to retrieve full history through the /v3/atomic/search API.

Practical Implementation Examples

Using the TypeScript SDK for Automatic Layer Injection

The MemoryClient automatically constructs the <session_context> block using L1-L3 layers:

import { MemoryClient } from '@memory-core/client';

const client = new MemoryClient({
  isolation: {
    userId: 'u123',          // mandatory
    agentId: 'agt-ea0b0wybln',
    spaceId: 'default',
    // optional taskId – narrows recall
    taskId: process.env.TDAI_TASK_ID,
  },
});

await client.chatCompletions({
  model: 'glm-5.2-vision',
  messages: [{ role: 'user', content: 'Explain the last design decision' }],
  // The SDK automatically adds a `<session_context>` block composed of L1-L3 layers.
});

Toggling Layers from the React Frontend

Developers can control which layers the API returns using the React hook in the Memory Panel:

// MemoryPanel/web/src/pages/ChatMemoryPage/hooks/useChatMemory.ts
const [layer, setLayer] = useState<MemoryLayer>('L1');
const toggle = (layer: MemoryLayer) => {
  // `layer` query param controls which layers the API returns.
  chatMemoryApi.layer(blockId, layer, 1, 0).then(setCurrentLayer);
};

Debugging Injected Context in Proxy Logs

Inspect the assembled session context through Docker logs:

docker logs tdai-proxy --since 2m | grep "<session_context>"

# Sample output shows the assembled block with L3, L2, L1 sections.

Isolation Keys for Layered Recall

The three-dimensional isolation logic underpinning layer retrieval is defined in MemoryCore/src/core/store/isolation.ts, which manages user_id, agent_id, session_id, and optional task_id keys used to scope memory queries.

Summary

• TencentDB Agent Memory uses a four-layer hierarchy (L0-L3) to compress conversation history and prevent token overflow.
• L0 stores raw turns temporarily, while L1 compresses them into 200-token summaries.
• L2 maintains a compact index of relevant artifacts (paths + summaries) without loading full content.
• L3 provides fixed-size global knowledge (≤100 tokens) that never grows with conversation length.
• The Session-Context Injector assembles only enabled layers into the system prompt, while the cost-guard module aborts requests that would exceed context limits.
• Full historic data remains accessible on-demand via tool calls and the /v3/atomic/search API.

Frequently Asked Questions

What happens when the assembled prompt approaches the context window limit?

The cost-guard module in MemoryProxy/src/storage/factory.ts monitors token counts during assembly and aborts the injection if the combined layers would overflow the model's context window. This prevents runtime errors while ensuring the LLM receives the most critical information first (L3 static knowledge, followed by L2 indexes, then L1 summaries).

How does L2 differ from L1 in the memory hierarchy?

L1 contains semantic summaries of recent conversation turns (temporal compression), while L2 contains structural indexes of relevant files and artifacts (spatial compression). L2 entries include only paths and optional one-line summaries rather than full content, enabling the LLM to reference large knowledge bases without token penalties.

Can developers customize which memory layers are injected per request?

Yes. Developers toggle layers through the SDK configuration or via the layer query parameter in the React frontend (as shown in useChatMemory.ts). The Session-Context Injector respects these flags and only concatenates enabled layers when building the <session_context> block.

Where is the raw conversation data stored if not in the prompt?

Raw L0 data persists in the backend storage layer and remains accessible through on-demand retrieval methods. The system stores full content in the backend while injecting only L2 path indexes into the prompt. When the LLM requires specific content, it requests the full artifact via tool calls rather than receiving it proactively in the context window.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}