How TencentDB Agent Memory Prevents Memory from Overwhelming the Context Window

TencentDB Agent Memory uses a tiered storage architecture with automatic summarization and strict token budgeting to ensure only condensed memory summaries reach the LLM, keeping total payload within model context limits.

The TencentCloud/TencentDB-Agent-Memory repository implements a sophisticated context window management system that prevents conversation history from overwhelming LLM token limits. By combining layered memory tiers with runtime cost guards, the system guarantees that agents operate within constrained context windows without losing semantic relevance.

Layered Memory Architecture Reduces Raw Token Volume

The system organizes conversation data into three hierarchical tiers (L0, L1, and L2) that progressively abstract and compress information before it ever reaches the prompt construction phase.

L0 Raw Conversation Storage

At the base layer, L0 retains raw conversation turns in their original form. This tier serves as the immutable source of truth but never injects directly into active prompts.

L1 Record Aggregation

The L1 tier stores per-turn "records" containing structured metadata about individual interactions. While more compact than raw text, L1 entries still represent unnecessary token overhead for long-running sessions.

L2 Scenario-Level Shards with Summaries

L2 aggregates multiple L1 records into scenario-level shards that expose only a path and an optional summary field. According to the source code in MemoryCore/src/utils/memory-cleaner.ts, the LocalMemoryCleaner periodically purges obsolete shards to bound storage growth. This tier ensures that downstream injectors receive pointers and summaries rather than full conversation text, dramatically reducing the data volume eligible for prompt injection.

Automatic Summarization and Injection

When building requests for LLM providers, the system bypasses raw history in favor of compressed representations, ensuring that even lengthy conversations contribute minimal tokens.

Summary-Only Injection via tdai-profile-memory-injector.ts

The tdai-profile-memory-injector.ts module (located at MemoryProxy/src/injection/injectors/) reads exclusively from L2 scenario shards. It extracts only the summary fields—or generates summaries on-the-fly—and inserts them into the system prompt as <session_context> blocks. This injector deliberately omits full L0/L1 payloads, ensuring that historical data never bloats the current context window.

Session Context Assembly in context-injector.ts

The context-injector.ts module (MemoryProxy/src/session/) concatenates the agent persona, task description, and summarized memory blocks into a single XML fragment. The injectSessionContextWithToggles function accepts a toggle configuration to enforce summary-only mode, appending the compact <session_context> block to the first system message. Because this fragment contains only distilled information, the total token count remains well below the LLM's maximum capacity.

Explicit Trace Summarization API

For long-running conversations that accumulate significant history, developers can force immediate compression through the /memories/trace_summarize endpoint implemented in MemoryProxy/src/handler.ts. This API generates concise summaries of specified turn ranges (or the entire session) and stores them as reusable summary fields.

import { MemoryClient } from '@context-proxy/memory-core';

const client = new MemoryClient({ isolation: { /* session config */ } });

// Generate summary for entire session or specific turn range
await client.post('/memories/trace_summarize', {
  start_turn_id: "turn_001",
  end_turn_id: "turn_050"
});

Once generated, these summaries replace the full trace in subsequent context injections, preventing historical bloat from re-entering the context window.

Cost-Guard Token Budget Enforcement

Beyond architectural compression, the system enforces hard limits at runtime through a plugin called cost-guard, which intercepts every outbound request before it reaches the LLM provider.

Context Window Configuration

The context_window parameter in sdk/memory-core/typescript/src/types.ts defines the maximum tokens the target model accepts. Configuration files specify this limit alongside safety margins:

{
  "context_window": 8192,
  "max_tokens_per_message": 2000,
  "summarization_mode": "summary"
}

Runtime Enforcement in guard-adapter.ts

The guard-adapter.ts module (MemoryProxy/src/guard-adapter.ts) calculates projected token usage for each request against the configured context_window. If injection blocks would exceed the budget, the guard either truncates content or aborts the call entirely. This guarantees that no request violates the LLM's context constraints, regardless of how much raw memory exists in storage.

Summary

TencentDB Agent Memory prevents context window overflow through a multi-layered defense system:

• Tiered storage (L0→L1→L2) keeps raw data at lower layers while exposing only summaries to the injection pipeline
• Automatic summarization via tdai-profile-memory-injector.ts replaces full conversation text with condensed <session_context> blocks
• Explicit trace summarization through the /memories/trace_summarize API allows on-demand compression of long-running sessions
• Token budget enforcement via cost-guard ensures runtime requests never exceed the configured context_window limit
• Periodic cleanup in memory-cleaner.ts removes obsolete shards to prevent unbounded storage growth

Frequently Asked Questions

What is the difference between L1 and L2 memory tiers?

L1 stores per-turn records containing structured metadata about individual interactions, while L2 aggregates multiple L1 entries into scenario-level shards that expose only paths and summaries. The L2 tier deliberately discards full text content, making it the only tier accessed during prompt injection while L1 serves as intermediate structured storage.

How does the cost-guard plugin handle token limit violations?

When guard-adapter.ts detects that injected content would exceed the context_window configuration, it either truncates the memory blocks to fit within the budget or aborts the request entirely. This ensures the LLM never receives a payload exceeding its capacity, acting as a final safety mechanism beyond the summarization pipeline.

Can developers force memory summarization manually?

Yes. The /memories/trace_summarize endpoint allows explicit generation of conversation summaries at any point. Developers specify turn ranges or omit parameters to summarize the entire session, producing compact representations that subsequent calls inject instead of raw history.

Where does the session context actually get inserted into the prompt?

The context-injector.ts module assembles the <session_context> XML fragment and appends it to the first system message in the message array. This occurs after the cost-guard validation but before the request leaves the proxy, ensuring the final payload contains minimal yet semantically rich memory references.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}