How Context Offloading Works in TencentDB Agent Memory: Architecture and Implementation

TencentDB Agent Memory handles context offloading through a three-stage pipeline (Ingest, Compact, Query) that separates short-term LLM interactions from long-term knowledge storage, enabling efficient long-horizon task processing without exhausting token budgets.

The context offloading subsystem in TencentDB Agent Memory is designed to manage long-horizon conversations by persisting raw interaction data and periodically compressing it into reusable context. This architecture allows the agent to maintain extensive conversation history while keeping active LLM prompts within token limits. The implementation spans the MemoryCore TypeScript server code and multi-language SDKs, with core orchestration handled by the offload server and pipeline worker.

The Three Stages of Context Offloading

The context offloading workflow operates through three distinct phases, each exposed via RESTful endpoints in MemoryCore/src/offload_server/router.ts.

Ingestion Stage

When tool calls generate results that exceed immediate processing needs, the system captures this data via the ingest API. The client SDK calls offload_ingest(), which POSTs to /v2/offload/ingest with the session ID, instance ID, and raw message payload.

In MemoryCore/src/offload_server/router.ts, the router matches this path and persists the data:

// MemoryCore/src/offload_server/router.ts
if (pathname === "/v2/offload/ingest") {
  // store incoming messages under offload/<sessionId>/…
}

The raw files are stored at offload/<sessionId>/… paths for subsequent compaction, creating a durable record of the full interaction history without bloating the active LLM context window.

Compaction Stage

The compaction process transforms stored raw data into condensed, token-efficient context summaries. Triggered via POST /v2/offload/compact (SDK method: offload_compact()), this stage runs asynchronously through the pipeline worker defined in MemoryCore/src/services/pipeline-worker.ts.

The worker consumes tasks from a Redis-backed queue and classifies them by offload type:

// MemoryCore/src/services/pipeline-worker.ts
if (task.type === "offload-l1") {
  // lock-free path – directly execute compact logic
}
if (task.type === "offload-l2") {
  // acquire distributed lock, then compact
}

During execution, the appropriate executor (executeOffloadL1, executeOffloadL2, etc.) reads the raw offload files, processes them through an LLM or summarization model, and writes the compact context back to offload/<sessionId>/compact. This compressed representation strips irrelevant noise while preserving semantic meaning, significantly reducing token usage when injected back into subsequent prompts.

Query MMD Stage

For debugging and observability, developers can visualize offloaded task flows through the Query MMD endpoint. The offload_query_mmd() SDK method POSTs to /v2/offload/query-mmd, which invokes the diagram generation logic in MemoryCore/src/offload_server/prompts/l2-prompt.ts.

This component assembles a Mermaid diagram from stored metadata, rendering the offloading pipeline's structure and state changes as a visual flowchart for troubleshooting long-horizon tasks.

Task Types and Concurrency Control

The pipeline worker distinguishes three offload task categories to balance throughput and consistency:

Offload-L1 (Lock-Free)

Offload-L1 tasks execute without distributed locking, providing maximum throughput for sessions that don't require strict coordination. When the worker detects task.type === "offload-l1", it bypasses lock acquisition and immediately executes the compaction:

// MemoryCore/src/services/pipeline-worker.ts
if (lockKey === null) {               // lock-free
  await this.executeTask(task, undefined);
  await this.backend.ackTask(msgId);
}

This path suits fast compaction of small sessions where concurrent processing poses no conflict risk.

Offload-L2 (Distributed Locking)

Offload-L2 tasks acquire per-MMD locks to ensure exclusive access during heavy compaction operations involving substantial LLM calls. The worker implements lock-conflict handling with exponential back-off (lines 48-62) and re-enqueues tasks that fail lock acquisition beyond the TTL (lines 84-102).

Offload-L15 (Specialized Pipelines)

Offload-L15 represents worker-level lock-free specialized pipelines, typically reserved for telemetry or monitoring contexts that run orthogonal to primary compaction workflows.

Resilience and Fault Tolerance

The context offloading system guarantees reliability through several mechanisms implemented in pipeline-worker.ts:

• Lock-conflict resolution: When distributed locks cannot be acquired, tasks either skip (for compatible types) or retry with exponential back-off
• Re-enqueue on timeout: Failed lock attempts beyond the TTL trigger re-enqueuing with new message IDs, preventing message loss
• Dead-letter queuing: Tasks exceeding maxRetries migrate to a dead-letter queue for manual inspection, ensuring no compaction operations disappear silently

These patterns ensure that long-horizon contexts are reliably offloaded, compacted, and made available without blocking the real-time LLM interaction loop.

Practical Usage Examples

The following TypeScript examples demonstrate the complete context offloading lifecycle using the TencentDB Agent Memory SDK:

import { MemoryClient } from "tencentdb-agent-memory";

// 1️⃣ Ingest raw tool output
await client.offload_ingest({
  sessionId: "session‑123",
  instanceId: "agent‑main",
  messages: [{ role: "assistant", content: "…tool result…" }],
});

// 2️⃣ Trigger compaction (often scheduled automatically)
const compacted = await client.offload_compact({
  sessionId: "session‑123",
  instanceId: "agent‑main",
});

// `compacted` holds a trimmed context that can be injected into subsequent prompts
await llm.chat({
  messages: [
    ...compacted.context,           // ← compacted context
    { role: "user", content: "Continue the conversation…" },
  ],
});

// 3️⃣ Retrieve a Mermaid diagram for debugging
const mmd = await client.offload_query_mmd({
  sessionId: "session‑123",
});
console.log(mmd.diagram);   // visualises the off‑load pipeline

Summary

• Context offloading in TencentDB Agent Memory separates immediate LLM interactions from historical data through three stages: Ingest, Compact, and Query MMD.
• Raw tool outputs persist to offload/<sessionId>/ via MemoryCore/src/offload_server/router.ts and are later compressed by the pipeline worker.
• The pipeline worker (MemoryCore/src/services/pipeline-worker.ts) handles three task types—offload-l1, offload-l2, and offload-l15—with varying locking strategies to optimize concurrency.
• Compaction reduces token usage by summarizing raw interaction data into compact context files using executors like executeOffloadL1 and executeOffloadL2.
• Resilience mechanisms include exponential back-off, re-enqueue logic, and dead-letter queues to ensure reliable processing of long-horizon tasks.

Frequently Asked Questions

What triggers context compaction in TencentDB Agent Memory?

Context compaction is typically triggered automatically by the system's scheduling logic or explicitly via the offload_compact() SDK method. The pipeline worker pulls tasks from a Redis-backed queue and processes them according to their assigned offload type (offload-l1, offload-l2, or offload-l15), executing the appropriate summarization logic to compress raw ingested data into token-efficient context.

How does the system handle concurrent compaction requests?

The system uses distributed locking for offload-l2 tasks to ensure exclusive access per MMD, while offload-l1 tasks run lock-free for higher throughput. When lock acquisition fails, the worker implements exponential back-off and re-enqueues tasks with new IDs after timeout, preventing resource contention while maintaining processing guarantees.

What is the difference between offload-l1 and offload-l2 tasks?

Offload-l1 tasks execute without distributed locking, making them ideal for fast compaction of small sessions where conflicts are unlikely. Offload-l2 tasks acquire per-MMD locks before execution, suitable for intensive compaction operations that involve heavy LLM calls and require exclusive access to prevent data corruption. The distinction is handled in MemoryCore/src/services/pipeline-worker.ts (lines 301-361).

How can developers debug offloaded context workflows?

Developers use the offload_query_mmd() method to generate Mermaid diagrams visualizing the offloaded task flow. This endpoint, implemented in MemoryCore/src/offload_server/prompts/l2-prompt.ts, renders the pipeline state and context transitions as a visual graph, enabling inspection of how raw data transforms through ingestion and compaction stages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}