# How Context Offloading Works in TencentDB Agent Memory: Architecture and Implementation

> Discover how TencentDB Agent Memory offloads context using a three-stage pipeline (Ingest, Compact, Query) for efficient long-horizon task processing and optimized token usage.

- Repository: [Tencent Cloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
- Tags: architecture
- Published: 2026-08-23

---

**TencentDB Agent Memory handles context offloading through a three-stage pipeline (Ingest, Compact, Query) that separates short-term LLM interactions from long-term knowledge storage, enabling efficient long-horizon task processing without exhausting token budgets.**

The **context offloading** subsystem in TencentDB Agent Memory is designed to manage long-horizon conversations by persisting raw interaction data and periodically compressing it into reusable context. This architecture allows the agent to maintain extensive conversation history while keeping active LLM prompts within token limits. The implementation spans the `MemoryCore` TypeScript server code and multi-language SDKs, with core orchestration handled by the offload server and pipeline worker.

## The Three Stages of Context Offloading

The context offloading workflow operates through three distinct phases, each exposed via RESTful endpoints in [`MemoryCore/src/offload_server/router.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/offload_server/router.ts).

### Ingestion Stage

When tool calls generate results that exceed immediate processing needs, the system captures this data via the **ingest** API. The client SDK calls `offload_ingest()`, which POSTs to `/v2/offload/ingest` with the session ID, instance ID, and raw message payload.

In [`MemoryCore/src/offload_server/router.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/offload_server/router.ts), the router matches this path and persists the data:

```typescript
// MemoryCore/src/offload_server/router.ts
if (pathname === "/v2/offload/ingest") {
  // store incoming messages under offload/<sessionId>/…
}

```

The raw files are stored at `offload/<sessionId>/…` paths for subsequent compaction, creating a durable record of the full interaction history without bloating the active LLM context window.

### Compaction Stage

The **compaction** process transforms stored raw data into condensed, token-efficient context summaries. Triggered via `POST /v2/offload/compact` (SDK method: `offload_compact()`), this stage runs asynchronously through the **pipeline worker** defined in [`MemoryCore/src/services/pipeline-worker.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/services/pipeline-worker.ts).

The worker consumes tasks from a Redis-backed queue and classifies them by offload type:

```typescript
// MemoryCore/src/services/pipeline-worker.ts
if (task.type === "offload-l1") {
  // lock-free path – directly execute compact logic
}
if (task.type === "offload-l2") {
  // acquire distributed lock, then compact
}

```

During execution, the appropriate executor (`executeOffloadL1`, `executeOffloadL2`, etc.) reads the raw offload files, processes them through an LLM or summarization model, and writes the **compact context** back to `offload/<sessionId>/compact`. This compressed representation strips irrelevant noise while preserving semantic meaning, significantly reducing token usage when injected back into subsequent prompts.

### Query MMD Stage

For debugging and observability, developers can visualize offloaded task flows through the **Query MMD** endpoint. The `offload_query_mmd()` SDK method POSTs to `/v2/offload/query-mmd`, which invokes the diagram generation logic in [`MemoryCore/src/offload_server/prompts/l2-prompt.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/offload_server/prompts/l2-prompt.ts).

This component assembles a Mermaid diagram from stored metadata, rendering the offloading pipeline's structure and state changes as a visual flowchart for troubleshooting long-horizon tasks.

## Task Types and Concurrency Control

The pipeline worker distinguishes three offload task categories to balance throughput and consistency:

### Offload-L1 (Lock-Free)

**Offload-L1** tasks execute without distributed locking, providing maximum throughput for sessions that don't require strict coordination. When the worker detects `task.type === "offload-l1"`, it bypasses lock acquisition and immediately executes the compaction:

```typescript
// MemoryCore/src/services/pipeline-worker.ts
if (lockKey === null) {               // lock-free
  await this.executeTask(task, undefined);
  await this.backend.ackTask(msgId);
}

```

This path suits fast compaction of small sessions where concurrent processing poses no conflict risk.

### Offload-L2 (Distributed Locking)

**Offload-L2** tasks acquire per-MMD locks to ensure exclusive access during heavy compaction operations involving substantial LLM calls. The worker implements lock-conflict handling with exponential back-off (lines 48-62) and re-enqueues tasks that fail lock acquisition beyond the TTL (lines 84-102).

### Offload-L15 (Specialized Pipelines)

**Offload-L15** represents worker-level lock-free specialized pipelines, typically reserved for telemetry or monitoring contexts that run orthogonal to primary compaction workflows.

## Resilience and Fault Tolerance

The context offloading system guarantees reliability through several mechanisms implemented in [`pipeline-worker.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/pipeline-worker.ts):

- **Lock-conflict resolution**: When distributed locks cannot be acquired, tasks either skip (for compatible types) or retry with exponential back-off
- **Re-enqueue on timeout**: Failed lock attempts beyond the TTL trigger re-enqueuing with new message IDs, preventing message loss
- **Dead-letter queuing**: Tasks exceeding `maxRetries` migrate to a dead-letter queue for manual inspection, ensuring no compaction operations disappear silently

These patterns ensure that **long-horizon contexts are reliably offloaded, compacted, and made available** without blocking the real-time LLM interaction loop.

## Practical Usage Examples

The following TypeScript examples demonstrate the complete context offloading lifecycle using the TencentDB Agent Memory SDK:

```typescript
import { MemoryClient } from "tencentdb-agent-memory";

// 1️⃣ Ingest raw tool output
await client.offload_ingest({
  sessionId: "session‑123",
  instanceId: "agent‑main",
  messages: [{ role: "assistant", content: "…tool result…" }],
});

// 2️⃣ Trigger compaction (often scheduled automatically)
const compacted = await client.offload_compact({
  sessionId: "session‑123",
  instanceId: "agent‑main",
});

// `compacted` holds a trimmed context that can be injected into subsequent prompts
await llm.chat({
  messages: [
    ...compacted.context,           // ← compacted context
    { role: "user", content: "Continue the conversation…" },
  ],
});

// 3️⃣ Retrieve a Mermaid diagram for debugging
const mmd = await client.offload_query_mmd({
  sessionId: "session‑123",
});
console.log(mmd.diagram);   // visualises the off‑load pipeline

```

## Summary

- **Context offloading** in TencentDB Agent Memory separates immediate LLM interactions from historical data through three stages: Ingest, Compact, and Query MMD.
- Raw tool outputs persist to `offload/<sessionId>/` via [`MemoryCore/src/offload_server/router.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/offload_server/router.ts) and are later compressed by the pipeline worker.
- The **pipeline worker** ([`MemoryCore/src/services/pipeline-worker.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/services/pipeline-worker.ts)) handles three task types—`offload-l1`, `offload-l2`, and `offload-l15`—with varying locking strategies to optimize concurrency.
- **Compaction** reduces token usage by summarizing raw interaction data into compact context files using executors like `executeOffloadL1` and `executeOffloadL2`.
- Resilience mechanisms include exponential back-off, re-enqueue logic, and dead-letter queues to ensure reliable processing of long-horizon tasks.

## Frequently Asked Questions

### What triggers context compaction in TencentDB Agent Memory?

Context compaction is typically triggered automatically by the system's scheduling logic or explicitly via the `offload_compact()` SDK method. The pipeline worker pulls tasks from a Redis-backed queue and processes them according to their assigned offload type (`offload-l1`, `offload-l2`, or `offload-l15`), executing the appropriate summarization logic to compress raw ingested data into token-efficient context.

### How does the system handle concurrent compaction requests?

The system uses distributed locking for `offload-l2` tasks to ensure exclusive access per MMD, while `offload-l1` tasks run lock-free for higher throughput. When lock acquisition fails, the worker implements exponential back-off and re-enqueues tasks with new IDs after timeout, preventing resource contention while maintaining processing guarantees.

### What is the difference between offload-l1 and offload-l2 tasks?

**Offload-l1** tasks execute without distributed locking, making them ideal for fast compaction of small sessions where conflicts are unlikely. **Offload-l2** tasks acquire per-MMD locks before execution, suitable for intensive compaction operations that involve heavy LLM calls and require exclusive access to prevent data corruption. The distinction is handled in [`MemoryCore/src/services/pipeline-worker.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/services/pipeline-worker.ts) (lines 301-361).

### How can developers debug offloaded context workflows?

Developers use the `offload_query_mmd()` method to generate Mermaid diagrams visualizing the offloaded task flow. This endpoint, implemented in [`MemoryCore/src/offload_server/prompts/l2-prompt.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/offload_server/prompts/l2-prompt.ts), renders the pipeline state and context transitions as a visual graph, enabling inspection of how raw data transforms through ingestion and compaction stages.