How L0 Memory Write-Back Works Asynchronously in TencentDB Agent Memory

TencentDB Agent Memory performs L0 memory write-back asynchronously through a dedicated extraction stage in the MemoryProxy pipeline that fires a background POST request to the MemoryCore service immediately after each human turn completes, ensuring LLM inference latency remains unaffected by storage operations.

The TencentCloud/TencentDB-Agent-Memory repository implements a tiered memory architecture where short-term conversation history (L0) must persist without blocking the critical path of large language model interactions. This design leverages an asynchronous extraction pattern that decouples storage latency from response generation, enabling smooth interactive experiences even under high load.

Understanding the L0 Memory Architecture

The L0 layer represents hot storage for immediate conversation context, managed by the separate MemoryCore service. Unlike synchronous persistence patterns that stall the response pipeline while waiting for disk writes, TencentDB Agent Memory adopts an async-first approach. The MemoryProxy acts as an intermediary, orchestrating a five-stage request pipeline where the extraction stage specifically handles L0 memory write-back without blocking the forward request.

The Asynchronous Write-Back Pipeline

When an LLM request passes through the MemoryProxy, the system executes a strict sequence of operations. The asynchronous nature of L0 persistence is encapsulated in the extraction stage, which operates as a background task after the LLM completes its response.

Session Initialization

The proxy first creates a session context that combines team, agent, and user identifiers. This step resolves the current session key, ensuring that L0 memory remains properly isolated per conversation thread according to the multi-tenancy requirements outlined in the source configuration.

Context Injection

Before forwarding to the LLM provider, the proxy fetches relevant L0 and L1 memories from MemoryCore and injects them into the prompt. This retrieval happens synchronously because the LLM requires immediate access to historical context, but it occurs before any new data needs persistence.

Extraction and Async Write-Back

After the LLM finishes processing a human turn, the proxy triggers the critical extraction stage. This stage captures the newly generated conversation slice and asynchronously POSTs it to the MemoryCore endpoint POST /v3/skill/conversation/add. As documented in MemoryProxy/README.md, this step runs as a fire-and-forget background operation, allowing the forward request to proceed immediately without waiting for storage confirmation. The MemoryProxy/v3-api-memoryproxy-doc.md file illustrates this pipeline diagram showing the extraction node operating independently of the main request flow.

Authentication and Usage Reporting

Finally, the request passes through authentication headers addition and usage statistics reporting before reaching the upstream LLM provider. By this point, the async write-back task has already been dispatched to MemoryCore.

Configuration and Implementation Examples

Enabling Async Write-Back in Pipeline Configuration

The extraction stage is configurable through YAML pipeline definitions in the MemoryProxy. The extraction entry must appear after the injection stage to ensure proper sequencing:

# MemoryProxy pipeline configuration

pipeline:
  - name: session_init      # creates the session context

  - name: injection         # injects L0/L1 memories into the prompt

  - name: extraction        # ← async L0 write-back (conversation add)

  - name: authentication    # adds auth headers

  - name: reporting         # usage statistics

The extraction stage triggers the asynchronous call to MemoryCore’s conversation add endpoint, executing the L0 memory write-back without blocking subsequent requests.

Direct API Invocation via TypeScript SDK

For direct integration bypassing the proxy, the TypeScript client provides the addConversation method that mirrors the proxy's internal behavior. This implementation resides in sdk/memory-core/typescript/src/v3/client.ts:

import { MemoryClient } from '@tencentdb/memory-core';

// Initialise a strict-session client
const client = new MemoryClient({
  endpoint: 'https://memorycore.example.com',
  teamId: 'team-123',
  agentId: 'agent-abc',
  userId: 'user-xyz',
  sessionId: 'sess-456'   // L0 is isolated per session
});

// Payload generated after a human turn
const conversationAdd = {
  messages: [
    { role: 'user', content: 'What is the price of DB-5?' },
    { role: 'assistant', content: 'The price is $0.23 per hour.' }
  ],
  metadata: { source: 'proxy-extraction' }
};

// Asynchronous write-back (fire-and-forget)
client.addConversation(conversationAdd).catch(console.error);

The request returns immediately while the server persists the conversation to L0 storage, maintaining the same non-blocking guarantees as the proxy pipeline.

Client Integration with cURL

When invoking the proxy directly, the async write-back happens transparently after the response returns:

curl -X POST https://proxy.example.com/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_TOKEN>" \
  -d '{
        "model": "gpt-4",
        "messages": [{"role":"user","content":"Explain replicas"}]
      }'

# → The proxy injects L0 memories, forwards to the LLM,

#   then asynchronously writes the new turn back to L0.

Core Implementation Files

The following source files in the TencentCloud/TencentDB-Agent-Memory repository define the async L0 memory write-back behavior:

• MemoryProxy/README.md – Documents the pipeline stages, explicitly describing the extraction step as "async write-back of conversation + L0 after the turn."

• MemoryProxy/v3-api-memoryproxy-doc.md – Contains the detailed request flow diagram showing the extraction node operating asynchronously alongside the main request path.

• MemoryCore/src/utils/pipeline-manager.ts – Implements the scheduling logic that launches the async storage task during the extraction stage without awaiting completion.

• MemoryCore/src/utils/pipeline-factory.ts – Constructs the processing pipeline, including the registration of the extraction step that handles L0 persistence.

• sdk/memory-core/typescript/src/v3/client.ts – Provides the addConversation method used by both the proxy and direct clients for asynchronous conversation persistence.

Summary

• L0 memory write-back operates asynchronously through the extraction stage in MemoryProxy, ensuring zero impact on LLM response latency.

• The extraction stage fires a background POST request to POST /v3/skill/conversation/add immediately after each human turn completes, decoupling storage operations from inference.

• Configuration occurs through YAML pipeline definitions where the extraction step can be enabled, disabled, or reordered according to deployment requirements.

• The TypeScript SDK provides direct access to the same async write-back mechanism via the addConversation method in sdk/memory-core/typescript/src/v3/client.ts.

• Core scheduling logic resides in MemoryCore/src/utils/pipeline-manager.ts, which handles the fire-and-forget async tasks without blocking the main request thread.

Frequently Asked Questions

What triggers the asynchronous L0 memory write-back in TencentDB Agent Memory?

The write-back triggers immediately after the LLM completes processing a human turn. The MemoryProxy's extraction stage captures the conversation slice and initiates a background POST request to MemoryCore, allowing the system to return the LLM response to the user without waiting for storage confirmation.

How does async write-back affect LLM response latency?

Because the extraction stage operates as a background task after the turn completes, the LLM can immediately begin processing subsequent requests while the conversation persists to L0. This architecture effectively eliminates storage latency from the critical inference path, ensuring consistent response times regardless of backend load.

Which API endpoint and method handle the L0 conversation persistence?

The endpoint POST /v3/skill/conversation/add in MemoryCore handles persistence. Client applications can invoke this directly through the addConversation method exposed in sdk/memory-core/typescript/src/v3/client.ts, or rely on the MemoryProxy to trigger it automatically during the extraction pipeline stage.

Can the asynchronous L0 write-back behavior be customized or disabled?

Yes. Administrators can modify the pipeline configuration in MemoryProxy by editing the YAML stage definitions. Removing the extraction stage disables automatic L0 persistence, while reordering it changes when the async write-back occurs relative to authentication and reporting stages. However, disabling it prevents conversation history from persisting to the short-term memory layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}