How TencentDB Agent Memory Implements a Dual-Layer Storage Strategy

TencentDB Agent Memory employs a dual-layer storage strategy that pairs TCVDB (Tencent Cloud Vector Database) for sub-millisecond similarity search with COS (Tencent Cloud Object Storage) for unlimited, cost-effective storage of full-text and binary payloads.

The TencentCloud/TencentDB-Agent-Memory repository architecturally separates indexing from persistence to solve the fundamental tension between retrieval speed and storage capacity. This dual-layer storage strategy ensures AI agents can perform high-recall semantic searches across millions of memories while retaining access to the complete conversation logs, wiki pages, and code graphs that provide rich context. According to the source code, this design is formalized in MemoryCore/openclaw-plugin/docs/design/2026-07-20-v3-plugin-install-dual-mode-design.md and implemented throughout the OpenClaw plugin integration.

The Two Storage Layers: Vector and Object

The system explicitly partitions memory assets into two complementary tiers. This separation allows each layer to optimize for its specific access pattern—random reads with complex similarity calculations versus sequential streaming of large objects.

Vector Layer: TCVDB for Semantic Search

The Vector Layer uses Tencent Cloud Vector Database (TCVDB) to store embeddings of atomic memory items, including L1 Atoms, L2 Scenarios, and L3 Personas. As implemented in the Gateway service, this layer maintains vector-search indexes that support hybrid ranking combining BM25 text matching and vector similarity (RRF). The Vector Layer exists to enable sub-millisecond, high-recall similarity search that powers the recall hooks in the OpenClaw plugin.

Object Layer: COS for Raw Assets

The Object Layer utilizes Tencent Cloud Object Storage (COS) as the authoritative store for raw files and large binary assets. This layer holds conversation logs, wiki pages, code-graph snapshots, and any payload too large for efficient vector storage. COS provides cheap, virtually unlimited capacity that scales to petabytes, ensuring the system never truncates historical context due to storage constraints.

Architecture: How the Layers Work Together

The dual-layer storage strategy operates through a three-tier architecture described in MemoryCore/openclaw-plugin/docs/architecture.md:

1. OpenClaw Plugin: Framework adaptation layer that imports MemoryClient and MemoryFileReader from the SDK.
2. SDK Layer: @tencentdb-agent-memory/memory-sdk-ts-v2 provides 14 API methods with zero framework dependencies.
3. Gateway v2: Stateless service coordinating VDB, COS, Redis, and Pipeline Workers.

Redis serves a critical but distinct role as a lightweight state layer for session IDs, ACL caches, and pipeline job status. It is explicitly excluded from the dual-layer storage core because it only coordinates transient state; the actual memory assets persist exclusively in TCVDB and COS.

Cross-Layer References: Every entry in the Vector Layer stores a tiny identifier pointing to its corresponding COS object. When client.searchAtomic() returns top-k results, the SDK lazily fetches only those specific COS objects via signed STS requests, minimizing network traffic while delivering complete context.

Implementation in the OpenClaw Plugin

The source files demonstrate how the dual-layer abstraction manifests in production code:

• MemoryCore/openclaw-plugin/src/hooks/recall.ts: Implements the recall hook that first queries the Vector Layer for similarity matches, then hydrates the results by fetching the corresponding COS payloads.
• MemoryCore/openclaw-plugin/src/tools/memory-search.ts: Wraps client.searchAtomic() to expose vector-only queries to the agent framework.
• MemoryCore/openclaw-plugin/src/tools/read-cos.ts: Provides direct access to the Object Layer for large documents that bypass vector search, such as complete wiki pages or code-graph blobs.

The migration guide at MemoryCore/scripts/migrate-v2-to-v3/README.md explains how this dual-layer design replaced an older flat-file storage system, splitting data into indexed vectors and archived objects to improve both performance and scalability.

Practical SDK Usage Examples

The following TypeScript snippets demonstrate how the SDK abstracts the dual-layer complexity, automatically obtaining temporary STS credentials and coordinating between TCVDB and COS.

import { MemoryClient, MemoryFileReader } from '@tencentdb-agent-memory/memory-sdk-ts-v2';

// Initialize client targeting the Gateway
const client = new MemoryClient({
  gatewayUrl: 'http://127.0.0.1:8420',
  apiKey: process.env.TEI_GATEWAY_KEY,
});

// 1. Capture conversation: writes raw text to COS, generates vector in TCVDB
await client.addConversation({
  sessionId: ctx.sessionKey,
  content: ctx.conversation,        // Stored as COS object
  metadata: { userId: ctx.userId }, // Indexed in TCVDB
});

// 2. Recall atomic facts: searches Vector Layer, then fetches COS payloads
const results = await client.searchAtomic({
  query: 'what is the default timeout for memory retrieval?',
  topK: 5,
});
for (const hit of results.hits) {
  // hit.id references a COS object; SDK handles STS-signed retrieval
  const payload = await client.readCos(hit.id);
  console.log('🔎', payload.text);
}

// 3. Direct Object Layer access for large files
const reader = new MemoryFileReader({ gatewayUrl: 'http://127.0.0.1:8420' });
const wikiPage = await reader.read('wiki_pages/architecture_overview.md');

The MemoryFileReader class specifically manages the STS (Security Token Service) credential exchange required to read from COS, ensuring clients never handle long-term secrets or construct raw HTTP requests to the Object Layer.

Summary

• Dual-Layer Design: TCVDB provides the Vector Layer for fast similarity search, while COS provides the Object Layer for durable, unlimited storage.
• State Management: Redis coordinates transient session state and ACL caching but does not participate in the persistent dual-layer storage core.
• Cross-Layer Coordination: Vector entries store COS identifiers, enabling the Gateway to fetch full payloads only for relevant search results.
• SDK Abstraction: The @tencentdb-agent-memory/memory-sdk-ts-v2 package hides implementation details, automatically handling STS authentication and layer-specific protocols.
• Scalability: Stateless Gateways allow horizontal scaling because persistent state resides in managed Tencent Cloud services (TCVDB and COS) rather than local disk.

Frequently Asked Questions

What are the two layers in the TencentDB Agent Memory dual-layer storage strategy?

The strategy consists of the Vector Layer, which uses TCVDB to store embeddings and similarity indexes for atomic memory items, and the Object Layer, which uses COS to store the raw text, binary assets, and full conversation logs referenced by those vectors. This separation optimizes for both retrieval speed and storage capacity.

How does the system ensure data consistency between the Vector and Object Layers?

Every vector record in TCVDB contains a unique identifier that points to its corresponding COS object. When the SDK executes searchAtomic(), it retrieves vectors from TCVDB, then fetches only the top-k COS objects using temporary STS credentials. This pointer-based architecture guarantees that recalled vectors always map to their authoritative full-text payloads.

Why is Redis excluded from the dual-layer storage core?

According to the architecture documentation, Redis functions exclusively as a lightweight state layer for temporary data such as session IDs, ACL caches, and pipeline job progress. It enables horizontal scaling of the stateless Gateway service but does not store the actual memory assets, which persist solely in the durable dual layers (TCVDB and COS).

Which source files implement the dual-layer retrieval logic?

The retrieval logic is implemented in MemoryCore/openclaw-plugin/src/hooks/recall.ts, which queries the Vector Layer and hydrates results from COS. Direct Object Layer access is handled in MemoryCore/openclaw-plugin/src/tools/read-cos.ts, while vector search operations are wrapped in MemoryCore/openclaw-plugin/src/tools/memory-search.ts. The design rationale is documented in MemoryCore/openclaw-plugin/docs/design/2026-07-20-v3-plugin-install-dual-mode-design.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}