How the Conversation Archive and Skill Extraction Pipeline Works in TencentDB Agent Memory

The conversation archive and skill extraction pipeline captures real-time chat turns through the /v3/skill/conversation/add endpoint, normalizes 5-role message arrays for skill processing, and persists conversations to long-term storage via automatic or manual calls to /v3/skill/conversation/force-archive.

The TencentDB Agent Memory project implements a robust memory layer that transforms ephemeral dialogues into structured knowledge. The conversation archive and skill extraction pipeline serves as the bridge between live user interactions and persistent L0-to-L1 memory storage, ensuring every turn is processed by extraction skills before archival.

Three-Stage Pipeline Architecture

The pipeline operates through three distinct stages that handle ingestion, processing, and persistence.

Stage 1: Per-Turn Ingestion

After the UI finalizes a conversation turn, the TypeScript SDK validates required fields and transmits the payload. The conversationAdd method in sdk/memory-core/typescript/src/v3/skill-client.ts (lines 418-424) enforces the presence of session_id, user_id, team_id, agent_id, and the 5-role message array before posting to /v3/skill/conversation/add.

Stage 2: Message Normalization and Skill Extraction

The MemoryProxy receives the request through MemoryProxy/src/skill/handler-glue.ts, where normalizeConversation (defined in MemoryProxy/src/skill/normalize-conversation.ts) transforms the payload into the exact schema expected by Core. The proxy forwards the normalized data via postConversationAdd, triggering the skill dispatcher in MemoryProxy/src/skill/skill-bridge.ts. This component routes the request to the extract skill, which processes the turn's content to generate structured metadata alongside raw L0 messages.

Stage 3: Automatic and Manual Archiving

When a conversation concludes or meets specific buffer conditions, the system triggers archival. The proxy invokes postForceArchive (lines 690-696 in handler-glue.ts) to call /v3/skill/conversation/force-archive, flushing accumulated messages to permanent storage and notifying downstream services like the Knowledge service. Clients may also trigger this manually via the SDK's conversationForceArchive method.

Step-by-Step Data Flow

Understanding the exact sequence of operations clarifies how data moves from the client to persistent storage.

1. SDK Validation and Submission: The frontend calls conversationAdd, which validates required string fields and message format before issuing an HTTP POST to the Core endpoint.

2. Proxy Normalization: handler-glue.ts receives the request and invokes normalizeConversation to ensure the 5-role array (user, assistant, system, and tool calls) matches Core's strict schema.

3. Core Skill Execution: The Core service receives the normalized payload and executes the configured skill pipeline. The extract skill runs first, analyzing message content to produce structured results that accompany the raw conversation data.

4. Automatic Archive Trigger: After processing a turn, the proxy evaluates session state. If the conversation ends or buffer limits are reached, it automatically calls postForceArchive with the session_id and space_id to persist the L0 buffer.

5. Manual Archive Option: For explicit persistence needs, client applications invoke conversationForceArchive directly, passing space_id to identify the target knowledge space for long-term storage.

Implementation Examples

Ingesting Turns with conversationAdd

Use the TypeScript SDK to submit conversation fragments with proper role attribution:

import { MemoryCoreClient } from '@tencent/memory-core';

const client = new MemoryCoreClient({ baseURL: 'https://core.example.com' });

await client.conversationAdd({
  session_id: 'sess-123',
  user_id:    'u-456',
  team_id:    't-789',
  agent_id:   'a-001',
  messages: [
    { role: 'user', content: 'How do I reset my password?' },
    { role: 'assistant', content: 'You can click “Forgot password”…' }
  ],
});

This validates required fields and posts to /v3/skill/conversation/add.

Triggering Manual Archiving

Persist sessions immediately using the force-archive method:

await client.conversationForceArchive({
  session_id: 'sess-123',
  space_id:   'space-xyz',   // identifies the knowledge space
  team_id:    't-789',
  agent_id:   'a-001',
});

This triggers the Core to flush buffered L0 messages and initiate downstream processing.

Proxy-Level Archive Logic

The proxy determines archival necessity before forwarding to Core:

// Simplified excerpt from MemoryProxy/src/skill/handler-glue.ts
if (needArchive) {
  await coreClient.postForceArchive({
    session_id: sessionKey,
    space_id:   spaceId,
  });
  log.info(`[skill-conversation-add] archived session=${sessionKey}`);
}

This logic evaluates session completion states to decide between automatic buffering and immediate persistence.

Key Source Files

File Path	Responsibility
sdk/memory-core/typescript/src/v3/skill-client.ts	SDK wrapper for conversationAdd and conversationForceArchive
MemoryProxy/src/skill/handler-glue.ts	Request routing, normalization orchestration, and archive decision logic
MemoryProxy/src/skill/normalize-conversation.ts	Schema validation and 5-role message array normalization
MemoryProxy/src/skill/skill-bridge.ts	Skill pipeline dispatch, including the extract skill route
MemoryCore/src/core/skill/conversation-add/add-handler.ts	Core handler processing conversation additions
MemoryCore/src/core/skill/conversation-force-archive/handler.ts	Core handler executing force-archive operations

Summary

• The conversation archive and skill extraction pipeline processes every chat turn through /v3/skill/conversation/add, ensuring 5-role message arrays are normalized before skill execution.
• The extract skill runs during ingestion to generate structured metadata from raw L0 conversation data.
• Automatic archiving flushes buffers to permanent storage when sessions end, while manual archiving via conversationForceArchive allows explicit persistence control.
• The MemoryProxy (handler-glue.ts) serves as the orchestration layer between SDK clients and the Core service, handling normalization and archive triggers.

Frequently Asked Questions

What distinguishes the conversation/add endpoint from force-archive?

The /v3/skill/conversation/add endpoint handles per-turn ingestion and immediate skill extraction, buffering L0 data in temporary storage. In contrast, /v3/skill/conversation/force-archive persists the accumulated buffer to long-term storage and triggers downstream knowledge processing, effectively finalizing the session's memory lifecycle.

How does the pipeline handle complex message roles?

According to MemoryProxy/src/skill/normalize-conversation.ts, the pipeline normalizes inputs into a 5-role message array that includes standard roles (user, assistant, system) plus tool call representations. This normalization ensures the Core service receives a consistent schema regardless of frontend formatting variations.

When should applications use manual archiving instead of automatic archiving?

Manual archiving via conversationForceArchive is required when explicit user actions (like clicking "Save") must persist a conversation before natural session termination, or when specific space_id targeting is needed for knowledge space organization beyond the automatic routing logic in handler-glue.ts.

What happens to extracted skills after the archiving process completes?

Once force-archive executes, the Core service writes both raw L0 messages and structured skill extractions to permanent storage, then notifies downstream services such as the Knowledge service. This makes the extracted patterns available for retrieval in future conversation contexts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}