Skill Extraction Workflow in TencentDB Agent Memory: How to Use `/extract`, `/conversation/add`, and `/conversation/force-archive`

The skill extraction pipeline in TencentDB Agent Memory uses three coordinated HTTP endpoints: /extract for asynchronous batch processing, /conversation/add for incremental turn ingestion with automatic threshold-based extraction, and /conversation/force-archive for manual buffer flushing.

The TencentDB Agent Memory system implements a sophisticated skill extraction workflow designed to capture reusable procedural knowledge from conversational sessions. This workflow balances automation with manual control, allowing agents to incrementally build skill candidates while providing escape hatches for explicit management. The three primary endpoints—/extract, /conversation/add, and /conversation/force-archive—work together to ensure no conversation data is lost while minimizing unnecessary processing overhead.

Architecture Overview: The Three-Stage Pipeline

The skill extraction pipeline operates across two architectural layers: the MemoryProxy (edge-facing) and MemoryCore (central processing). Understanding how requests flow between these layers clarifies when to use each endpoint.

Endpoint	Layer	Primary Function	Trigger Mode
POST /v3/skill/conversation/add	Proxy → Core	Incremental turn ingestion	Automatic per conversation turn
POST /v3/skill/extract	Core (via Proxy)	Asynchronous skill extraction	Automatic on threshold breach or Manual explicit call
POST /v3/skill/conversation/force-archive	Proxy → Core	Immediate buffer archival	Manual on-demand

Request Flow Through the System

When a client sends a conversation turn, the path through the codebase follows this sequence:

1. SDK ([skill-client.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/sdk/memory-core/typescript/src/v3/skill-client.ts), line 394) → constructs normalized 5-role message array
2. Proxy Bridge ([skill-bridge.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/skill/skill-bridge.ts), lines 690-696) → validates session isolation, enriches headers
3. Core Handler ([skill-handlers.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-handlers.ts), line 113) → persists to L0 memory, evaluates thresholds
4. Auto-trigger ([workbuddyHandler.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/workbuddyHandler.ts), lines 393-403) → conditionally delegates to /extract

Endpoint Deep Dive: /conversation/add

The /conversation/add endpoint serves as the primary ingestion point for conversational data. Unlike simple logging, this endpoint performs schema validation, memory tiering, and threshold monitoring in a single operation.

Request Schema: 5-Role Normalized Format

TencentDB Agent Memory requires a strict normalized format for all conversation messages. This ensures consistent processing regardless of the original LLM provider's native format.

// sdk/memory-core/typescript/src/v3/skill-types.ts (lines 322-400)
interface ConversationAddRequest {
  session_id: string;      // Required: isolation boundary for conversation
  space_id: string;        // Required: multi-tenant namespace
  messages: Array<{
    role: 'user' | 'assistant' | 'system' | 'tool_call' | 'tool_result';
    content: string;
    name?: string;         // For tool_call: function name
    tool_call_id?: string; // For correlation between tool_call and tool_result
    metadata?: Record<string, unknown>;
  }>;
  // Optional: override threshold detection for this specific call
  force_extraction?: boolean;
}

Processing Logic in Core

The handler at [MemoryCore/src/gateway/skill-handlers.ts line 113](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-handlers.ts) performs these operations atomically:

1. Schema validation against [skill-schemas.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-schemas.ts) definitions
2. L0 memory write: Append to short-term session buffer (in-memory + persisted)
3. Threshold evaluation: Check accumulated token count, tool-call density, or explicit force_extraction flag
4. Conditional extract delegation: If thresholds met, enqueue async extraction job

Automatic Extraction Triggers

The proxy-side logic in [workbuddyHandler.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/workbuddyHandler.ts) implements the auto-trigger heuristics:

// Simplified excerpt from lines 393-403
if (sessionBuffer.tokenCount > SKILL_ARCHIVE_TOKEN_THRESHOLD ||
    sessionBuffer.toolCallCount > SKILL_ARCHIVE_TOOL_THRESHOLD ||
    request.force_extraction === true) {
  // Delegate to core extraction without blocking response
  this.skillBridge.triggerExtract(sessionId, spaceId, {
    source: 'auto_threshold',
    bufferSnapshot: sessionBuffer.snapshot()
  });
}

Key design decision: The extraction runs asynchronously—the /conversation/add response returns immediately with extraction_queued: true, allowing the conversation to continue without waiting for skill processing.

Endpoint Deep Dive: /extract

The /extract endpoint initiates explicit skill extraction from a specified conversation buffer. While often triggered automatically, direct invocation provides control over timing, scope, and custom message collections.

Direct Invocation Use Cases

• Historic conversation import: Process pre-existing conversation logs not captured live
• Cross-session skill mining: Extract patterns across multiple related sessions
• Debugging and iteration: Re-run extraction with modified parameters after tuning

Request Structure

// sdk/memory-core/typescript/src/v3/skill-client.ts (line 370)
interface ExtractRequest {
  session_id: string;
  space_id: string;
  // Optional: override with custom message array instead of buffered session
  messages?: Array<NormalizedMessage>;
  // Optional: constrain extraction to specific skill types
  skill_type_filters?: Array<'procedure' | 'constraint' | 'preference'>;
  // Optional: priority hint for job scheduler
  priority?: 'default' | 'background' | 'urgent';
}

Asynchronous Processing Model

Extraction jobs run in a background worker pool managed by the core service. The immediate response contains:

{
  "job_id": "extract-job-uuid-v4",
  "status": "queued",
  "estimated_completion": "2024-01-15T09:23:00Z",
  "poll_endpoint": "/v3/skill/extract/jobs/extract-job-uuid-v4"
}

Historical note: Earlier versions exposed /skill/extract/result for polling; this has been deprecated in favor of the unified job status endpoint.

Endpoint Deep Dive: /conversation/force-archive

The /conversation/force-archive endpoint addresses a critical operational necessity: guaranteed buffer persistence regardless of threshold state. This endpoint has no message payload—its sole purpose is state management.

When to Force Archive

Scenario	Why Force Archive Matters
Session termination	Ensure incomplete turns aren't lost when user disconnects
Bulk import preparation	Clear residual buffer before importing historic data (see [agents/asset-import.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/agents/asset-import.ts))
Multi-device handoff	Synchronize buffer state across client instances
Debugging and audit	Capture exact buffer state for inspection

Implementation Details

The handler at [skill-handlers.ts line 1156](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-handlers.ts) and its proxy delegate in [core-client.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/skill/core-client.ts) (lines 283-290) implement this as:

1. Buffer lock acquisition: Prevent concurrent writes during archival
2. Snapshot persistence: Write in-memory buffer to skill store with archived_at timestamp
3. Buffer reset: Clear session-local accumulator without destroying session metadata
4. Optional extraction trigger: If buffer non-empty, enqueue final extraction job

// sdk/memory-core/typescript/src/v3/skill-client.ts (line 424)
await client.forceArchive({
  session_id: 'sess-123',
  space_id: 'space-abc',
  // Optional: also trigger extraction on archived buffer
  trigger_extract: true
});

Complete Workflow Example

This TypeScript example demonstrates a realistic session lifecycle using all three endpoints:

import { SkillClient } from '@tencentdb/memory-core';

const client = new SkillClient({
  baseUrl: 'https://memory.tencentcloudapi.com',
  credentials: { secretId: process.env.TENCENT_SECRET_ID, secretKey: process.env.TENCENT_SECRET_KEY }
});

async function managedConversationSession() {
  const sessionId = `sess-${Date.now()}`;
  const spaceId = 'prod-customer-support';

  try {
    // === Phase 1: Incremental ingestion with automatic extraction ===
    for (const turn of liveConversation.turns) {
      const result = await client.conversationAdd({
        session_id: sessionId,
        space_id: spaceId,
        messages: normalizeToFiveRole(turn) // Convert from OpenAI/Claude format
      });
      
      if (result.extraction_queued) {
        console.log(`Auto-extraction triggered at turn ${turn.index}`);
      }
    }

    // === Phase 2: Explicit extraction of refined scope ===
    // Re-run extraction with custom parameters after session completion
    const extractJob = await client.extract({
      session_id: sessionId,
      space_id: spaceId,
      skill_type_filters: ['procedure'], // Focus on procedural skills only
      priority: 'background'
    });
    
    // Poll or webhook-await completion...
    await waitForExtraction(extractJob.job_id);

    // === Phase 3: Guaranteed cleanup before session destruction ===
    await client.forceArchive({
      session_id: sessionId,
      space_id: spaceId,
      trigger_extract: false // Already extracted above
    });

  } catch (error) {
    // Emergency force-archive on failure path
    await client.forceArchive({ session_id: sessionId, space_id: spaceId });
    throw error;
  }
}

Key Implementation Files

File	Role in Workflow	Critical Lines
[MemoryCore/src/gateway/skill-handlers.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-handlers.ts)	HTTP request dispatch for all three endpoints	113 (/conversation/add), 1154 (/extract), 1156 (/force-archive)
[MemoryCore/src/gateway/skill-schemas.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryCore/src/gateway/skill-schemas.ts)	JSON Schema validation for request bodies	5-role message format, isolation ID requirements
[MemoryProxy/src/skill/skill-bridge.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/skill/skill-bridge.ts)	Cross-layer request forwarding and header enrichment	690-696 (sub-path routing for force-archive)
[MemoryProxy/src/skill/core-client.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/skill/core-client.ts)	Low-level client for core service communication	283-290 (force-archive buffer clearing)
[MemoryProxy/src/workbuddyHandler.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryProxy/src/workbuddyHandler.ts)	Auto-trigger orchestration after threshold detection	393-403 (threshold evaluation and extract delegation)
[sdk/memory-core/typescript/src/v3/skill-client.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/sdk/memory-core/typescript/src/v3/skill-client.ts)	High-level SDK with typed methods	370 (extract), 394 (conversationAdd), 424 (forceArchive)
[sdk/memory-core/typescript/src/v3/skill-types.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/sdk/memory-core/typescript/src/v3/skill-types.ts)	TypeScript interface definitions	272-408 (request/response types), 322-400 (5-role schema)

Summary

• /conversation/add is the default ingestion path for live conversations, automatically triggering extraction when configurable thresholds are exceeded.
• /extract provides explicit control over extraction timing and scope, essential for batch processing and debugging scenarios.
• /conversation/force-archive guarantees buffer persistence on demand, critical for session lifecycle management and operational safety.
• The 5-role normalized format (user, assistant, system, tool_call, tool_result) ensures consistent processing across LLM providers and tool frameworks.
• Automatic threshold detection (token count, tool-call density) minimizes unnecessary extraction overhead while capturing rich conversational contexts.

Frequently Asked Questions

What happens if I call /extract while an automatic extraction is already running for the same session?

The core service implements job deduplication per (session_id, space_id) pair. If an extraction job is already queued or running for a session, subsequent requests return the existing job_id rather than creating duplicate work. For forced re-extraction with modified parameters, include force_new_job: true in the request body.

Can I use /conversation/add without triggering automatic extraction?

Yes. Set force_extraction: false explicitly in the request, or configure threshold values to artificially high limits for the session via the session_config parameter. However, this is generally discouraged—unarchived buffers reside only in L0 memory and are subject to eviction under memory pressure.

How does /conversation/force-archive differ from simply calling /extract with the current buffer?

/extract creates a skill extraction job that analyzes the buffer for reusable procedural patterns and writes candidates to the skill store. /force-archive performs buffer state persistence without analysis—it's a lower-level operation that ensures no data loss. You typically call both: force-archive for guaranteed persistence, then extract if you want skill analysis.

What isolation guarantees do session_id and space_id provide?

The space_id implements hard multi-tenancy—data from different spaces is physically segregated at the storage layer. Within a space, session_id provides conversational scope—buffers, extraction jobs, and derived skills are namespaced per session. This allows concurrent processing of multiple user sessions without cross-contamination, as enforced by the isolation validation in [skill-types.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/sdk/memory-core/typescript/src/v3/skill-types.ts) lines 272-408.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}