Where to Find the Full Extraction Pipeline Code in TencentDB-Agent-Memory

The complete extraction pipeline spans the MemoryProxy and MemoryCore packages, with the main entry point at MemoryProxy/src/extraction-gate.ts and UI-specific parsers located in MemoryProxy/src/session/codebuddy/extractor.ts, claude-code/extractor.ts, and the generic extractor.ts.

The TencentDB-Agent-Memory repository implements a sophisticated extraction pipeline that transforms raw user messages into structured session data representing team, agent, and task selections. This pipeline handles multiple UI formats including CodeBuddy and Claude-Code while providing fallback mechanisms for free-form text. Understanding the exact file locations and processing flow is essential for developers extending the system or debugging extraction failures.

Architecture Overview of the Extraction Pipeline

The extraction pipeline operates through three logical stages: request routing through a central gate, format-specific parsing of option text, and structured fallback extraction. According to the TencentDB-Agent-Memory source code, these stages are distributed across the MemoryProxy package with enrichment layers in MemoryCore.

Stage 1: Entry Point and Request Routing

All inbound messages first hit the central dispatcher in MemoryProxy/src/extraction-gate.ts. This gate determines which concrete extractor to invoke based on two factors: the current session phase (team selection, agent selection, or task selection) and the UI source identifier (CodeBuddy, Claude-Code, etc.).

The gate acts as a traffic controller, ensuring that XML payloads from CodeBuddy interfaces route to the XML-aware parser, while JSON responses from Claude-Code tooling flow to the appropriate JSON handler.

Stage 2: UI-Specific Option Parsing

Once routed, messages undergo heavy parsing in source-specific extractor modules. These files contain the core logic for transforming UI payloads into structured identities.

CodeBuddy XML and JSON Processing

For CodeBuddy interfaces, the pipeline delegates to MemoryProxy/src/session/codebuddy/extractor.ts. This module handles:

• XML parsing of <question_answer> payloads via parseQuestionAnswerXml
• JSON parsing of multi_question_result structures
• String matching against cached team/agent/task labels using exact label matching, ID-suffix matching, and longest-name-fallback strategies
• Special markers including __bypass__ (skip extraction) and __more__ (pagination controls)

Claude-Code JSON Handling

For Claude-Code integrations, MemoryProxy/src/session/claude-code/extractor.ts processes AskUserQuestion tool results and multi_question_result JSON objects. It extracts the agent and task fields from the questions array and maps them to internal identifiers using the same caching and fuzzy-matching strategies as the CodeBuddy implementation.

Generic Fallback Parser

The default implementation in MemoryProxy/src/session/extractor.ts provides a generic option-text parser that serves as the foundation for both specialized extractors. It handles standard string normalization and cache lookups when UI-specific formatting is absent.

Stage 3: Structured Fallback Extraction

When UI-driven parsers cannot produce a valid result, the gate falls back to extractStructured within MemoryProxy/src/session/extractor.ts. This regex-based parser scans free-form text for agent: and task: tokens, enabling extraction from unstructured user inputs when formal UI payloads are unavailable.

MemoryCore Enrichment and Intent Extraction

Beyond the basic team-agent-task extraction in MemoryProxy, the MemoryCore package adds higher-level semantic extraction that enriches session data with inferred intent and contextual awareness.

Skill Extraction and Command Parsing

The MemoryCore/src/core/skill/skill-extractor.ts module parses skill-related commands (such as "search" or "analyze") and maps them to internal skill IDs. This layer transforms natural language requests into actionable skill invocations.

Scene Detection and Prompt Selection

Scene extraction occurs in MemoryCore/src/core/scene/scene-extractor.ts, which detects conversational contexts (such as "debug" or "review" modes) and selects appropriate prompt templates. The actual prompt templates reside in MemoryCore/src/core/prompts/scene-extraction.ts and l1-extraction.ts, providing the textual instructions that drive LLM-based extraction when rule-based parsing is insufficient.

Low-level traceability extraction happens in MemoryCore/src/core/record/l1-extractor.ts, which extracts L1 information from raw messages for logging and observability purposes.

Practical Implementation Examples

The following examples demonstrate how to invoke the extraction pipeline programmatically:

import { extractIdentity } from "MemoryProxy/src/session/extractor.js";

// Structured free-form text extraction
const raw1 = "agent: agent_bug_fixer task: task_12345678";
const result1 = await extractIdentity(raw1);
// → { agent_id: "agent_bug_fixer", task_id: "task_12345678" }

// CodeBuddy XML payload processing
const raw2 = `<question_answer>…<answers>Bug Fixer — 自动定位并修复代码缺陷</answers></question_answer>`;
const result2 = await extractIdentity(raw2);
// → { agent_id: "agent_bug_fixer", task_id: undefined }

// Claude-Code JSON result parsing
const raw3 = JSON.stringify({
  result: {
    type: "multi_question_result",
    questions: [
      { id: "agent", answer: "Bug Fixer" },
      { id: "task", answer: "Fix memory leak" }
    ]
  }
});
const result3 = await extractIdentity(raw3);
// → { agent_id: "agent_bug_fixer", task_id: "task_abcdef12" }

These calls route through extraction-gate.ts, which selects the appropriate parser based on payload structure and returns a SessionInitData object ready for downstream processing.

Summary

• Entry Point: MemoryProxy/src/extraction-gate.ts serves as the central dispatcher for all extraction requests.
• UI-Specific Parsers: CodeBuddy XML/JSON handling resides in MemoryProxy/src/session/codebuddy/extractor.ts, while Claude-Code processing lives in MemoryProxy/src/session/claude-code/extractor.ts.
• Core Logic: Generic parsing and structured fallbacks are implemented in MemoryProxy/src/session/extractor.ts.
• Enrichment Layer: MemoryCore adds skill detection (skill-extractor.ts), scene detection (scene-extractor.ts), and prompt templates (prompts/*.ts) for LLM-based extraction.
• Output Format: All paths converge on producing a structured SessionInitData object containing team, agent, and task identifiers.

Frequently Asked Questions

Where is the main entry point for the extraction pipeline in TencentDB-Agent-Memory?

The main entry point is MemoryProxy/src/extraction-gate.ts. This file contains the routing logic that inspects incoming messages and delegates to the appropriate UI-specific extractor based on the session phase and source interface.

How does the pipeline handle different UI sources like CodeBuddy versus Claude-Code?

The pipeline maintains separate extractor modules for each UI format. CodeBuddy XML and JSON payloads route to MemoryProxy/src/session/codebuddy/extractor.ts, which handles <question_answer> structures, while Claude-Code JSON responses process through MemoryProxy/src/session/claude-code/extractor.ts for AskUserQuestion tool results. Both implement the same caching and fuzzy-matching strategies but specialize in their respective data formats.

What happens when the automatic parser fails to identify an agent or task?

When UI-specific parsers return empty results, the gate falls back to extractStructured in MemoryProxy/src/session/extractor.ts. This regex-based extractor scans for agent: and task: tokens in free-form text, providing a robust fallback for unstructured inputs or malformed payloads.

Where are the LLM prompt templates located for scene and skill extraction?

Prompt templates for the enrichment layer reside in MemoryCore/src/core/prompts/. Specifically, scene-extraction.ts contains templates for conversational context detection, while l1-extraction.ts provides prompts for low-level information extraction used in logging and traceability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}