# Pipeline Configuration for L1 Memory Extraction in TencentDB Agent Memory

> Configure the L1 memory extraction pipeline in TencentDB Agent Memory using ExtractionConfig and PipelineTriggerConfig. Control deduplication, model selection, and trigger intervals for efficient memory conversion.

- Repository: [Tencent Cloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
- Tags: how-to-guide
- Published: 2026-08-28

---

**The L1 memory extraction pipeline in TencentDB Agent Memory is configured through the typed `ExtractionConfig` and `PipelineTriggerConfig` sections in [`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts), controlling deduplication, model selection, and trigger intervals for converting raw L0 conversations into structured memories.**

TencentDB Agent Memory implements a hierarchical memory system that processes raw conversation logs (L0) into extracted memories (L1), scene blocks (L2), and personas (L3). The configuration driving this pipeline is defined in **[`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts)** and parsed at runtime by the plugin using **[`MemoryCore/src/utils/pipeline-manager.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/utils/pipeline-manager.ts)**. Understanding these configuration options allows developers to tune extraction frequency, deduplication strategies, and model selection for their specific use cases.

## Core Configuration Sections for L1 Extraction

The L1 memory extraction stage relies on two primary configuration objects that control both the extraction behavior and the scheduling mechanics.

### ExtractionConfig Settings

The **`ExtractionConfig`** section in [`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts) defines how raw L0 logs are processed into structured L1 memory entries. Key fields include:
- **`enabled`** – Boolean toggle to activate L1 extraction (default: `true`)
- **`enableDedup`** – Activates deduplication to avoid storing duplicate memories (default: `true`)
- **`maxMemoriesPerSession`** – Caps the number of memories extracted per conversation session (default: `20`)
- **`model`** – Specifies the LLM model identifier for extraction tasks (e.g., `"openai/gpt-4o-mini"`)
- **`promptMode`** – Determines prompt style, either `"chat"` or `"code"` (default: `"chat"`)

When `enableDedup` is active, the pipeline uses either dense vector embeddings or BM25 sparse vectors to identify duplicate memories before storage.

### PipelineTriggerConfig Orchestration

The **`PipelineTriggerConfig`** section controls when the L1 extraction stage executes within the broader L0→L1→L2→L3 pipeline:

- **`everyNConversations`** – Triggers L1 extraction after every N conversation rounds (default: `5`)
- **`l1IdleTimeoutSeconds`** – Forces L1 extraction after idle time expires (default: `30`)
- **`enableWarmup`** – Allows pipeline warmup scheduling on startup (default: `true`)
- **`sessionActiveWindowHours`** – Excludes stale sessions from L2 polling after inactivity threshold

L1 extraction runs immediately when either the conversation count threshold or the idle timeout condition is met.

## Pipeline Flow and Deduplication Strategy

The complete pipeline flow is orchestrated by the trigger configuration, with L1 serving as the critical transformation layer between raw logs and structured knowledge.

### L1 Extraction Trigger Mechanics

According to the source implementation in [`MemoryCore/src/utils/pipeline-manager.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/utils/pipeline-manager.ts), the L1 stage executes based on the following logic:

1. **Conversation-based triggering** – After accumulating `everyNConversations` rounds, the pipeline initiates L1 extraction
2. **Idle-based triggering** – If no new conversations arrive within `l1IdleTimeoutSeconds`, extraction runs automatically
3. **L2 cascading** – Once L1 completes, the system waits `l2DelayAfterL1Seconds` (default: `90`) before initiating L2 scene generation

Sessions idle longer than `sessionActiveWindowHours` are excluded from subsequent L2 polling regardless of L1 completion status.

### Deduplication and Embedding Configuration

The L1 deduplication step relies on the **`EmbeddingConfig`** section to determine similarity calculation methods:

- **Dense embedding** – When `EmbeddingConfig.provider` is set to `"openai"` or another provider, the pipeline uses vector similarity to detect duplicates
- **BM25 fallback** – When `EmbeddingConfig.provider` is `"none"` (default), the pipeline falls back to **`BM25Config`** for sparse vector scoring (default `language: "zh"`)

The **`BM25Config`** section controls the sparse encoder with fields for `enabled` status and language settings, ensuring deduplication works even without external embedding services.

## Configuration Examples

### Minimal Zero-Config Setup

An empty JSON configuration relies on sensible defaults for immediate L1 extraction functionality:

```json
{}

```

This configuration enables capture, L1 extraction with deduplication, automatic warmup scheduling, and BM25-only embedding (no external API required).

### Production Configuration with Dense Embeddings

For production deployments requiring high-quality deduplication and cloud-based vector storage:

```json
{
  "capture": {
    "enabled": true,
    "excludeAgents": ["bench-judge-*"],
    "l0l1RetentionDays": 30,
    "allowAggressiveCleanup": false
  },
  "extraction": {
    "enabled": true,
    "enableDedup": true,
    "maxMemoriesPerSession": 30,
    "model": "openai/gpt-4o-mini",
    "promptMode": "chat"
  },
  "pipelineTrigger": {
    "everyNConversations": 4,
    "enableWarmup": true,
    "l1IdleTimeoutSeconds": 20,
    "l2DelayAfterL1Seconds": 60,
    "l2MinIntervalSeconds": 600,
    "l2MaxIntervalSeconds": 1800,
    "sessionActiveWindowHours": 12
  },
  "embedding": {
    "enabled": true,
    "provider": "openai",
    "baseUrl": "https://api.openai.com/v1",
    "apiKey": "<YOUR_API_KEY>",
    "model": "text-embedding-3-large",
    "dimensions": 1536,
    "sendDimensions": true,
    "conflictRecallTopK": 7,
    "maxInputChars": 4000,
    "timeoutMs": 12000
  },
  "bm25": {
    "enabled": true,
    "language": "zh"
  }
}

```

This configuration enables aggressive L1 extraction (every 4 conversations) with OpenAI embeddings for semantic deduplication and hybrid recall.

## Implementation Architecture

The configuration system uses strict TypeScript definitions in [`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts) (lines 14-180) to ensure type safety across the pipeline.

### Configuration Parsing

The **`parseConfig`** utility in [`MemoryCore/src/utils/pipeline-manager.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/utils/pipeline-manager.ts) validates and transforms the raw JSON into runtime configuration objects. Concrete pipeline stages are instantiated by **[`MemoryCore/src/utils/pipeline-factory.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/utils/pipeline-factory.ts)**, which reads the config to create L1 extractors, L2 scene generators, and L3 persona synthesizers.

### Prompt Handling

The `promptMode` field in `ExtractionConfig` maps to specific prompt templates implemented in **[`MemoryCore/src/gateway/memory-prompt-handlers.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/gateway/memory-prompt-handlers.ts)**, determining whether the extraction LLM receives chat-formatted or code-formatted instructions when processing L0 logs into L1 memories.

## Summary

- The **L1 memory extraction pipeline** is controlled by `ExtractionConfig` and `PipelineTriggerConfig` in [`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts)
- **Trigger conditions** include conversation count (`everyNConversations`) and idle timeout (`l1IdleTimeoutSeconds`)
- **Deduplication** uses either dense embeddings (via `EmbeddingConfig`) or BM25 sparse vectors (via `BM25Config`) when embeddings are disabled
- The **standalone LLM configuration** allows dedicated model endpoints for memory extraction separate from the host runner
- Default settings enable immediate operation with BM25-only deduplication and conservative memory limits (20 memories per session)

## Frequently Asked Questions

### Where is the L1 memory extraction configuration defined?

The configuration is defined in **[`MemoryCore/src/config.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/config.ts)**, which exports TypeScript interfaces and default values for `ExtractionConfig`, `PipelineTriggerConfig`, and related sections. The runtime parsing logic resides in **[`MemoryCore/src/utils/pipeline-manager.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/src/utils/pipeline-manager.ts)**.

### How does the L1 extraction trigger mechanism work?

L1 extraction triggers on two conditions: after every `everyNConversations` conversation rounds (default: 5), or after `l1IdleTimeoutSeconds` seconds of inactivity (default: 30). The system evaluates these conditions continuously in the background pipeline manager.

### What is the difference between BM25 and dense embedding deduplication?

**BM25** uses sparse vector scoring based on term frequency and is controlled by `BM25Config` (enabled by default when `EmbeddingConfig.provider` is `"none"`). **Dense embedding** uses vector similarity through external providers like OpenAI and requires `EmbeddingConfig.enabled` set to `true` with valid API credentials. Dense embeddings generally provide better semantic matching but require external API access.

### How do I configure a separate LLM for memory extraction?

Use the **`standaloneLlm`** configuration section to specify a dedicated endpoint for L1 extraction tasks. When defined with `model`, `apiKey`, and `baseUrl` fields, the pipeline bypasses the host runner and uses this isolated LLM connection for all memory extraction operations, preventing interference with primary conversational AI tasks.