# How the TDAI Core Pipeline Is Initialized and Configured: A Deep Dive into TencentDB-Agent-Memory

> Discover how the TDAI Core pipeline initializes and configures its multi-stage memory pipeline. Learn about directory setup, store creation, and LLM runner wiring.

- Repository: [Tencent Cloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
- Tags: deep-dive
- Published: 2026-08-22

---

**The TDAI Core initializes its multi-stage memory pipeline (L0 → L1 → L2 → L3) through a sequential four-step process in `initialize()`: directory setup, store creation, pipeline manager construction, and LLM runner wiring, with all timing thresholds defined in `MemoryTdaiConfig`.**

The TencentDB-Agent-Memory repository implements a sophisticated memory extraction system for database AI agents. Understanding **TDAI Core pipeline initialization and configuration** is essential for developers who need to customize memory extraction behavior or debug pipeline execution flows. The core constructs its pipeline through a carefully orchestrated sequence of asynchronous operations during the bootstrap phase.

## Core Construction and Dependency Injection

The pipeline lifecycle begins when the `TdaiCore` class is instantiated. The constructor in [`src/core/tdai-core.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/core/tdai-core.ts) (lines 24-34) accepts a `TdaiCoreOptions` object and stores references to all external dependencies required for pipeline operation.

```ts
constructor(opts: TdaiCoreOptions) {
  this.hostAdapter = opts.hostAdapter;
  this.cfg = opts.config;
  this.logger = opts.hostAdapter.getLogger();
  this.dataDir = opts.hostAdapter.getRuntimeContext().dataDir;
  // ... additional dependency storage
}

```

The constructor captures the **host adapter** for runtime context, the parsed **configuration object**, the **logger**, and the **data directory** path. It also stores optional references to a **session filter**, **storage adapter**, and **skill hooks** that will be injected into the pipeline later. At this stage, no I/O operations occur—the constructor merely prepares the internal state for the subsequent initialization sequence.

## The Initialization Sequence

The `initialize()` method in [`src/core/tdai-core.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/core/tdai-core.ts) orchestrates the actual construction of the memory pipeline through three sequential actions. This method must be awaited before the core can process any conversation data.

### Data Directory Setup

First, `initDataDirectories(this.dataDir)` creates the filesystem layout required for checkpoint persistence and COS (Cloud Object Storage) file operations (line 44). This ensures that the directory structure exists before any persistence operations attempt to write state to disk.

### Store Initialization

Next, `initStores()` asynchronously creates the **vector store** and optional **embedding service** based on the configuration (lines 48-53). These storage backends serve as the foundation for L0 (raw message) buffering and semantic retrieval operations during later pipeline stages.

### Pipeline Manager Construction

If memory extraction is enabled via `cfg.extraction.enabled`, the core constructs a `MemoryPipelineManager` through the factory function `createPipelineManager`:

```ts
if (this.cfg.extraction.enabled) {
  this.scheduler = createPipelineManager(this.cfg, this.logger, this.sessionFilter);
  // ...
}

```

Located in [`src/utils/pipeline-factory.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/pipeline-factory.ts), this factory assembles the pipeline manager with configuration thresholds, timers, and warm-up mode settings derived from `MemoryTdaiConfig`. The factory also prepares the L1, L2, and L3 runner slots and a persister that will handle checkpoint serialization.

## Wiring the LLM Runners

After store initialization completes, `wirePipelineRunners()` determines whether to use the host-provided LLM runner or a **stand-alone LLM runner** based on `cfg.llm.enabled` or a `provider=proxy` setting (lines 69-75). This method creates and registers four critical components:

- **L1 runner** (`createL1Runner`) – Ingests buffered L0 messages and performs initial extraction batches.
- **L2 runner** (`createL2Runner`) – Extracts scene information from processed conversations.
- **L3 runner** (`createL3Runner`) – Generates personas based on accumulated context.
- **Persister** (`createPersister`) – Stores checkpoint data after each pipeline state change.

The wiring operation executes once during standard initialization, though it can be re-run after storage adapter injection if dynamic reconfiguration is required. All wiring operations are logged for debugging purposes.

## Scheduler Activation and Checkpoint Recovery

The pipeline remains dormant until the first conversation turn is committed. When `handleTurnCommitted()` is invoked, `ensureSchedulerStarted()` performs lazy initialization:

1. The method reads existing checkpoint data via `CheckpointManager` from [`src/utils/checkpoint.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/checkpoint.ts).
2. It calls `scheduler.start()` to restore pending session states and re-arm timers.
3. Subsequent calls reuse the same in-flight promise, guaranteeing a single start sequence even under concurrency.

This lazy-start pattern ensures that pipeline resources are not consumed until actual memory extraction work is required, while still maintaining idempotency for the bootstrap process.

## Configuration Schema and Timing Parameters

All pipeline behavior is governed by `MemoryTdaiConfig`, populated from [`MemoryCore/tdai-gateway.yaml`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/tdai-gateway.yaml). The configuration defines extraction enablement, model providers, and precise timing controls:

- **`everyNConversations`** – Triggers immediate L1 batch processing after N conversations.
- **`l1.idleTimeoutSeconds`** – Idle debounce duration before L1 runs on buffered messages.
- **`l2.delayAfterL1Seconds`** – Delay between L1 completion and L2 scene extraction.
- **`l2.minIntervalSeconds`** / **`l2.maxIntervalSeconds`** – bounds L2 scheduling frequency.
- **`enableWarmup`** – Activates exponential warm-up thresholds for new sessions.

The stand-alone LLM runner in [`src/adapters/standalone/llm-runner.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/adapters/standalone/llm-runner.ts) uses these timing parameters when the core operates in proxy mode, decoupled from the host's LLM infrastructure.

## Summary

- **Four-stage bootstrap**: The TDAI Core initializes through directory setup, store creation, pipeline manager construction, and LLM runner wiring.
- **Factory pattern**: [`src/utils/pipeline-factory.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/pipeline-factory.ts) decouples pipeline assembly from configuration parsing.
- **Lazy activation**: The scheduler starts only when the first turn is committed, restoring state from [`src/utils/checkpoint.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/checkpoint.ts).
- **YAML-driven**: All timing thresholds and extraction settings originate from [`MemoryCore/tdai-gateway.yaml`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/tdai-gateway.yaml) and populate `MemoryTdaiConfig`.
- **Modular runners**: L1, L2, and L3 runners are injected via `wirePipelineRunners()`, supporting both host-provided and stand-alone LLM adapters.

## Frequently Asked Questions

### What triggers the TDAI Core pipeline to start processing?

The pipeline scheduler starts lazily when `handleTurnCommitted()` is first called, invoking `ensureSchedulerStarted()`. This method reads the checkpoint file and calls `scheduler.start()` only once, even if multiple requests arrive simultaneously, by caching the start promise.

### How does the pipeline handle restarts and state persistence?

The `CheckpointManager` class in [`src/utils/checkpoint.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/checkpoint.ts) serializes per-session pipeline states to disk after each significant operation. When the core restarts, `ensureSchedulerStarted()` reads this checkpoint data and restores pending timers and warm-up thresholds, ensuring no conversation context is lost between process lifecycles.

### Can the LLM runner be replaced with a custom implementation?

Yes. The `wirePipelineRunners()` method checks `cfg.llm.enabled` and `provider=proxy` to determine whether to use the host adapter's runner or instantiate the stand-alone runner from [`src/adapters/standalone/llm-runner.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/adapters/standalone/llm-runner.ts). Developers can inject custom runners by modifying the factory logic in [`src/utils/pipeline-factory.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/pipeline-factory.ts) or by providing a compatible host adapter interface.

### Where are the pipeline timing thresholds configured?

All timing parameters—including `idleTimeoutSeconds`, `delayAfterL1Seconds`, and `everyNConversations`—are defined in [`MemoryCore/tdai-gateway.yaml`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryCore/tdai-gateway.yaml). These values populate the `MemoryTdaiConfig` interface, which is passed through the constructor to `createPipelineManager` and ultimately controls the timer logic in [`src/utils/pipeline-manager.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/src/utils/pipeline-manager.ts).