# How to Set Up Wiki Ingestion with LLM-Powered Document Processing in TencentDB-Agent-Memory

> Set up wiki ingestion with LLM document processing in TencentDB Agent Memory. Register wiki sources, trigger ingest, and configure LLM_MODE=proxy for summarization via Knowledge Service.

- Repository: [Tencent Cloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
- Tags: how-to-guide
- Published: 2026-08-26

---

**Setting up wiki ingestion requires registering the wiki source via `MetadataClient`, triggering the `/wiki/ingest` endpoint, and configuring `LLM_MODE=proxy` to enable LLM-powered summarization through the Knowledge Service on port `:8421`.**

TencentDB-Agent-Memory provides a comprehensive knowledge service that ingests wiki pages, generates LLM-powered summaries, and exposes them to agents as searchable tools. This guide walks through the complete pipeline—from registering a wiki source to injecting processed summaries into agent prompts—using the official Python SDK and TypeScript APIs. The implementation relies on three distinct layers: the management-plane for metadata, the data-plane for LLM processing, and the injection-pipeline for agent integration.

## Architecture Overview

The wiki ingestion pipeline operates across three specialized layers that handle distinct responsibilities:

| Layer | Responsibility | Key Component |
|-------|----------------|---------------|
| **Management-plane** | CRUD operations for knowledge metadata | `MetadataClient` ([`sdk/memory-core/python/tencentdb_agent_memory/v3/metadata_client.py`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/sdk/memory-core/python/tencentdb_agent_memory/v3/metadata_client.py)) |
| **Data-plane** | Page retrieval, LLM summarization, and storage | Knowledge Service (`:8421`) |
| **Injection-pipeline** | Exposure of summaries to agents | `KnowledgeToolsInjector` ([`MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts)) |

When a wiki source is registered, the Knowledge Service stores raw pages under `/data/knowledge` and processes them through an LLM configured via the context proxy. The generated summaries populate a SQLite database within the `tdai-panel-data` volume, which agents later query for context-aware responses.

## Prerequisites and Configuration

Before triggering ingestion, verify that your deployment includes the following required settings:

- **`LLM_MODE=proxy`** (line 82 in [`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh)): Enables the LLM-powered ingest path through the context proxy.
- **`KNOWLEDGE_DATA_DIR=/data/knowledge`** (defined in [`deploy/panel-knowledge-combined/README.md`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/README.md)): Specifies the directory for raw wiki files, SQLite databases, and processing logs.
- **`tdai-panel-data` volume**: Mounted into the Knowledge Service container to persist wiki data across restarts (documented in [`deploy/global-images/README.md`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/global-images/README.md)).
- **`llm_wiki.enabled` flag** in [`MemoryProxy/config.example.yaml`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/config.example.yaml): Controls whether wiki assets are exposed to agents through the injection system.

## Step-by-Step Implementation

### Register the Wiki Source

Use the Python SDK's `MetadataClient` to register your wiki with the knowledge service. This creates the metadata record that the ingestion pipeline references.

```python
from tencentdb_agent_memory.v3 import MetadataClient

meta = MetadataClient(base_url="http://memory:8080/v3")
meta.create_knowledge(
    {
        "knowledge_id": "wiki-docs",
        "type": "wiki",
        "service_url": "http://ks:8421/v3",
        "name": "Project Wiki",
        "team_id": "team-1",
    }
)

```

This call targets the `create_knowledge` method implemented in [`sdk/memory-core/python/tencentdb_agent_memory/v3/metadata_client.py`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/sdk/memory-core/python/tencentdb_agent_memory/v3/metadata_client.py), which validates the payload and registers the wiki in the system metadata store.

### Trigger Ingestion

Initiate the LLM processing by calling the wiki ingest endpoint. The front-end wrapper in [`MemoryPanel/web/src/lib/api/knowledge-api.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryPanel/web/src/lib/api/knowledge-api.ts) provides a clean abstraction:

```typescript
import { knowledgeApi } from '@/lib/api/knowledge-api';

async function startWikiIngest(wikiId: string) {
  await knowledgeApi.ingest(wikiId);
}

```

The underlying implementation posts to `/wiki/ingest` with the wiki identifier (lines 318-322 in [`knowledge-api.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/knowledge-api.ts)). The Knowledge Service then queues the pages for background processing.

### Monitor Ingestion Status

Poll the status endpoint until `internal_status` transitions from `ingesting` to `ready`. The SDK provides a helper method that implements this polling logic:

```typescript
async function ingestWithPolling(wikiId: string) {
  await knowledgeApi.ingest(wikiId);
  while (true) {
    const status = await knowledgeApi.get(wikiId);
    if (status.internal_status !== 'ingesting') break;
    await new Promise(r => setTimeout(r, 2000));
  }
}

```

This pattern appears in [`knowledge-api.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/knowledge-api.ts) (lines 389-425), where the `ingestWithPolling` method orchestrates the status check loop with a two-second interval.

### Inject Summaries into Agent Context

Once the status reaches `ready`, the `KnowledgeToolsInjector` ([`MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts)) discovers assets of type `llm_wiki` and injects their summaries into the system prompt. The injector assembles a **knowledge** block containing the title and summary for each processed page:

```json
{
  "knowledge": [
    {
      "type": "wiki",
      "id": "wiki-docs",
      "title": "Design Overview",
      "summary": "This wiki page describes the overall design of the system..."
    }
  ]
}

```

The discovery flow (lines 3-15) identifies eligible wiki assets, while the summary handling (lines 93-101) populates the `about` field used in the prompt injection.

## Code Reference Guide

### Complete Ingestion Workflow (Python)

```python
from tencentdb_agent_memory.v3 import MetadataClient
import time

meta = MetadataClient(base_url="http://memory:8080/v3")

# 1. Register

meta.create_knowledge({
    "knowledge_id": "wiki-docs",
    "type": "wiki",
    "service_url": "http://ks:8421/v3",
    "name": "Project Wiki",
    "team_id": "team-1"
})

# 2. Trigger via HTTP or SDK methods

# 3. Poll until ready

```

### Agent Consumption Pattern

Agents access processed wiki content through two primary methods exposed by the Knowledge Service: `search` for semantic retrieval across summaries, and `read_page` for direct access to specific documents. The LLM-summarized content serves as the single source of truth for agent decision-making, keeping the agent loop lightweight by avoiding full document retrieval during inference.

## Summary

- **Three-layer architecture**: Management-plane (`MetadataClient`), data-plane (Knowledge Service on `:8421`), and injection-pipeline (`KnowledgeToolsInjector`) work sequentially to process wiki content.
- **Configuration requirements**: Set `LLM_MODE=proxy` in the startup script and ensure `KNOWLEDGE_DATA_DIR=/data/knowledge` exists with the `tdai-panel-data` volume mounted.
- **Implementation flow**: Register via Python SDK ([`metadata_client.py`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/metadata_client.py)), trigger via TypeScript API ([`knowledge-api.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/knowledge-api.ts)), monitor status through polling, and inject into prompts via the knowledge tools injector.
- **Storage**: Processed summaries persist in SQLite within the mounted volume, with raw pages stored under `/data/knowledge`.

## Frequently Asked Questions

### What LLM configuration is required for wiki ingestion?

Set `LLM_MODE=proxy` in [`deploy/panel-knowledge-combined/start-combined.sh`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/deploy/panel-knowledge-combined/start-combined.sh) (line 82) to route wiki content through the context proxy for summarization. This enables the LLM-powered processing pipeline that generates the `about` summaries and title-level metadata stored in the knowledge database.

### How do I monitor the status of a wiki ingestion job?

Poll the `/wiki/get` endpoint using the `ingestWithPolling` method in [`MemoryPanel/web/src/lib/api/knowledge-api.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryPanel/web/src/lib/api/knowledge-api.ts) (lines 389-425). The endpoint returns an `internal_status` field that transitions from `ingesting` to `ready` once the LLM has processed all pages and stored the summaries in the SQLite database.

### Where are the processed wiki summaries stored?

The Knowledge Service stores raw wiki files and generated summaries in `/data/knowledge` (configured via `KNOWLEDGE_DATA_DIR`), which maps to the `tdai-panel-data` Docker volume. The LLM-generated summaries specifically reside in a SQLite database within this directory, ensuring persistence across container restarts.

### How do agents access the ingested wiki content?

The `KnowledgeToolsInjector` ([`MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts`](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/main/MemoryProxy/src/injection/injectors/knowledge-tools-injector.ts)) automatically discovers wiki assets with `llm_wiki.enabled` and injects their summaries into the agent's system prompt under the **knowledge** block. Agents can then call `search` or `read_page` capabilities against these summaries to retrieve context without processing raw documents.