# How to Troubleshoot LightRAG Entity Extraction Issues When the Knowledge Graph Is Not Building

> Troubleshoot LightRAG entity extraction issues and knowledge graph errors. Learn to fix token-count guards, LLM output, and storage configurations for successful graph building.

- Repository: [✨Data Intelligence Lab@HKU✨/LightRAG](https://github.com/HKUDS/LightRAG)
- Tags: how-to-guide
- Published: 2026-03-23

---

**When LightRAG fails to populate the knowledge graph, the issue typically stems from the token-count guard blocking extraction gleaning, malformed LLM output that breaks the tuple parser, or unconfigured storage objects that skip persistence calls.**

LightRAG is a modular retrieval-augmented generation framework that constructs a knowledge graph by extracting entities and relationships from text chunks using LLM calls. When the graph remains empty after processing documents, the failure usually occurs silently during the `extract_entities` workflow in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py). This guide examines the exact source locations—from line 2813 where the chunk loop starts to the `_persist_graph_updates` utility—providing specific diagnostics to identify and resolve extraction failures.

## How LightRAG Extracts Entities to Build the Knowledge Graph

The extraction process follows a strict pipeline defined in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py). Understanding these steps is essential for locating where the process silently drops data.

- **Chunk iteration (line 2813):** The `extract_entities` function receives a dictionary of `TextChunkSchema` objects and iterates over each chunk.
- **Initial LLM extraction:** For each chunk, it calls `use_llm_func_with_cache` with the `entity_extraction_user_prompt`, then parses the result via `_process_extraction_result`.
- **Optional gleaning loop (lines 2816-2845):** If `entity_extract_max_gleaning` is greater than 0 and the token count is under `max_extract_input_tokens`, a second LLM request is issued using `entity_continue_extraction_user_prompt`. The token guard at lines 2830-2835 prevents gleaning when input exceeds the limit.
- **Result merging (lines 2850-2875):** Gleaned entities and relations replace originals only if they contain longer descriptions.
- **Persistence:** Finally, `_persist_graph_updates` in [`lightrag/utils_graph.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/utils_graph.py) writes vectors, graph edges, and chunk-tracking data to storage.

If any step returns empty or malformed data without raising an exception, the downstream graph will appear empty.

## Common Reasons the Knowledge Graph Stays Empty

Several specific misconfigurations cause the extraction pipeline to return empty results. Check these symptoms against your logs and configuration.

**Symptom: No LLM calls after the first request**

- **Cause:** `entity_extract_max_gleaning` is set to `0` in your global configuration.
- **Fix:** Verify `global_config["entity_extract_max_gleaning"] > 0` to enable multi-pass extraction.

**Symptom: "Gleaning stopped" warning appears**

- **Cause:** The token-count guard at lines 2833-2837 in [`operate.py`](https://github.com/HKUDS/LightRAG/blob/main/operate.py) halts processing because `token_count > max_extract_input_tokens`.
- **Fix:** Increase `max_extract_input_tokens` or reduce chunk size to stay within limits.

**Symptom: LLM returns malformed tuples**

- **Cause:** The extraction output does not match the expected delimiters. The parser expects tuples formatted as `(entity<|#|>NAME<|#|>TYPE<|#|>DESC)<|COMPLETE|>` using `PROMPTS["DEFAULT_TUPLE_DELIMITER"]` (`<|#|>`) and `DEFAULT_COMPLETION_DELIMITER` (`<|COMPLETE|>`).
- **Fix:** Inspect raw LLM output and verify delimiter alignment.

**Symptom: `_process_extraction_result` raises exceptions**

- **Cause:** Invalid JSON or missing fields in the LLM output cause the parser to fail.
- **Fix:** Wrap `extract_entities` in try/except blocks and run unit tests in [`tests/test_extract_entities.py`](https://github.com/HKUDS/LightRAG/blob/main/tests/test_extract_entities.py) to isolate malformed responses.

**Symptom: Graph storage not updated**

- **Cause:** `chunk_entity_relation_graph`, `entities_vdb`, or `relationships_vdb` are `None` or do not implement `BaseKGStorage` / `BaseVectorStorage` from [`lightrag/base.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/base.py).
- **Fix:** Confirm storage objects are instantiated before calling extraction.

**Symptom: Deadlock during processing**

- **Cause:** The `get_storage_keyed_lock` function receives wrong namespace or duplicate keys.
- **Fix:** Enable DEBUG logging for `lightrag.utils` and verify "acquired lock" messages appear consistently.

## Step-by-Step Troubleshooting Workflow

Run this checklist in a Python REPL or test script to isolate the failure point. Each step references specific source lines for verification.

### Inspect Global Configuration

Start by verifying your token limits and gleaning settings.

```python
from lightrag.utils import Tokenizer

class DummyTokenizer:
    def encode(self, txt): return [ord(c) for c in txt]
    def decode(self, toks): return "".join(chr(t) for t in toks)

cfg = {
    "llm_model_func": AsyncMock(return_value="…"),
    "entity_extract_max_gleaning": 1,
    "max_extract_input_tokens": 10,   # tiny to trigger guard

    "tokenizer": Tokenizer("dummy", DummyTokenizer()),
    "addon_params": {"language": "en"},
}
print(cfg["entity_extract_max_gleaning"], cfg["max_extract_input_tokens"])

```

*Reference:* See [`tests/test_extract_entities.py`](https://github.com/HKUDS/LightRAG/blob/main/tests/test_extract_entities.py) in the `_make_global_config` helper (lines 20-33).

### Run Extraction with Limited Tokens

Force the token guard to fire and observe the warning.

```python
chunks = {
    "chunk-001": {
        "content": "Some long text that certainly pushes the token count beyond ten tokens…",
        "tokens": 200,
        "full_doc_id": "doc-001",
        "chunk_order_index": 0,
    }
}

await extract_entities(
    chunks=chunks,
    global_config=cfg,
)

```

Expect a single LLM call and a warning logged at **operate.py** lines 2833-2836.

### Check for the Warning Message

The warning contains the phrase **"Gleaning stopped … exceeded limit"**. If this message is absent, the guard is either bypassed due to configuration errors or your logger level is muted.

### Increase Token Limits and Verify Gleaning

Raise the token budget to allow the second LLM call.

```python
cfg["max_extract_input_tokens"] = 999_999
await extract_entities(chunks=chunks, global_config=cfg)

```

The test `test_gleaning_proceeds_when_tokens_within_limit` expects `await_count == 2` on the LLM mock.

### Validate LLM Output Format

Manually test the parser with correctly formatted tuples.

```python
result = "(entity<|#|>TEST<|#|>CONCEPT<|#|>A test entity)<|COMPLETE|>"
nodes, edges = await _process_extraction_result(
    result, "chunk-001", time.time(), "demo.py",
    tuple_delimiter="<|#|>", completion_delimiter="<|COMPLETE|>"
)
print(nodes)   # should contain a dict entry for "TEST"

```

The parser logic resides in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py) around line 2910.

### Confirm Persistence

Verify that storage callbacks are invoked after extraction.

```python
class DummyStorage:
    async def index_done_callback(self): 
        print("persisted")

await extract_entities(
    chunks=chunks,
    global_config=cfg,
    chunk_entity_relation_graph=DummyStorage(),
    entities_vdb=DummyStorage(),
    relationships_vdb=DummyStorage(),
)

```

Check that `index_done_callback` fires for each storage instance (see [`utils_graph.py`](https://github.com/HKUDS/LightRAG/blob/main/utils_graph.py) lines 23-64).

### Run Unit Tests

Execute the built-in test suite to verify core logic.

```bash
python -m pytest tests/test_extract_entities.py

```

All three tests must pass; failures indicate regressions in the token-guard logic.

## Practical Code Examples for Debugging

These scripts demonstrate how to reproduce specific failure modes and verify fixes.

### Reproducing the Token Guard Warning

This minimal script triggers the "Gleaning stopped" warning by setting a restrictive token limit.

```python
import asyncio
from unittest.mock import AsyncMock
from lightrag.utils import Tokenizer
from lightrag.operate import extract_entities

class DummyTokenizer:
    def encode(self, txt): return [ord(c) for c in txt]
    def decode(self, toks): return "".join(chr(t) for t in toks)

global_cfg = {
    "llm_model_func": AsyncMock(
        return_value="(entity<|#|>E1<|#|>TYPE<|#|>Desc)<|COMPLETE|>"
    ),
    "entity_extract_max_gleaning": 1,
    "max_extract_input_tokens": 10,  # Exceeded by chunk text

    "tokenizer": Tokenizer("dummy", DummyTokenizer()),
    "addon_params": {"language": "en"},
}

chunks = {
    "chunk-001": {
        "content": "Some long text that certainly pushes the token count beyond ten tokens…",
        "tokens": 200,
        "full_doc_id": "doc-001",
        "chunk_order_index": 0,
    }
}

async def main():
    await extract_entities(chunks=chunks, global_config=global_cfg)

asyncio.run(main())

```

Running this produces the log warning:

```

WARNING:lightrag.operate:Gleaning stopped for chunk chunk-001: Input tokens (123) exceeded limit (10).

```

### Disabling the Token Guard

When you know your model can handle the context size, effectively disable the guard by setting a high limit.

```python
global_cfg["max_extract_input_tokens"] = 1_000_000
global_cfg["entity_extract_max_gleaning"] = 1

```

This configuration triggers **two** LLM calls (initial extraction plus gleaning). Verify this behavior with:

```python
assert global_cfg["llm_model_func"].await_count == 2

```

### Verifying Graph Persistence

Ensure that extraction results actually reach your storage backend.

```python
class DummyStorage:
    async def index_done_callback(self):
        print("Graph data persisted")

dummy_graph = DummyStorage()
dummy_entities = DummyStorage()
dummy_relations = DummyStorage()

await extract_entities(
    chunks=chunks,
    global_config=global_cfg,
    chunk_entity_relation_graph=dummy_graph,
    entities_vdb=dummy_entities,
    relationships_vdb=dummy_relations,
)

```

Expected output: three "Graph data persisted" messages, confirming that `_persist_graph_updates` invoked each storage's callback.

## Key Source Files to Inspect

Understanding these files helps you trace data flow during debugging:

- **[`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py)** (line 2813): Contains `extract_entities`, the token-guard logic, and the merging algorithm.
- **[`lightrag/utils_graph.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/utils_graph.py)** (line 23): Implements `_persist_graph_updates`, which handles writing to vector and graph stores.
- **[`tests/test_extract_entities.py`](https://github.com/HKUDS/LightRAG/blob/main/tests/test_extract_entities.py)**: Unit tests covering token-guard behavior and gleaning logic.
- **[`lightrag/utils.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/utils.py)**: Defines the `Tokenizer` class used for the token-count estimates that trigger the guard.
- **[`lightrag/base.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/base.py)**: Abstract base classes `BaseKGStorage` and `BaseVectorStorage` that your storage objects must implement.

## Summary

- The **token-count guard** (lines 2830-2835) silently skips gleaning when chunks exceed `max_extract_input_tokens`, leaving extractions incomplete.
- **Malformed LLM output** that deviates from the expected `(entity<|#|>...)<|COMPLETE|>` format causes `_process_extraction_result` to return empty results.
- **Storage misconfiguration**—passing `None` or non-compliant objects to `extract_entities`—prevents `_persist_graph_updates` from persisting data.
- Enabling `entity_extract_max_gleaning` and verifying token limits are the first steps to ensuring complete entity extraction.
- Always validate extraction by running `pytest tests/test_extract_entities.py` and inspecting logs for the "Gleaning stopped" warning.

## Frequently Asked Questions

### Why does LightRAG extract entities but fail to build the knowledge graph?

LightRAG may successfully call the LLM but fail to populate the graph if the token-count guard halts gleaning or if the output parser encounters malformed tuples. Additionally, if `chunk_entity_relation_graph`, `entities_vdb`, or `relationships_vdb` are not properly initialized as `BaseKGStorage` or `BaseVectorStorage` instances, the `_persist_graph_updates` function cannot save the results, leaving the graph empty despite successful extraction.

### How do I fix the "Gleaning stopped" warning in LightRAG?

Increase the `max_extract_input_tokens` value in your global configuration to accommodate your chunk sizes, or reduce the chunk size to stay below the token limit. The warning at lines 2833-2837 in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py) triggers when `token_count > max_extract_input_tokens`, preventing the second-pass extraction that captures additional entities.

### What format does LightRAG expect from the LLM for entity extraction?

LightRAG expects tuples delimited by the string defined in `PROMPTS["DEFAULT_TUPLE_DELIMITER"]` (default `<|#|>`) and terminated by `DEFAULT_COMPLETION_DELIMITER` (default `<|COMPLETE|>`). A valid response looks like `(entity<|#|>EntityName<|#|>EntityType<|#|>Description)<|COMPLETE|>`. If your LLM returns JSON or uses different separators, `_process_extraction_result` will fail to parse the entities.

### How can I test if my LightRAG storage configuration is correct?

Create mock storage classes that implement `index_done_callback` and pass them to `extract_entities`. If the callback fires after extraction, your configuration is wired correctly. Also verify that your storage objects inherit from `BaseKGStorage` for graph data and `BaseVectorStorage` for vector data as defined in [`lightrag/base.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/base.py).