How to Configure Token Budget Controls in LightRAG: Managing max_entity_tokens, max_relation_tokens, and max_total_tokens

LightRAG caps LLM context usage through three token-budget settings—max_entity_tokens, max_relation_tokens, and max_total_tokens—which you can set via environment variables, REST API parameters, or the Python SDK to prevent prompt overflow.

The HKUDS/LightRAG repository implements a unified token control system that prunes retrieved knowledge-graph context before sending it to your LLM. Configuring these budgets correctly ensures your queries stay within model context limits while maximizing relevant information retrieval. These parameters act as safety valves during the retrieval pipeline, automatically trimming entities, relationships, and text chunks when the total token count approaches the ceiling.

Understanding LightRAG's Token Budget Parameters

LightRAG provides three distinct levers to control context window consumption. Each parameter targets a specific component of the retrieved knowledge graph and text chunks.

max_entity_tokens

The max_entity_tokens parameter limits the token count allocated to entity context (knowledge-graph nodes). When processing a query, LightRAG stops adding entity nodes once this budget is reached. The default value is 6000 tokens, defined as DEFAULT_MAX_ENTITY_TOKENS in the source.

max_relation_tokens

The max_relation_tokens parameter caps the tokens consumed by relationship context (edges between nodes). This ensures that verbose relationship descriptions do not dominate the context window. The default is 8000 tokens (DEFAULT_MAX_RELATION_TOKENS).

max_total_tokens

The max_total_tokens parameter sets the overall ceiling for the entire prompt, including entities, relations, retrieved text chunks, and the system prompt. If the combined content exceeds this limit, LightRAG trims the lowest-ranked chunks until the budget is satisfied. The default is 30000 tokens (DEFAULT_MAX_TOTAL_TOKENS).

Default Values and Source Locations

The default constants reside in lightrag/constants.py (lines 46‑52):

# Query and retrieval configuration defaults

DEFAULT_MAX_ENTITY_TOKENS = 6000          # ← default for max_entity_tokens

DEFAULT_MAX_RELATION_TOKENS = 8000       # ← default for max_relation_tokens

DEFAULT_MAX_TOTAL_TOKENS = 30000         # ← default for max_total_tokens

These values are ingested into the QueryParam dataclass in lightrag/base.py (lines 17‑30), which reads environment variables or falls back to the defaults:

max_entity_tokens: int = int(
    os.getenv("MAX_ENTITY_TOKENS", str(DEFAULT_MAX_ENTITY_TOKENS))
)
max_relation_tokens: int = int(
    os.getenv("MAX_RELATION_TOKENS", str(DEFAULT_MAX_RELATION_TOKENS))
)
max_total_tokens: int = int(
    os.getenv("MAX_TOTAL_TOKENS", str(DEFAULT_MAX_TOTAL_TOKENS))
)

Configuration Methods

You can configure token budgets at three levels: global environment defaults, per-request API overrides, or direct Python SDK instantiation.

Via Environment Variables (Global Defaults)

Set these variables in your .env file or export them before starting the LightRAG service:

MAX_ENTITY_TOKENS=5000
MAX_RELATION_TOKENS=7000
MAX_TOTAL_TOKENS=25000

When the service initializes, QueryParam automatically picks up these values according to the logic in lightrag/base.py.

Via API Request Payload

The /query and /query/stream endpoints accept these parameters in the JSON body. The QueryRequest model in lightrag/api/routers/query_routes.py (lines 55‑71) exposes these fields:

POST /query
{
  "query": "Explain the relationship between quantum computing and cryptography.",
  "max_entity_tokens": 4000,
  "max_relation_tokens": 6000,
  "max_total_tokens": 20000,
  "mode": "mix"
}

Values provided in the request payload override environment defaults for that specific query only.

Via Python SDK (Programmatic Usage)

When calling LightRAG directly from Python, construct a QueryParam instance explicitly:

from lightrag.base import QueryParam

params = QueryParam(
    query="How does reinforcement learning work?",
    max_entity_tokens=3000,
    max_relation_tokens=5000,
    max_total_tokens=18000,
    mode="local",
)

# Pass params to your LightRAG instance

response = rag_instance.query(params)

This approach provides fine-grained control when integrating LightRAG into custom applications.

How Token Budgets Affect Query Execution

During the retrieval pipeline, LightRAG applies these budgets sequentially:

1. Entity pruning – Stops adding knowledge-graph nodes once max_entity_tokens is reached.
2. Relation pruning – Caps relationship descriptions at max_relation_tokens.
3. Chunk trimming – After gathering text chunks, the system checks the combined token count (entities + relations + chunks + system prompt). If the total exceeds max_total_tokens, it removes the lowest-ranked chunks until the budget is satisfied.

This hierarchical enforcement ensures the final prompt sent to the LLM never exceeds your specified limits, preventing context window overflow errors from providers like OpenAI or Anthropic.

Summary

• Three control knobs: Use max_entity_tokens for nodes, max_relation_tokens for edges, and max_total_tokens for the complete prompt ceiling.
• Default values: 6000, 8000, and 30000 respectively, defined in lightrag/constants.py.
• Configuration hierarchy: Environment variables provide global defaults; API request payloads override them per query; Python SDK allows runtime programmatic control.
• Safety mechanism: The system automatically prunes excess context to stay within budgets, prioritizing higher-ranked information.

Frequently Asked Questions

What configuration method takes precedence in LightRAG?

Per-request parameters in the API payload or Python SDK always override environment variable defaults. If you pass max_total_tokens in a JSON request to /query, that value is used instead of the MAX_TOTAL_TOKENS environment variable for that specific call. If no value is provided in the request, LightRAG falls back to the environment variable or the hardcoded default.

Can I disable token limits by setting them to zero?

No. The API schema in lightrag/api/routers/query_routes.py enforces a minimum value of 1 (using ge=1 in Pydantic's Field validator). Attempting to set 0 or negative values will trigger a validation error. To effectively disable pruning, set the budgets to a very high number (e.g., 100000) that exceeds your LLM's context window.

How do these token budgets relate to the LLM's actual context window?

The max_total_tokens parameter should be set lower than your LLM's absolute context limit to leave room for the model's response generation. For example, if using a model with a 128k token limit, you might set max_total_tokens to 100000 to ensure input and output tokens combined do not exceed the threshold. LightRAG only controls the input context; you must account for output tokens separately.

What happens if the sum of default budgets exceeds max_total_tokens?

LightRAG applies the max_total_tokens cap as the absolute final check. If the default entity and relation budgets (6000 + 8000 = 14000) plus retrieved chunks exceed your custom max_total_tokens (e.g., 12000), the system will trim chunks first, then relations and entities if necessary, to meet the total budget. The total limit acts as the hard ceiling regardless of individual component settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}