How LightRAG's LLM Caching (enable_llm_cache) Reduces API Costs

LightRAG's enable_llm_cache feature eliminates redundant calls to expensive LLM APIs by storing responses in a local key-value store keyed to deterministic hashes of request arguments, directly reducing token-based billing costs.

The enable_llm_cache mechanism in the HKUDS/LightRAG repository provides a deterministic deduplication layer that intercepts identical LLM requests before they reach paid external providers. By caching every response in a configurable KV store and serving subsequent matching queries from local storage, LightRAG turns repeated operations—such as entity extraction and query generation—into zero-cost cache retrievals.

How LLM Caching Works in LightRAG

Deterministic Cache Key Generation

In lightrag/utils.py (lines 561-570), the generate_cache_key function creates a unique identifier by hashing the request arguments and combining them with the operating mode and cache type:

flattened_key = f"{mode}:{cache_type}:{args_hash}"

This flattened key ensures that identical inputs produce identical cache keys, regardless of when they are called.

The Cache Lookup Flow

Before invoking the LLM, use_llm_func_with_cache calls handle_cache to check the KV store. When hashing_kv.global_config.get("enable_llm_cache") returns True, the system queries the storage backend (see lightrag/utils.py, lines 1391-1405):

cache_entry = await hashing_kv.get_by_id(flattened_key)

This lookup determines whether to proceed with an external API call or return cached data immediately.

Cache Hits and Misses

When a cache hit occurs, the stored response (cache_entry["return"]) is returned immediately without contacting the LLM provider, and the statistic_data["llm_cache"] counter increments. On a cache miss, the LLM function executes and save_to_cache stores the result for future reuse (see lightrag/utils.py, lines 1444-1467):

await hashing_kv.upsert({flattened_key: cache_entry})

This mechanism ensures that duplicate requests never incur duplicate charges.

Configuring LLM Caching

The feature is controlled through environment variables read in lightrag/api/config.py (lines 373-378). Set ENABLE_LLM_CACHE to enable global caching (default: True), or use ENABLE_LLM_CACHE_FOR_EXTRACT to toggle caching specifically for entity extraction workflows:

export ENABLE_LLM_CACHE=true
export ENABLE_LLM_CACHE_FOR_EXTRACT=false

These values map to the enable_llm_cache and enable_llm_cache_for_entity_extract fields in the LightRAG dataclass defined in lightrag/lightrag.py (lines 395-401).

Monitoring Cost Savings

LightRAG tracks cache performance through the statistic_data dictionary. Inspect the llm_cache counter to verify how many API calls were avoided:

from lightrag.utils import statistic_data

print(f"LLM calls: {statistic_data['llm_call']}")
print(f"Cache hits: {statistic_data['llm_cache']}")

Each increment in llm_cache represents a direct cost saving, as that request was served from local storage rather than a token-billed API.

Implementation Examples

Basic Configuration

Enable caching via environment variables before initializing your LightRAG instance:

export ENABLE_LLM_CACHE=true

Programmatic Usage

When building custom wrappers, use use_llm_func_with_cache to automatically respect cache settings:

from lightrag.lightrag import LightRAG
from lightrag.utils import use_llm_func_with_cache

rag = LightRAG(enable_llm_cache=True)

async def cached_llm_call(query: str) -> str:
    return await use_llm_func_with_cache(
        llm_response_cache=rag.llm_response_cache,
        llm_func=rag.llm_wrapper,
        args_hash=rag.hash_args(query),
        cache_type="query",
        cache_keys_collector=rag.cache_keys,
        arg_hash=query,
    )

Cache Migration

Migrate existing cached data between storage backends using the included CLI tool without losing cost-saving history:

python -m lightrag.tools.migrate_llm_cache \
    --src sqlite:///old_cache.db \
    --dst redis://localhost:6379/0

Summary

• LightRAG's LLM caching uses deterministic hashing in lightrag/utils.py to create unique cache keys from request arguments.
• The handle_cache function checks the KV store before every LLM call, skipping external API requests when cached data exists.
• Configuration is managed via ENABLE_LLM_CACHE and ENABLE_LLM_CACHE_FOR_EXTRACT environment variables, parsed in lightrag/api/config.py.
• The statistic_data["llm_cache"] counter provides measurable evidence of cost reduction by tracking avoided API calls.
• Cached entries can be migrated between backends using lightrag/tools/migrate_llm_cache.py to preserve optimization history.

Frequently Asked Questions

What is the default value of ENABLE_LLM_CACHE in LightRAG?

By default, ENABLE_LLM_CACHE is set to True in lightrag/api/config.py. This means LightRAG caches LLM responses automatically unless explicitly disabled, ensuring immediate cost savings for repeated queries without additional configuration.

How does LightRAG determine if two LLM requests are identical?

LightRAG generates a deterministic hash of the request arguments and combines it with the operating mode and cache type to create a flattened key in generate_cache_key. If two requests produce the same hash and metadata, they are considered identical and share the same cache entry.

Can I disable caching for specific operations like entity extraction only?

Yes. Set ENABLE_LLM_CACHE_FOR_EXTRACT=false while keeping ENABLE_LLM_CACHE=true to disable caching specifically for the entity extraction pipeline. This granular control is handled in the handle_cache function when mode == "default", allowing you to optimize costs for repeated queries while ensuring fresh extractions when needed.

Does the cache persist across application restarts?

Yes, because LightRAG uses pluggable KV storage backends (such as SQLite, Redis, or OpenSearch). The cached responses are stored in these persistent backends, so they survive application restarts and can be migrated between storage systems using the migrate_llm_cache.py utility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}