JsonKVStorage vs PostgreSQL in LightRAG: Choosing Between File and Database Storage

JsonKVStorage persists data as local JSON files ideal for rapid prototyping, while PGKVStorage leverages PostgreSQL with ACID transactions and vector extensions for production-scale workloads.

LightRAG unifies cache, embedding, and graph storage behind the BaseKVStorage interface defined in lightrag/base.py. The framework provides two primary concrete implementations: JsonKVStorage for serverless local development and PGKVStorage for enterprise PostgreSQL deployments. Understanding the architectural trade-offs between these backends ensures you select the appropriate storage layer for your durability and concurrency requirements.

The BaseKVStorage Abstraction

All storage backends in LightRAG inherit from the abstract BaseKVStorage class in lightrag/base.py. This interface defines standard methods including upsert(), get(), and iter_keys(), allowing the orchestration logic in lightrag.lightrag.Lightrag to remain agnostic to the underlying persistence mechanism. When you instantiate Lightrag, the framework injects the configured storage class based on environment variables, requiring zero code changes when switching between file-based and database backends.

JsonKVStorage: File-Based Implementation

Located in lightrag/kg/json_kv_impl.py, JsonKVStorage implements the KV interface using plain JSON files stored in your workspace directory. Each write operation serializes data to disk using standard Python file I/O, while reads deserialize the JSON files back into memory objects.

This backend requires no external services—only a writable directory specified by the LIGHTRAG_WORKSPACE environment variable. However, it lacks transactional guarantees; concurrent writes to the same workspace can create race conditions since operations rely on simple file writes without locking mechanisms. Performance degrades significantly with millions of keys due to linear file scans during iteration.

JsonKVStorage supports only exact-key lookups via get(key) and basic iteration through iter_keys(). It cannot perform server-side filtering or vector similarity search, making it suitable exclusively for small-scale experiments, unit tests, and CI pipelines where database infrastructure is undesirable.

PGKVStorage: Production PostgreSQL Backend

The PGKVStorage implementation in lightrag/kg/postgres_impl.py uses psycopg2 or asyncpg drivers to execute SQL against a PostgreSQL instance. This backend provides full ACID compliance, row-level locking, and safe concurrent access for multi-user production deployments.

Unlike the file-based approach, PGKVStorage leverages PostgreSQL's query optimizer and indexing, enabling efficient upserts, bulk inserts, and complex filtering operations. When configured with the pgvector extension, the database can perform native vector similarity searches on embedding columns. Scalability extends through connection pooling, table partitioning, and standard database operations like backup and point-in-time recovery.

Configuration requires setting LIGHTRAG_KV_STORAGE=PGKVStorage and providing a connection URL via LIGHTRAG_PG_URL, plus ensuring the target database instance is running and accessible.

Key Architectural Differences

Feature	JsonKVStorage	PGKVStorage
Persistence Model	Local JSON files in workspace directory	PostgreSQL tables with schema
Transaction Support	None (direct file I/O)	Full ACID compliance
Concurrency Control	No protection against race conditions	Row-level locking, MVCC
Query Capabilities	Exact key lookup only	SQL queries, upserts, pgvector similarity search
Scalability	Limited by filesystem; degrades with millions of keys	Horizontal scaling via PostgreSQL replication and partitioning
Setup Requirements	Zero dependencies	Running PostgreSQL instance with connection URL
Performance Characteristics	Low latency for <10k keys; linear scan penalties at scale	Indexed queries; network latency offset by batch operations

Configuration and Usage Examples

Switching between storage backends requires only environment variable changes, as both implement the same BaseKVStorage contract.

To use JsonKVStorage for local development:

import os
from lightrag.lightrag import Lightrag

os.environ["LIGHTRAG_KV_STORAGE"] = "JsonKVStorage"
os.environ["LIGHTRAG_WORKSPACE"] = "./lightrag_workspace"

rag = Lightrag()
rag.add_documents([...])

To configure PostgreSQL for production:

import os
from lightrag.lightrag import Lightrag

os.environ["LIGHTRAG_KV_STORAGE"] = "PGKVStorage"
os.environ["LIGHTRAG_PG_URL"] = "postgresql://user:password@localhost:5432/lightrag"

rag = Lightrag()
rag.add_documents([...])

Both examples use the identical Lightrag API; the framework instantiates the appropriate storage class based on the LIGHTRAG_KV_STORAGE value, transparently routing all KV operations through either JSON files or SQL statements.

Summary

• JsonKVStorage in lightrag/kg/json_kv_impl.py provides zero-setup file persistence suitable for prototyping and testing, but lacks concurrency safety and scalability beyond small datasets.
• PGKVStorage in lightrag/kg/postgres_impl.py delivers production-grade durability, ACID transactions, and vector search capabilities through PostgreSQL with psycopg2 or asyncpg drivers.
• Both backends implement the BaseKVStorage interface from lightrag/base.py, enabling seamless backend switching via the LIGHTRAG_KV_STORAGE environment variable.
• Choose JsonKVStorage for local experiments without database infrastructure; select PGKVStorage for concurrent, large-scale deployments requiring data integrity and advanced query capabilities.

Frequently Asked Questions

Can I migrate data from JsonKVStorage to PostgreSQL in LightRAG?

LightRAG does not provide automated migration tools between storage backends. To migrate, export your JSON files from the workspace directory and write a custom script to upsert the data into PostgreSQL using the PGKVStorage.upsert() method. Ensure your PostgreSQL schema matches the expected table structures defined in lightrag/kg/postgres_impl.py.

Does JsonKVStorage support concurrent writes from multiple processes?

No. JsonKVStorage writes directly to the filesystem without file locking or transaction isolation. Concurrent writes from separate processes or threads can result in race conditions and data corruption. For multi-process deployments, use PGKVStorage, which leverages PostgreSQL's row-level locking and ACID compliance to ensure data integrity.

What PostgreSQL extensions does LightRAG require?

LightRAG optionally utilizes the pgvector extension for vector similarity search when storing embeddings in PostgreSQL. While basic KV operations work with standard PostgreSQL, enabling pgvector in your database unlocks optimized similarity queries for retrieval-augmented generation workflows. The extension must be installed and configured separately in your PostgreSQL instance.

How does performance compare between JsonKVStorage and PostgreSQL for small datasets?

JsonKVStorage exhibits lower latency for datasets under approximately 10,000 keys because it eliminates network round-trips and query planning overhead. However, as data grows beyond this threshold, PostgreSQL's B-tree indexing and connection pooling provide superior throughput, while JsonKVStorage suffers from linear scan penalties during iteration and lookups.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}