How to Configure LightRAG for High Availability with Distributed Storage Backends

LightRAG achieves high availability by externalizing all state to distributed storage backends (Redis Cluster, Qdrant, Neo4j, PostgreSQL, MongoDB) and running multiple stateless FastAPI replicas behind a load balancer.

LightRAG's modular architecture separates the RAG orchestration layer from persistent storage, enabling true LightRAG high availability distributed storage deployments without modifying application code. By implementing the abstract interfaces defined in lightrag/kg/__init__.py, each storage backend can be swapped for clustered, fault-tolerant alternatives. This guide leverages the HKUDS/LightRAG source code to configure horizontal scaling, automatic failover, and zero-downtime maintenance.

Architecture Overview for High Availability

LightRAG delegates persistence to four distinct storage abstractions. Each layer can be independently clustered to eliminate single points of failure:

Storage Layer	HA Strategy	Concrete Implementation
KV Store (metadata, locks)	Redis Cluster or replicated in-memory store	lightrag/kg/redis_impl.py
Vector Store (embeddings)	Sharded/replicated vector index	lightrag/kg/qdrant_impl.py, lightrag/kg/milvus_impl.py
Graph Store (entity relations)	Neo4j Causal Cluster or PostgreSQL with logical replication	lightrag/kg/neo4j_impl.py, lightrag/kg/postgres_impl.py
Document Status (processing state)	MongoDB replica set or PostgreSQL streaming replication	lightrag/kg/mongo_impl.py, lightrag/kg/postgres_impl.py

The FastAPI server in lightrag/api/lightrag_server.py is completely stateless. All request handling depends solely on the external storage connections established at startup, allowing you to scale API instances horizontally using Docker Compose replicas or Kubernetes Deployments.

Step 1: Deploy Distributed Storage Services

Provision production-grade clusters for each storage type. Below are minimal Docker Compose snippets for local testing; production environments should use managed cloud services or Kubernetes StatefulSets.

Redis Cluster (KV store):

services:
  redis:
    image: redis:7
    command: ["redis-server", "--cluster-enabled", "yes", "--cluster-config-file", "nodes.conf", "--appendonly", "yes"]
    ports: ["6379:6379"]

Qdrant Cluster (vector store):

services:
  qdrant:
    image: qdrant/qdrant:latest
    environment:
      - QDRANT__CLUSTER__ENABLED=true
    ports: ["6333:6333"]

Neo4j Causal Cluster (graph store):

services:
  neo4j:
    image: neo4j:5-enterprise
    environment:
      - NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
      - NEO4J_dbms_mode=CORE
    ports: ["7687:7687"]

PostgreSQL with Streaming Replication (graph and doc-status stores):

services:
  postgres:
    image: gzdaniel/postgres-for-rag:16.6
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=secret
    ports: ["5432:5432"]

MongoDB Replica Set (document status):

services:
  mongo:
    image: mongo:6
    command: ["mongod", "--replSet", "rs0"]
    ports: ["27017:27017"]

Step 2: Generate the HA-Aware Environment Configuration

Use the interactive storage wizard to generate an .env file pointing to your external clusters rather than local containers:

make env-storage

When prompted, select "no" for "Run storage services locally via Docker?" and provide the cluster endpoints. The wizard (implemented in scripts/setup/lib/storage_wizard.py) produces environment variables mapping to the concrete storage classes:

# KV Store (Redis Cluster)

REDIS_HOST=redis-cluster.example.com
REDIS_PORT=6379

# Vector Store (Qdrant Cluster)

QDRANT_HOST=qdrant-cluster.example.com
QDRANT_PORT=6333

# Graph Store (Neo4j Causal Cluster)

NEO4J_HOST=neo4j-core-0.example.com
NEO4J_BOLT_PORT=7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=very_secret

# Document Status (PostgreSQL Primary)

POSTGRES_HOST=postgres-primary.example.com
POSTGRES_PORT=5432
POSTGRES_USER=rag
POSTGRES_PASSWORD=secret
POSTGRES_DATABASE=rag

Step 3: Configure LightRAG for Horizontal Scaling

With state externalized, the LightRAG API layer can run as multiple stateless replicas. Modify docker-compose.final.yml or create an override file to specify replica count:

services:
  lightrag:
    image: ghcr.io/hkuds/lightrag:latest
    env_file:
      - .env
    deploy:
      mode: replicated
      replicas: 3
      restart_policy:
        condition: on-failure
    ports:
      - "9621:9621"

Apply the configuration:

docker compose -f docker-compose.final.yml up -d

Step 4: Verify High Availability

Test connectivity to all distributed backends:

from lightrag import LightRAG

rag = LightRAG()
print("All storage backends reachable")

Simulate node failure to validate failover. If using Redis Cluster, terminate one node:

docker stop redis-node-1

Confirm the API remains responsive:

curl -X POST http://localhost:9621/query \
  -d '{"query": "What is LightRAG?"}'

Successful responses indicate the system is operating in HA mode with automatic failover.

Advanced Configuration Patterns

Explicit Storage Initialization

For scenarios requiring custom connection parameters beyond environment variables, instantiate concrete storage classes directly from lightrag/kg/postgres_impl.py and lightrag/kg/qdrant_impl.py:

from lightrag import LightRAG
from lightrag.kg.postgres_impl import PostgresVectorDBStorage
from lightrag.kg.qdrant_impl import QdrantVectorDBStorage

vector_storage = QdrantVectorDBStorage(
    host="qdrant-cluster.example.com",
    port=6333,
    collection_name="rag_embeddings",
)

graph_storage = PostgresVectorDBStorage(
    host="postgres-primary.example.com",
    port=5432,
    user="rag",
    password="secret",
    database="rag",
)

rag = LightRAG(
    vector_storage=vector_storage,
    graph_storage=graph_storage,
)

Production Docker Compose Overrides

Create docker-compose.override.yml for environment-specific scaling:

version: "3.8"
services:
  lightrag:
    deploy:
      replicas: 5
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    environment:
      POSTGRES_HOST: postgres-primary.example.com
      QDRANT_HOST: qdrant-cluster.example.com

Deploy with:

docker compose -f docker-compose.final.yml -f docker-compose.override.yml up -d

Kubernetes Deployment Specification

For Kubernetes environments, use the same environment variables via Secrets:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: lightrag
spec:
  replicas: 4
  selector:
    matchLabels:
      app: lightrag
  template:
    metadata:
      labels:
        app: lightrag
    spec:
      containers:
      - name: lightrag
        image: ghcr.io/hkuds/lightrag:latest
        envFrom:
        - secretRef:
            name: lightrag-config
        ports:
        - containerPort: 9621

Key Implementation Files

Understanding these source files is essential for troubleshooting distributed deployments:

• lightrag/kg/__init__.py – Defines the abstract storage interfaces that enable backend swapping
• lightrag/kg/postgres_impl.py – PostgreSQL implementations for vector, graph, and document-status storage
• lightrag/kg/qdrant_impl.py – Qdrant vector store with cluster support
• lightrag/kg/milvus_impl.py – Milvus distributed vector database integration
• lightrag/kg/neo4j_impl.py – Neo4j graph store supporting Causal Clustering
• lightrag/kg/mongo_impl.py – MongoDB replica set integration for document status
• lightrag/api/lightrag_server.py – Stateless FastAPI entry point
• scripts/setup/lib/storage_wizard.py – Logic mapping environment variables to storage implementations
• docs/InteractiveSetup.md and docs/DockerDeployment.md – Official configuration guides

Summary

Configuring LightRAG for high availability requires these essential steps:

• Externalize state to distributed storage clusters (Redis, Qdrant, Neo4j, PostgreSQL, MongoDB) rather than local containers
• Use the storage wizard (make env-storage) to generate environment variables pointing to HA endpoints
• Scale the API layer horizontally using Docker Compose replicas or Kubernetes Deployments since lightrag/api/lightrag_server.py is stateless
• Verify failover by testing connectivity after simulating node failures
• Reference concrete implementations in lightrag/kg/*_impl.py when configuring custom connection parameters programmatically

Frequently Asked Questions

Can LightRAG run in high availability mode without Kubernetes?

Yes. LightRAG achieves high availability through storage abstraction, not container orchestration. You can deploy HA storage using Docker Swarm, managed cloud services (AWS RDS, Azure Cosmos DB, GCP Memorystore), or standalone VMs, then run multiple LightRAG containers via Docker Compose with deploy.replicas. The critical requirement is that all API instances connect to the same distributed storage endpoints, not that you use Kubernetes specifically.

What happens if the vector store cluster loses a node during a query?

LightRAG's storage implementations in lightrag/kg/qdrant_impl.py and lightrag/kg/milvus_impl.py utilize the underlying client's built-in retry and load-balancing logic. When configured with cluster endpoints, the Qdrant or Milvus client automatically routes requests to healthy nodes. Queries may experience slightly higher latency during rebalancing, but the request succeeds as long as the cluster maintains a quorum. The API server itself remains unaffected because it holds no connection state.

Is the FastAPI server in LightRAG completely stateless?

Yes. The FastAPI server defined in lightrag/api/lightrag_server.py does not maintain any in-memory session state or local caching between requests. All conversation history, vector embeddings, graph data, and processing locks reside in the external storage backends (Redis, PostgreSQL, Qdrant, Neo4j, MongoDB). This design allows you to terminate and restart API containers without data loss and to scale replicas up or down instantly without sticky sessions or graceful shutdown requirements.

How do I migrate from single-node storage to distributed storage without losing data?

Migrate by first provisioning your distributed cluster, then using the storage-specific export/import tools (e.g., pg_dump for PostgreSQL, mongodump for MongoDB, or Qdrant's snapshot API) to transfer data. Update the .env file generated by make env-storage to point to the new cluster endpoints, then restart LightRAG containers. Because LightRAG treats storage as pluggable implementations of the interfaces in lightrag/kg/__init__.py, no application code changes are required. Verify the migration by querying existing documents before decommissioning the single-node instance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}