How to Migrate Existing Vector Databases to LightRAG's Storage System: A Complete Guide

LightRAG automatically detects and migrates legacy collections from Qdrant, Milvus, or PostgreSQL to its workspace-isolated schema when you initialize a storage instance, validating dimensions and preserving data while rewriting point IDs and adding workspace metadata.

LightRAG abstracts vector-store operations behind a unified BaseVectorStorage interface. When you initialize a workspace, the framework checks for legacy collections and seamlessly migrates them to the new schema that supports workspace isolation and model-specific suffixes. This process ensures backward compatibility while enabling advanced features like multi-workspace support and embedding model isolation.

Understanding LightRAG's Migration Architecture

LightRAG implements migration logic directly within each storage backend class. The process follows a consistent pattern across Qdrant, Milvus, and PostgreSQL: detect legacy data, validate compatibility, copy to new schema, and verify integrity.

Legacy Collection Detection

Each storage backend includes specialized detection logic to identify pre-migration collections. In lightrag/kg/qdrant_impl.py, the _find_legacy_collection method searches for collections matching patterns like lightrag_vdb_{namespace} or {workspace}_{namespace}. Similarly, Milvus and PostgreSQL implementations in lightrag/kg/milvus_impl.py and lightrag/kg/postgres_impl.py check for existing tables or collections that lack the new workspace-aware naming conventions.

Workspace-Aware Schema Initialization

When a storage instance initializes, the __post_init__ method (visible in qdrant_impl.py lines ~20-~48) computes an effective_workspace from environment variables or constructor arguments. The system then constructs new collection names using the pattern lightrag_vdb_{namespace}_{model_suffix}, ensuring isolation between different embedding models and workspaces.

Automatic Migration Trigger

The LightRAG orchestrator in lightrag/lightrag.py calls check_and_migrate_data (lines ~788-~858) during initialization. This triggers storage-specific migration routines:

• Qdrant: setup_collection (lines ~150-~380) handles batch data copying
• Milvus: _migrate_collection_schema (lines ~690-~1030) manages temporary collection creation and atomic renaming
• PostgreSQL: Various _migrate_* helpers (lines ~585-~1385) execute ALTER statements

Migrating Qdrant Collections

Qdrant migrations use a batch-copy approach that preserves data integrity while transforming collection structure.

Automatic Migration Flow

When you initialize a LightRAG instance with Qdrant storage, the system automatically executes the migration:

from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,  # Returns vectors (e.g., 1536-dim)

    vector_db_storage_cls="QdrantVectorDBStorage",
    workspace="my_workspace"
)
await rag.initialize()  # Triggers migration if legacy collection detected

During initialize(), QdrantVectorDBStorage.setup_collection() performs the following:

1. Detects legacy collections using _find_legacy_collection
2. Creates a new collection with the workspace-aware name lightrag_vdb_chunks_{model_suffix}
3. Copies data batch-wise while adding workspace_id payload fields
4. Rewrites point IDs with workspace prefixes to ensure uniqueness
5. Verifies migrated record counts match the source

Manual Migration Control

For advanced use cases requiring explicit control, instantiate the storage class directly:

from lightrag.kg.qdrant_impl import QdrantVectorDBStorage
from qdrant_client import QdrantClient

client = QdrantClient(url="http://localhost:6334")
storage = QdrantVectorDBStorage(
    namespace="chunks",
    global_config={"embedding_batch_num": 128},
    embedding_func=my_embedding_function,
    workspace="my_workspace"
)

# Force migration logic explicitly

await storage._create_collection_if_not_exist()

Monitor logs for messages like:

• Qdrant: Found legacy collection 'chunks' with 1245 records to migrate.
• ✅ SUCCESS: All records migrated successfully!

Migrating Milvus Collections

Milvus requires a more complex migration strategy due to schema immutability constraints, utilizing temporary collections and iterator-based copying.

Iterator-Based Schema Migration

The migration in lightrag/kg/milvus_impl.py handles schema changes through:

1. Temporary Collection Creation: _create_schema_for_namespace builds a new collection with the correct schema
2. Data Iteration: Uses client.query_iterator (batch size 2000) to stream data from the legacy collection
3. Atomic Swapping: Renames the original collection to <name>_old and the temporary collection to the target name

from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,
    vector_db_storage_cls="MilvusVectorDBStorage",
    workspace="demo_ws"
)
await rag.initialize()  # Triggers _migrate_collection_schema if needed

Verifying Migration Status

Inspect the temporary collection before the atomic rename:

from lightrag.kg.milvus_impl import MilvusVectorDBStorage

storage = MilvusVectorDBStorage(
    namespace="chunks",
    global_config={"embedding_batch_num": 256},
    embedding_func=my_embedding_function,
    workspace="manual_ws"
)

# Execute migration steps manually

await storage._migrate_collection_schema()

# Verify temporary collection exists

client = storage._create_milvus_client()
print(client.has_collection("lightrag_vdb_chunks_temp"))  # True during migration

Migrating PostgreSQL Vector Tables

PostgreSQL migrations focus on schema evolution, adding columns and converting data formats without recreating tables.

Schema Migration Helpers

The PostgreSQL implementation in lightrag/kg/postgres_impl.py (lines ~585-~1385) includes several migration methods:

• _migrate_llm_cache_schema: Adds chunk_id, cache_type, and queryparam columns
• _migrate_field_lengths: Expands entity_name and source_id fields to accommodate larger identifiers
• _migrate_llm_cache_to_flattened_keys: Converts old LLM-cache keys to the new flattened format

from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,
    vector_db_storage_cls="PostgresVectorDBStorage",
    workspace="pg_ws"
)
await rag.initialize()  # Runs all _migrate_* helpers automatically

Running Specific Migrations

Target individual migrations for troubleshooting:

from lightrag.kg.postgres_impl import PostgreSQLDB

pg = PostgreSQLDB(config={
    "host": "localhost",
    "port": 5432,
    "user": "lightrag",
    "password": "secret",
    "database": "lightrag",
    "workspace": "manual_pg",
    "max_connections": 10,
})

# Execute specific migrations

await pg._migrate_llm_cache_schema()
await pg._migrate_field_lengths()

Configuration and Environment Variables

LightRAG uses environment variables to configure connection details and migration behavior. Place these in a .env file at the repository root:

Variable	Purpose	Example
QDRANT_URL / MILVUS_URI / POSTGRES_HOST	Connection endpoints	QDRANT_URL=http://localhost:6334
QDRANT_WORKSPACE / MILVUS_WORKSPACE / POSTGRES_WORKSPACE	Override default workspace names	QDRANT_WORKSPACE=production
MODEL_NAME	Determines the model suffix for collection names	text-embedding-ada-002
QDRANT_UPSERT_MAX_PAYLOAD_BYTES	Prevent payload size errors during batch migration	16777216
QDRANT_UPSERT_MAX_POINTS_PER_BATCH	Control batch sizes for large vectors	100

Troubleshooting Common Migration Issues

Dimension Mismatch Errors

If you encounter DataMigrationError: Dimension mismatch between legacy collection..., verify that your embedding function's output dimension matches the legacy collection's vector size. This check occurs in Qdrant at setup_collection (lines ~200-~225) and in Milvus at _check_vector_dimension (lines ~1010-~1060).

Large Payload Failures in Qdrant

Qdrant migrations may fail with payload size errors when migrating high-dimensional vectors or large metadata. Increase QDRANT_UPSERT_MAX_PAYLOAD_BYTES or reduce QDRANT_UPSERT_MAX_POINTS_PER_BATCH in your environment configuration.

Undetected Legacy Collections

If LightRAG creates empty new collections instead of migrating existing data, verify your legacy collection names match expected patterns (lightrag_vdb_{namespace} or {workspace}_{namespace}). Set the appropriate *_WORKSPACE environment variable to match the legacy collection's workspace context.

Manual Cleanup Requirements

After successful migration, LightRAG logs a warning: "Manual deletion is required after data migration verification." The legacy collection remains intact as a safety measure. Once you verify the new collection contains all records using client.count, delete the legacy collection manually via the database API or management UI.

Summary

• LightRAG automatically migrates existing Qdrant, Milvus, and PostgreSQL vector databases when initializing storage instances with matching workspace configurations.
• Migration preserves all data while adding workspace isolation metadata and model-specific naming conventions to prevent collisions.
• Dimension validation prevents corruption by raising DataMigrationError when embedding model dimensions differ from legacy collections.
• Environment variables control connection details, workspace names, and batch processing limits for large-scale migrations.
• Manual cleanup of legacy collections is required after verification, as LightRAG leaves source data intact for safety.

Frequently Asked Questions

What happens if my embedding model dimensions don't match the legacy collection?

LightRAG aborts the migration with a DataMigrationError to prevent data corruption. According to the source code in lightrag/kg/qdrant_impl.py (lines ~200-~225) and lightrag/kg/milvus_impl.py (lines ~1010-~1060), the system validates vector dimensions before copying data. You must either use an embedding function with matching dimensions or delete the legacy collection if you don't need to preserve that data.

Can I migrate data from multiple legacy workspaces into one LightRAG workspace?

No, LightRAG's migration logic maps one legacy workspace to one new workspace-aware collection. The effective_workspace calculation in __post_init__ methods ensures isolation. If you need to consolidate data from multiple legacy sources, run separate migration instances for each source workspace, then query across them using LightRAG's multi-storage capabilities.

Does the migration process lock the database or cause downtime?

The migration creates new collections or tables rather than modifying existing ones in place. For Qdrant, data is copied batch-wise while the legacy collection remains readable. Milvus uses a temporary collection strategy with atomic renaming at the end. PostgreSQL runs ALTER statements that may briefly lock tables but maintains availability throughout. Always back up critical data before migration.

How do I know when migration is complete and safe?

LightRAG logs explicit success messages like ✅ SUCCESS: All records migrated successfully! and reports the record count verification. For Qdrant, check that client.count matches between source and destination. For Milvus, verify the temporary collection no longer exists and the main collection contains expected records. PostgreSQL migrations log completion of each _migrate_* helper. After confirmation, manually remove legacy collections to reclaim storage space.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}