# How to Migrate Existing Vector Databases to LightRAG's Storage System: A Complete Guide

> Easily migrate your vector databases from Qdrant Milvus or PostgreSQL to LightRAGs storage system. Our guide ensures seamless data validation dimension checks and preserves your valuable data through automatic migration.

- Repository: [✨Data Intelligence Lab@HKU✨/LightRAG](https://github.com/HKUDS/LightRAG)
- Tags: migration-guide
- Published: 2026-03-23

---

**LightRAG automatically detects and migrates legacy collections from Qdrant, Milvus, or PostgreSQL to its workspace-isolated schema when you initialize a storage instance, validating dimensions and preserving data while rewriting point IDs and adding workspace metadata.**

LightRAG abstracts vector-store operations behind a unified `BaseVectorStorage` interface. When you initialize a workspace, the framework checks for legacy collections and seamlessly migrates them to the new schema that supports workspace isolation and model-specific suffixes. This process ensures backward compatibility while enabling advanced features like multi-workspace support and embedding model isolation.

## Understanding LightRAG's Migration Architecture

LightRAG implements migration logic directly within each storage backend class. The process follows a consistent pattern across Qdrant, Milvus, and PostgreSQL: detect legacy data, validate compatibility, copy to new schema, and verify integrity.

### Legacy Collection Detection

Each storage backend includes specialized detection logic to identify pre-migration collections. In [`lightrag/kg/qdrant_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/qdrant_impl.py), the `_find_legacy_collection` method searches for collections matching patterns like `lightrag_vdb_{namespace}` or `{workspace}_{namespace}`. Similarly, Milvus and PostgreSQL implementations in [`lightrag/kg/milvus_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/milvus_impl.py) and [`lightrag/kg/postgres_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/postgres_impl.py) check for existing tables or collections that lack the new workspace-aware naming conventions.

### Workspace-Aware Schema Initialization

When a storage instance initializes, the `__post_init__` method (visible in [`qdrant_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/qdrant_impl.py) lines ~20-~48) computes an `effective_workspace` from environment variables or constructor arguments. The system then constructs new collection names using the pattern `lightrag_vdb_{namespace}_{model_suffix}`, ensuring isolation between different embedding models and workspaces.

### Automatic Migration Trigger

The `LightRAG` orchestrator in [`lightrag/lightrag.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/lightrag.py) calls `check_and_migrate_data` (lines ~788-~858) during initialization. This triggers storage-specific migration routines:

- **Qdrant**: `setup_collection` (lines ~150-~380) handles batch data copying
- **Milvus**: `_migrate_collection_schema` (lines ~690-~1030) manages temporary collection creation and atomic renaming
- **PostgreSQL**: Various `_migrate_*` helpers (lines ~585-~1385) execute ALTER statements

## Migrating Qdrant Collections

Qdrant migrations use a batch-copy approach that preserves data integrity while transforming collection structure.

### Automatic Migration Flow

When you initialize a `LightRAG` instance with Qdrant storage, the system automatically executes the migration:

```python
from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,  # Returns vectors (e.g., 1536-dim)

    vector_db_storage_cls="QdrantVectorDBStorage",
    workspace="my_workspace"
)
await rag.initialize()  # Triggers migration if legacy collection detected

```

During `initialize()`, `QdrantVectorDBStorage.setup_collection()` performs the following:

1. Detects legacy collections using `_find_legacy_collection`
2. Creates a new collection with the workspace-aware name `lightrag_vdb_chunks_{model_suffix}`
3. Copies data batch-wise while adding `workspace_id` payload fields
4. Rewrites point IDs with workspace prefixes to ensure uniqueness
5. Verifies migrated record counts match the source

### Manual Migration Control

For advanced use cases requiring explicit control, instantiate the storage class directly:

```python
from lightrag.kg.qdrant_impl import QdrantVectorDBStorage
from qdrant_client import QdrantClient

client = QdrantClient(url="http://localhost:6334")
storage = QdrantVectorDBStorage(
    namespace="chunks",
    global_config={"embedding_batch_num": 128},
    embedding_func=my_embedding_function,
    workspace="my_workspace"
)

# Force migration logic explicitly

await storage._create_collection_if_not_exist()

```

Monitor logs for messages like:
- `Qdrant: Found legacy collection 'chunks' with 1245 records to migrate.`
- `✅ SUCCESS: All records migrated successfully!`

## Migrating Milvus Collections

Milvus requires a more complex migration strategy due to schema immutability constraints, utilizing temporary collections and iterator-based copying.

### Iterator-Based Schema Migration

The migration in [`lightrag/kg/milvus_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/milvus_impl.py) handles schema changes through:

1. **Temporary Collection Creation**: `_create_schema_for_namespace` builds a new collection with the correct schema
2. **Data Iteration**: Uses `client.query_iterator` (batch size 2000) to stream data from the legacy collection
3. **Atomic Swapping**: Renames the original collection to `<name>_old` and the temporary collection to the target name

```python
from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,
    vector_db_storage_cls="MilvusVectorDBStorage",
    workspace="demo_ws"
)
await rag.initialize()  # Triggers _migrate_collection_schema if needed

```

### Verifying Migration Status

Inspect the temporary collection before the atomic rename:

```python
from lightrag.kg.milvus_impl import MilvusVectorDBStorage

storage = MilvusVectorDBStorage(
    namespace="chunks",
    global_config={"embedding_batch_num": 256},
    embedding_func=my_embedding_function,
    workspace="manual_ws"
)

# Execute migration steps manually

await storage._migrate_collection_schema()

# Verify temporary collection exists

client = storage._create_milvus_client()
print(client.has_collection("lightrag_vdb_chunks_temp"))  # True during migration

```

## Migrating PostgreSQL Vector Tables

PostgreSQL migrations focus on schema evolution, adding columns and converting data formats without recreating tables.

### Schema Migration Helpers

The PostgreSQL implementation in [`lightrag/kg/postgres_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/postgres_impl.py) (lines ~585-~1385) includes several migration methods:

- `_migrate_llm_cache_schema`: Adds `chunk_id`, `cache_type`, and `queryparam` columns
- `_migrate_field_lengths`: Expands `entity_name` and `source_id` fields to accommodate larger identifiers
- `_migrate_llm_cache_to_flattened_keys`: Converts old LLM-cache keys to the new flattened format

```python
from lightrag import LightRAG

rag = LightRAG(
    embedding_func=my_embedding_function,
    vector_db_storage_cls="PostgresVectorDBStorage",
    workspace="pg_ws"
)
await rag.initialize()  # Runs all _migrate_* helpers automatically

```

### Running Specific Migrations

Target individual migrations for troubleshooting:

```python
from lightrag.kg.postgres_impl import PostgreSQLDB

pg = PostgreSQLDB(config={
    "host": "localhost",
    "port": 5432,
    "user": "lightrag",
    "password": "secret",
    "database": "lightrag",
    "workspace": "manual_pg",
    "max_connections": 10,
})

# Execute specific migrations

await pg._migrate_llm_cache_schema()
await pg._migrate_field_lengths()

```

## Configuration and Environment Variables

LightRAG uses environment variables to configure connection details and migration behavior. Place these in a `.env` file at the repository root:

| Variable | Purpose | Example |
|----------|---------|---------|
| `QDRANT_URL` / `MILVUS_URI` / `POSTGRES_HOST` | Connection endpoints | `QDRANT_URL=http://localhost:6334` |
| `QDRANT_WORKSPACE` / `MILVUS_WORKSPACE` / `POSTGRES_WORKSPACE` | Override default workspace names | `QDRANT_WORKSPACE=production` |
| `MODEL_NAME` | Determines the model suffix for collection names | `text-embedding-ada-002` |
| `QDRANT_UPSERT_MAX_PAYLOAD_BYTES` | Prevent payload size errors during batch migration | `16777216` |
| `QDRANT_UPSERT_MAX_POINTS_PER_BATCH` | Control batch sizes for large vectors | `100` |

## Troubleshooting Common Migration Issues

### Dimension Mismatch Errors

If you encounter `DataMigrationError: Dimension mismatch between legacy collection...`, verify that your embedding function's output dimension matches the legacy collection's vector size. This check occurs in Qdrant at `setup_collection` (lines ~200-~225) and in Milvus at `_check_vector_dimension` (lines ~1010-~1060).

### Large Payload Failures in Qdrant

Qdrant migrations may fail with payload size errors when migrating high-dimensional vectors or large metadata. Increase `QDRANT_UPSERT_MAX_PAYLOAD_BYTES` or reduce `QDRANT_UPSERT_MAX_POINTS_PER_BATCH` in your environment configuration.

### Undetected Legacy Collections

If LightRAG creates empty new collections instead of migrating existing data, verify your legacy collection names match expected patterns (`lightrag_vdb_{namespace}` or `{workspace}_{namespace}`). Set the appropriate `*_WORKSPACE` environment variable to match the legacy collection's workspace context.

### Manual Cleanup Requirements

After successful migration, LightRAG logs a warning: "Manual deletion is required after data migration verification." The legacy collection remains intact as a safety measure. Once you verify the new collection contains all records using `client.count`, delete the legacy collection manually via the database API or management UI.

## Summary

- **LightRAG automatically migrates** existing Qdrant, Milvus, and PostgreSQL vector databases when initializing storage instances with matching workspace configurations.
- **Migration preserves all data** while adding workspace isolation metadata and model-specific naming conventions to prevent collisions.
- **Dimension validation** prevents corruption by raising `DataMigrationError` when embedding model dimensions differ from legacy collections.
- **Environment variables** control connection details, workspace names, and batch processing limits for large-scale migrations.
- **Manual cleanup** of legacy collections is required after verification, as LightRAG leaves source data intact for safety.

## Frequently Asked Questions

### What happens if my embedding model dimensions don't match the legacy collection?

LightRAG aborts the migration with a `DataMigrationError` to prevent data corruption. According to the source code in [`lightrag/kg/qdrant_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/qdrant_impl.py) (lines ~200-~225) and [`lightrag/kg/milvus_impl.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/kg/milvus_impl.py) (lines ~1010-~1060), the system validates vector dimensions before copying data. You must either use an embedding function with matching dimensions or delete the legacy collection if you don't need to preserve that data.

### Can I migrate data from multiple legacy workspaces into one LightRAG workspace?

No, LightRAG's migration logic maps one legacy workspace to one new workspace-aware collection. The `effective_workspace` calculation in `__post_init__` methods ensures isolation. If you need to consolidate data from multiple legacy sources, run separate migration instances for each source workspace, then query across them using LightRAG's multi-storage capabilities.

### Does the migration process lock the database or cause downtime?

The migration creates new collections or tables rather than modifying existing ones in place. For Qdrant, data is copied batch-wise while the legacy collection remains readable. Milvus uses a temporary collection strategy with atomic renaming at the end. PostgreSQL runs ALTER statements that may briefly lock tables but maintains availability throughout. Always back up critical data before migration.

### How do I know when migration is complete and safe?

LightRAG logs explicit success messages like `✅ SUCCESS: All records migrated successfully!` and reports the record count verification. For Qdrant, check that `client.count` matches between source and destination. For Milvus, verify the temporary collection no longer exists and the main collection contains expected records. PostgreSQL migrations log completion of each `_migrate_*` helper. After confirmation, manually remove legacy collections to reclaim storage space.