How to Migrate FAISS IndexPQ to turbovec: A Complete Migration Guide

Migrating from FAISS IndexPQ to turbovec eliminates the training step, doubles compression from 8× to 16×, and adds built-in ID filtering while keeping the familiar add() and search() API.

FAISS IndexPQ has been the standard for memory-efficient similarity search, but RyanCodrai/turbovec provides a training-free alternative with hand-optimized SIMD kernels and higher compression ratios. This guide explains how to migrate your FAISS IndexPQ implementation to turbovec, covering the architectural differences, API mapping, and performance optimizations available in the Rust source code.

Why Switch from FAISS IndexPQ to turbovec?

Training-Free Quantization

FAISS IndexPQ requires a separate training step to learn codebook centroids via k-means on a representative dataset. In turbovec, this step is completely eliminated. According to the source code in turbovec/src/encode.rs (lines 1-8), the encoding pipeline uses a data-oblivious random rotation combined with a pre-computed Lloyd-Max codebook. This means you can start adding vectors immediately after instantiation without any train() call.

Memory Efficiency and Compression

FAISS IndexPQ typically uses 8-bit quantization (256 centroids per sub-vector), achieving roughly 4× compression for float32 vectors. turbovec supports 2-bit (4 centroids) and 4-bit (16 centroids) modes, delivering 16× and 8× compression respectively. For example, 10 million documents at 4-bit compression require approximately 4 GB of memory.

SIMD-Optimized Search with Filtering

The search kernels in turbovec/src/search.rs (lines 34-84, 150-170) implement hand-written NEON instructions for ARM and AVX-512BW for x86, outperforming FAISS FastScan by 12-20% on ARM while matching or beating it on x86. Unlike FAISS, turbovec supports hybrid filtering through the block_has_allowed logic (lines 13-21), which applies an allowlist mask during the kernel execution to guarantee that only allowed IDs are considered.

Migration Steps

1. Install turbovec

For Python projects, install via pip:

pip install turbovec

For Rust projects, add to your Cargo.toml:

[dependencies]
turbovec = "0.1"

Then import the required types:

use turbovec::{TurboQuantIndex, IdMapIndex};

2. Replace Index Creation

Replace the FAISS IndexPQ constructor with TurboQuantIndex, omitting the training step:

FAISS (Before):

import faiss

d = 1536
M = d // 8  # 8-bit sub-quantizer

index = faiss.IndexPQ(d, M, 8)
index.train(train_vectors)  # Required training step

index.add(db_vectors)

turbovec (After):

from turbovec import TurboQuantIndex

d = 1536
index = TurboQuantIndex(dim=d, bit_width=4)  # 4-bit = 16 centroids per sub-vector

index.add(db_vectors)  # Immediate indexing, no training

3. Migrate External ID Management

If you use FAISS IndexIDMap for stable external IDs, switch to IdMapIndex in turbovec. This structure stores a stable uint64 ID per vector and offers remove(id) in O(1) time.

from turbovec import IdMapIndex

idx = IdMapIndex(dim=d, bit_width=4)
idx.add_with_ids(vectors, external_ids)  # external_ids is a uint64 numpy array

# Deletion by external ID

idx.remove(some_id)  # O(1) operation

4. Update Search Calls

The search signature remains identical to FAISS:

# TurboQuantIndex returns internal indices

scores, ids = index.search(query, k=10)

# IdMapIndex returns external IDs directly

scores, ext_ids = idx.search(query, k=10)

5. Implement Hybrid Filtering

turbovec supports filtering at the kernel level via an allowlist parameter, eliminating the need for post-filtering:

import numpy as np

# Candidate set from another system (e.g., BM25 or SQL)

allowed = np.array([123, 456, 789], dtype=np.uint64)
scores, ids = idx.search(query, k=10, allowlist=allowed)

This leverages the block_has_allowed logic in turbovec/src/search.rs to check candidate blocks against the mask before scoring.

6. Persist and Load Indexes

Serialization works similarly to FAISS:

# Python

index.write("my_index.tq")
loaded = TurboQuantIndex.load("my_index.tq")

// Rust
index.write("my_index.tv")?;
let loaded = TurboQuantIndex::load("my_index.tv")?;

Key Architectural Differences

Understanding the internal implementation helps optimize your migration:

Feature	FAISS IndexPQ	turbovec TurboQuant
Training	Required k-means on training set	None – uses random rotation + Lloyd-Max codebook
Encoding	Float32 → 8-bit codes	Normalize → Rotate → TQ+ calibration → Quantize → Pack (encode.rs lines 46-53)
Scoring	Decodes codebooks during search	Uses per-vector scale factors (norm / <u, x̂>) for unbiased inner-product scoring
Filtering	Manual post-filtering	Kernel-level allowlist mask (search.rs lines 13-21)
Deletion	Not supported	O(1) removal by external ID in IdMapIndex

Complete Migration Examples

Python Migration

import numpy as np
from turbovec import IdMapIndex

# Configuration

dim = 1536
bit_width = 4  # 4-bit quantization

# Initialize index (no training needed)

index = IdMapIndex(dim=dim, bit_width=bit_width)

# Generate sample data

n_vectors = 100_000
vectors = np.random.randn(n_vectors, dim).astype(np.float32)
external_ids = np.arange(n_vectors, dtype=np.uint64) + 1_000_000

# Add vectors (encoding happens here)

index.add_with_ids(vectors, external_ids)

# Search

query = np.random.randn(1, dim).astype(np.float32)
scores, results = index.search(query, k=10)

print(f"Top IDs: {results[0]}")
print(f"Scores: {scores[0]}")

# Filtered search

candidates = np.array([1_000_123, 1_000_456], dtype=np.uint64)
scores_f, ids_f = index.search(query, k=5, allowlist=candidates)

Rust Migration

use turbovec::IdMapIndex;

fn main() -> anyhow::Result<()> {
    let dim = 1536usize;
    
    // Create 4-bit index
    let mut index = IdMapIndex::new(dim, 4);
    
    // Add vectors with external IDs
    let vectors = vec![0.1f32; 10_000 * dim];
    let ids: Vec<u64> = (0..10_000).map(|i| i + 1_000_000).collect();
    index.add_with_ids(&vectors, &ids);
    
    // Search
    let query = vec![0.1f32; dim];
    let (scores, ext_ids) = index.search(&query, 10);
    
    println!("Results: {:?}", &ext_ids[..10]);
    
    // Persist
    index.write("my_index.tvim")?;
    let loaded = IdMapIndex::load("my_index.tvim")?;
    
    Ok(())
}

Critical Migration Considerations

Bit-Width Selection: FAISS typically uses 8-bit codes, while turbovec offers 2-bit and 4-bit modes. Start with 4-bit for the best recall-speed trade-off, or use 2-bit for aggressive compression (16×).

Vector Normalization: turbovec stores a per-vector scale factor (f32) for unbiased scoring. While the encoding pipeline in turbovec/src/encode.rs handles normalization internally, ensure your input vectors have reasonable norms to avoid underflow.

GPU Support: Unlike FAISS, turbovec is CPU-only, optimized for NEON (ARM) and AVX-512BW (x86) instruction sets. Use turbovec when you need low-latency CPU search on edge devices or servers without GPUs.

Online Ingestion: FAISS may require rebuilding when adding vectors after training. turbovec supports dynamic updates—simply call add() or add_with_ids() again to index new vectors on the fly.

Summary

• Drop the training step: turbovec requires no train() call—instantiate TurboQuantIndex or IdMapIndex and start adding vectors immediately.
• Increase compression: Switch from 8-bit FAISS codes to 4-bit or 2-bit turbovec codes for 8× or 16× compression.
• Use IdMapIndex: Replace FAISS IndexIDMap with IdMapIndex for stable uint64 external IDs and O(1) deletion.
• Filter in the kernel: Pass allowlist arrays directly to search() instead of post-filtering results.
• Leverage SIMD: Benefit from hand-optimized NEON and AVX-512BW kernels in turbovec/src/search.rs without changing your API calls.

Frequently Asked Questions

Do I need to retrain my index when migrating from FAISS IndexPQ to turbovec?

No. turbovec does not require a training step. While FAISS IndexPQ needs k-means clustering on a training set to learn codebook centroids, turbovec uses a data-oblivious random rotation and pre-computed Lloyd-Max codebook implemented in turbovec/src/encode.rs. You can instantiate the index and immediately call add() with your vectors.

How does turbovec achieve higher compression than FAISS IndexPQ?

FAISS IndexPQ typically uses 8-bit quantization (256 centroids per sub-vector), while turbovec supports 2-bit (4 centroids) and 4-bit (16 centroids) quantization. This is implemented in the encoding pipeline at turbovec/src/encode.rs, which packs quantized codes into bits. A 4-bit index achieves 8× compression, and 2-bit achieves 16× compression compared to float32 vectors.

Can I use my existing FAISS index files with turbovec?

No, the file formats are incompatible. FAISS stores trained codebooks and quantized codes in its own format, while turbovec stores a random rotation matrix, Lloyd-Max codebooks, and per-vector scale factors. You must rebuild the index by loading your original vectors into turbovec and calling add() or add_with_ids(), then persist using write() to create .tq or .tvim files.

What SIMD instructions does turbovec use for search?

turbovec uses hand-written SIMD kernels for both ARM and x86 architectures. On ARM, it uses NEON instructions (score_4bit_block_neon in turbovec/src/search.rs). On x86, it uses AVX-2 and AVX-512BW (search_multi_query_avx2 and search_multi_query_avx512bw). These kernels operate directly on packed bit codes and apply per-vector scales during scoring, beating FAISS FastScan by 12-20% on ARM according to the implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}