# TurboQuantIndex vs IdMapIndex: What’s the Difference in Turbovec?

> Discover the difference between TurboQuantIndex and IdMapIndex in turbovec. Learn how IdMapIndex adds stable external ID mapping to quantized vectors for persistent user-defined identifiers.

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: deep-dive
- Published: 2026-08-22

---

**Turbovec’s `TurboQuantIndex` stores and searches quantized vectors by internal slot numbers, while `IdMapIndex` wraps that index to add stable external ID mapping, letting you work with user-defined identifiers that persist across mutating operations.**

Both index types are part of the [turbovec repository](https://github.com/RyanCodrai/turbovec) and share the same underlying quantization engine. The key difference lies in how you reference vectors: `TurboQuantIndex` exposes raw slot numbers, whereas `IdMapIndex` maintains an ID‑to‑slot map that keeps your external IDs stable when the index is updated or persisted.

## Overview of Turbovec’s Two Index Types

Turbovec is a Rust‐based vector search library that stores vectors in a compact, block‑wise quantized format. As shown in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs), the two primary index types are:

- **`TurboQuantIndex`** — the core approximate‑nearest‑neighbor (ANN) index. It manages vectors directly by their internal slot numbers and supports `add`, `search`, `delete`, `sync`, and serialization.
- **`IdMapIndex`** — a wrapper around `TurboQuantIndex` that adds a mapping from external IDs (e.g., `u64`) to internal slots. This layer is what makes user‑facing identifiers stable over time.

Both types are defined in the same source file—[`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) around line 295—and share methods like `add`, `search`, `write`, and `load`. The major difference is the API surface and the persisted data.

## TurboQuantIndex: Direct Slot‑Based Vector Search

`TurboQuantIndex` is the foundational structure. It handles the low‑level quantized storage and calculates distances using SIMD instructions when possible. When you add a vector, you receive an integer **slot** that identifies that vector inside the index. This slot is only meaningful while the index is unchanged; operations like deletion or re‑ordering can invalidate slot numbers.

Here’s how it appears in practice:

```rust
use turbovec::TurboQuantIndex;

let mut qidx = TurboQuantIndex::new(128, 4).unwrap();
let slot = qidx.add(&vec![0.1_f32; 128]).unwrap();
let results = qidx.search(&vec![0.1_f32; 128], 5).unwrap();
println!("Nearest slots: {:?}", results.indices);

```

This direct interface is ideal for low‑overhead experimentation or when you, the caller, already manage your own vector IDs externally.

## IdMapIndex: Stable External ID Mapping

`IdMapIndex` is designed for production scenarios where you need to refer to vectors by a stable identifier—one that remains unchanged even if the index internally rearranges its vector blocks. When you add a vector, you supply an ID (for example, a `u64`), and the index stores the mapping between that ID and the actual slot. Later searches return those IDs instead of raw slots.

```rust
use turbovec::IdMapIndex;

let mut iidx = IdMapIndex::new(128, 4).unwrap();
iidx.add(42u64, &vec![0.1_f32; 128]).unwrap();
let results = iidx.search(&vec![0.1_f32; 128], 5).unwrap();
println!("Nearest IDs: {:?}", results.ids);

```

The ID map is kept in memory and is serialized to disk when the index is written, guaranteeing that your external identifiers stay in sync with the vector data across `write`/`load` cycles.

## Key Differences at a Glance

| Feature | **TurboQuantIndex** | **IdMapIndex** |
|---|---|---|
| **Core reference** | Internal slot numbers returned by `add` | User‑supplied stable IDs (e.g., `u64`) |
| **Data layout** | Magic header `TV7\0` followed by quantized blocks | Same block data plus a serialized ID‑to‑slot map |
| **Public API** | `add(vec)` returns slot; `search` returns slot indices | `add(id, vec)`; `search` returns user IDs |
| **Use case** | Prototyping, or when you control ID assignment externally | Persisting a retrieval system that needs stable keys |
| **Implementation** | Core [`lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/lib.rs) struct (~line 295) | Wrapper struct in the same [`lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/lib.rs) module |

Both types share the same block‑wise quantization scheme, SIMD distance calculations, and optional calibration. The only real functional delta is the ID mapping layer.

## Code Examples: Using Both Index Types

The following examples show the full lifecycle for each index, including persisting to disk:

```rust
use turbovec::{TurboQuantIndex, IdMapIndex};

// ---------- TurboQuantIndex ----------
let mut qidx = TurboQuantIndex::new(128, 4).unwrap();
let slot = qidx.add(&vec![0.1_f32; 128]).unwrap();
let results = qidx.search(&vec![0.1_f32; 128], 5).unwrap();
println!("Nearest slots: {:?}", results.indices);

qidx.write("my_quant.idx").unwrap();
let loaded_q = TurboQuantIndex::load("my_quant.idx").unwrap();

// ---------- IdMapIndex ----------
let mut iidx = IdMapIndex::new(128, 4).unwrap();
iidx.add(42u64, &vec![0.1_f32; 128]).unwrap();
let results = iidx.search(&vec![0.1_f32; 128], 5).unwrap();
println!("Nearest IDs: {:?}", results.ids);

iidx.write("my_idmap.idx").unwrap();
let loaded_i = IdMapIndex::load("my_idmap.idx").unwrap();

```

Note that both `write` and `load` methods return `Result<(), Box<dyn Error>>`; the `.unwrap()` here is for brevity—production code should handle the errors explicitly.

## Where Each Index Shines

- **`TurboQuantIndex`** is the better choice when you are building a quick prototype or when you already hold your own identifier scheme separate from the index. Slot numbers are efficient and avoid the overhead of a mapping table.
- **`IdMapIndex`** is recommended for long‑running services or databases that need to add and remove vectors while keeping a stable external ID. For example, an e‑commerce search backend can store product IDs as keys, and the index stays consistent across inserting new products or removing old ones.

Because `IdMapIndex` wraps the same `TurboQuantIndex` engine inside, you get all the performance benefits of the quantized storage with the extra lifecycle safety of stable IDs.

## Summary

- **`TurboQuantIndex`** gives you raw slot‑based access to quantized vectors—fast and low‑overhead.
- **`IdMapIndex`** adds a persistent mapping from external IDs to slots, making it ideal for production use cases requiring stable references.
- Both types share the same quantized block format and remain defined in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) (structs begin around line 295).
- The persistence files (`write`/`load`) behave identically except that `IdMapIndex` serializes the ID map alongside the vector blocks.

## Frequently Asked Questions

### Can a TurboQuantIndex be converted to an IdMapIndex?

There is no direct in‑memory conversion method in the public API. You would need to add all vectors into a new `IdMapIndex` using your own IDs, or load the `TurboQuantIndex` and re‑insert them with a chosen ID scheme.

### Do both index types use the same quantized storage format?

Yes. Both `TurboQuantIndex` and `IdMapIndex` use the same block‑wise, quantized vector storage layout in their underlying engine, including the same magic header and SIMD‑accelerated distance calculations.

### Are `add` and `search` methods safe to call concurrently?

The public API as of the current source is not thread‑safe. You must guard the index with a mutex or another synchronization primitive if multiple threads need to call `add`, `remove`, or `search` on the same instance.

### Which index type is better for a large‑scale deployment?

`IdMapIndex` is generally recommended for production because it decouples the internal slot layout from the external identifiers. That way, even if the index reorders slots internally (e.g., after deletions), your application’s references remain valid. `TurboQuantIndex` remains ideal for lightweight, single‑threaded experiments where slot numbers are already under the caller’s control.