# How Lazy Dimension Initialization Works in TurboQuantIndex

> Understand lazy dimension initialization in TurboQuantIndex. Learn how dimensions are set on first add, ensuring consistency for efficient data handling in your projects.

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: internals
- Published: 2026-06-16

---

**TurboQuantIndex supports lazy dimension initialization by allowing construction with `dim = None`, committing the dimension only when the first valid `add_2d` call succeeds, and validating that subsequent additions match the committed dimension.**

The `TurboQuantIndex` struct in the [RyanCodrai/turbovec](https://github.com/RyanCodrai/turbovec) repository implements a deferred initialization pattern that eliminates the need to specify vector dimensionality at construction time. This **lazy dimension initialization** workflow defers the commitment of the dimension parameter until the first batch of vectors is actually added, making the API more flexible for dynamic ingestion pipelines.

## Constructing a Lazy Index

To create an index without knowing the dimension up-front, use `TurboQuantIndex::new_lazy(bit_width)` implemented in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) (lines 200-210). This constructor only validates that the provided bit-width falls within the valid range of 2-4, leaving the internal `dim` field as `None`.

```rust
use turbovec::TurboQuantIndex;

// Create a lazy index without specifying dimension
let mut idx = TurboQuantIndex::new_lazy(4).expect("valid bit-width");
// At this point, idx.dim == None

```

## Committing the Dimension on First Add

The dimension is committed during the first successful call to `add_2d(vectors, dim)` at lines 352-368 of [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs). The logic validates that the supplied `dim` is a non-zero multiple of 8 and does not exceed `MAX_DIM`. Only after the input passes the invalid-coordinate check does the index set `self.dim = Some(dim)`, permanently locking the dimensionality.

If validation fails, the index remains uncommitted (`dim = None`), allowing the caller to retry with corrected parameters. This ensures the index can never be left in a half-initialized state.

## Enforcing Dimension Consistency

After the initial commitment, all subsequent `add_2d` calls must supply the same dimension. The check at lines 345-352 compares the incoming `dim` against `self.dim`, returning `AddError::DimMismatch` if they differ. This invariant guarantees that all vectors in the index share the same dimensionality, preventing corruption of the quantized internal structure.

## Safe Fallbacks for Uncommitted Indices

While `dim` remains `None` (before the first successful `add_2d`), the index exhibits specific defensive behaviors:

- **`search`** returns an empty result set without panicking (lines 424-432)
- **`prepare`** is a no-op because caches cannot be built without a known dimension
- **`add`** (the flat-vector form) panics with a clear message, as this API expects a pre-committed dimension

This design prevents crashes during pipeline setup while clearly signaling which operations require a committed dimension.

## Serializing Lazy State to Disk

Lazy indices persist their uncommitted state through the `write` method in [`turbovec/src/io.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/io.rs) (lines 283-294). The file header stores `dim = 0` as a sentinel value. When loaded, this sentinel restores the index to the same lazy-uncommitted state, ready for the first `add_2d` to establish the dimension. The `IdMapIndex` wrapper mirrors this lazy-initialization logic for higher-level ID management.

## Summary

- **Create lazy indices** with `TurboQuantIndex::new_lazy(bit_width)` without specifying vector dimensions
- **Dimension commits** only after the first `add_2d` call validates the input is a non-zero multiple of 8 and within bounds
- **Uncommitted indices** return empty search results, skip preparation, and panic on flat `add` calls
- **Consistency enforced** via `AddError::DimMismatch` for any subsequent additions with mismatched dimensions
- **Persistence** uses a `dim = 0` sentinel in [`turbovec/src/io.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/io.rs) to preserve lazy state across serialization rounds

## Frequently Asked Questions

### Can I search a TurboQuantIndex before adding any vectors?

Yes. Calling `search` on a lazy index before committing the dimension returns an empty result set rather than panicking. This behavior at lines 424-432 of [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) allows safe pipeline initialization, though the returned indices and scores vectors will be empty until vectors are actually added.

### What happens if I provide the wrong dimension in the second add_2d call?

The function returns `AddError::DimMismatch` immediately. The check at lines 345-352 enforces that the supplied `dim` parameter matches the value stored in `self.dim` from the first successful addition. This prevents mixing vectors of different dimensionalities in the same index.

### How does lazy initialization affect file serialization?

When writing a lazy index to disk, the `write` implementation stores `0` as the dimension sentinel in the header. Upon loading, this sentinel value restores `dim = None`, returning the index to its lazy-uncommitted state. The first `add_2d` after loading will then commit the dimension as if the index were newly constructed.

### Why does the flat add method panic on a lazy index?

The `add` method (which accepts a flat vector without an explicit dimension parameter) requires a pre-committed dimension to calculate offsets. Since a lazy index has `dim = None`, the method cannot determine how to interpret the flat buffer and panics with a clear message directing the user to use `add_2d` instead. This strict enforcement prevents undefined behavior from ambiguous vector layouts.