# How to Debug Rotation Drift Errors During Turbovec Index Loading: A Complete Guide

> Debug rotation drift errors during Turbovec index loading. Understand common causes like faer library, SIMD targets, and environment differences with this comprehensive guide.

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: how-to-guide
- Published: 2026-07-27

---

**A “rotation drift” error means the rebuilt orthogonal matrix does not match the fingerprint stored in the Turbovec index header, almost always caused by differences in the `faer` linear-algebra library, SIMD compile targets, or build environments.**

When working with the `RyanCodrai/turbovec` crate, loading a previously saved index triggers a strict verification step that compares a freshly rebuilt rotation matrix against a compact fingerprint embedded in the v4 file header. If you encounter a rotation drift error during Turbovec index loading, the crate refuses to proceed rather than risk silently corrupting nearest-neighbor search results. Understanding the fingerprint format, tolerance thresholds, and rebuild logic in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs) lets you pinpoint whether the mismatch is benign numerical noise or a genuine environment mismatch.

## Why Rotation Drift Occurs in Turbovec

Turbovec does not store the full rotation matrix inside `.tv` or `.tvim` files; deterministic reconstruction keeps indexes compact. Instead, the header stores a **rotation fingerprint**.

### The Fingerprint Format in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs)

The fingerprint is defined in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs) and consists of two pieces:

- An **FNV-1a 64-bit hash** of the exact bit-wise rotation matrix.
- **64 sampled probe values** taken from fixed matrix positions.

These are governed by two constants declared at lines 56–68:

```rust
/// Number of rotation probe values stored in a v4 file header.
pub(crate) const N_PROBES: usize = 64;
/// Absolute tolerance for comparing stored rotation probes against the
/// rebuilt rotation.
pub(crate) const PROBE_TOLERANCE: f32 = 1e-4;

```

During load, the matrix is rebuilt from the deterministic `ROTATION_SEED` and the loader invokes `RotationFingerprint::matches` to decide whether the index is safe to use.

### Sources of Non-Determinism in Matrix Reconstruction

The orthogonal matrix is generated via a QR decomposition through `faer::qr`. According to the `RyanCodrai/turbovec` source code, QR is **non-deterministic across CPU architectures, thread counts, or `faer` versions**. While microscopic single-ULP differences are expected and absorbed by `PROBE_TOLERANCE`, larger deviations trigger a drift error:

- **Different `faer` version or SIMD target** — matrix elements can shift by roughly `1e-2`.
- **Changed random-number generator or seed** — the entire matrix changes.
- **Sign-convention changes in QR output** — systematic sign flips.
- **Corrupted file header** — hash mismatch combined with out-of-range probes.

## Step-by-Step Debugging Workflow for Rotation Drift

### 1. Reproduce the Error and Capture the Header

Start by attempting to load the index exactly as your production code does:

```rust
let idx = Index::open("my_index.tvim")?;

```

If this returns a `RotationDrift` variant, use the low-level helper `turbovec::io::load_header`—defined in [`turbovec/src/io.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/io.rs)—to inspect the stored fingerprint without rebuilding the full rotation:

```rust
use turbovec::io::load_header;
let (header, _) = load_header("my_index.tvim")?;
println!("Stored hash: 0x{:016x}", header.rotation_fingerprint.hash);
println!("Stored probes: {:?}", &header.rotation_fingerprint.probes[..5]);

```

### 2. Rebuild the Fingerprint in the Same Process

Retrieve the expected dimension from the header, then regenerate the matrix and fingerprint using the exact functions the loader calls:

```rust
let dim = header.dim;
let rebuilt_rot = turbovec::rotation::make_rotation_matrix(dim);
let rebuilt_fp = turbovec::rotation::RotationFingerprint::compute(&rebuilt_rot, dim);

```

`make_rotation_matrix` is implemented in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs) at lines 13–51, and `RotationFingerprint::compute` lives at lines 6–23.

### 3. Compare Hash and Probe Differences

Evaluate the delta exactly as `RotationFingerprint::matches` does at lines 31–38:

```rust
if header.rotation_fingerprint.hash != rebuilt_fp.hash {
    println!("Hash mismatch!");
}
for (i, (stored, fresh)) in header.rotation_fingerprint.probes
        .iter()
        .zip(rebuilt_fp.probes.iter())
        .enumerate()
{
    let diff = (stored - fresh).abs();
    if diff > turbovec::rotation::PROBE_TOLERANCE {
        println!("Probe {} differs: {diff:e} > tolerance", i);
    }
}

```

If every probe difference is ≤ `1e-4`, the drift is benign. If any probe exceeds `PROBE_TOLERANCE`, the loader is correct to reject the index.

### 4. Validate the Build Environment and Seed

Confirm that the environment matches the one used during index creation:

- `faer` version: run `cargo tree | grep faer`.
- `rand_chacha` and `rand_distr` versions.
- The `ROTATION_SEED` constant in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) (default is `0xdead_beef_cafe_f00d`).

Inconsistent environments are the most common root cause of genuine rotation drift.

### 5. Fix Benign or Genuine Drift

- **Benign drift** — When all probe differences are within `PROBE_TOLERANCE`, `matches` returns `true` and the loader accepts the index automatically. If you need to force acceptance on a different machine temporarily, you can raise `PROBE_TOLERANCE` in a local copy of [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs), recompile, and reload. **Do not ship this change**, because it weakens the safety guarantee against silent search corruption.
- **Genuine drift** — When probes exceed tolerance, recreate the index on the current machine or migrate the source data so the stored fingerprint aligns with the local rotation.

### 6. Add Regression Tests

The test suite in [`turbovec/tests/io_v4.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/tests/io_v4.rs) (around lines 260–280) deliberately injects drift to verify detection. Add a similar case to your own tests:

```rust
let mut corrupted = original_fingerprint;
for p in &mut corrupted.probes {
    *p *= 1.01; // ~1% change → clearly > 1e-4
}
assert!(!original_fingerprint.matches(&corrupted));

```

## Full Diagnostic Rust Example

Combine the steps into a single utility you can run against any failing index:

```rust
use turbovec::{Index, rotation::RotationFingerprint};

fn main() -> turbovec::Result<()> {
    let path = "my_index.tvim";

    match Index::open(path) {
        Ok(i) => {
            println!("Index loaded successfully.");
            let results = i.search(&[0.1_f32, 0.2, 0.3], 10)?;
            println!("{:?}", results);
        }
        Err(e) if e.to_string().contains("rotation drift") => {
            eprintln!("Rotation drift detected; running diagnostics…");
            diagnose(path);
            return Err(e);
        }
        Err(e) => return Err(e),
    }
    Ok(())
}

fn diagnose(path: &str) {
    let (header, _) = turbovec::io::load_header(path).expect("failed to read header");
    println!("Stored dim = {}", header.dim);
    println!("Stored hash = 0x{:016x}", header.rotation_fingerprint.hash);

    let dim = header.dim;
    let rebuilt = turbovec::rotation::make_rotation_matrix(dim);
    let rebuilt_fp = RotationFingerprint::compute(&rebuilt, dim);

    let mut max_diff = 0.0;
    for (i, (s, r)) in header.rotation_fingerprint.probes
            .iter()
            .zip(rebuilt_fp.probes.iter())
            .enumerate()
    {
        let diff = (s - r).abs();
        if diff > max_diff {
            max_diff = diff;
        }
        if diff > turbovec::rotation::PROBE_TOLERANCE {
            println!("Probe {i} diff = {diff:e} (exceeds tolerance)");
        }
    }
    println!("Max probe difference = {max_diff:e}");
}

```

## How Rotation Fingerprint Verification Works in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs)

The verification logic is intentionally strict. `RotationFingerprint::matches`, defined at lines 31–38, requires either an exact hash match or full probe agreement within tolerance:

```rust
pub fn matches(&self, rebuilt: &Self) -> bool {
    if self.hash == rebuilt.hash {
        return true;                     // exact match → no drift
    }
    self.probes
        .iter()
        .zip(rebuilt.probes.iter())
        .all(|(&stored, &fresh)| (stored - fresh).abs() <= PROBE_TOLERANCE)
}

```

`Index::open` in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs) (around line 680) invokes this check during the load sequence. If both the hash and the probes fail, the function returns `false` and the loader raises the rotation drift error.

## Summary

- Turbovec stores an **FNV-1a hash** plus **64 probe values** in the v4 header instead of the full rotation matrix.
- During loading, `RotationFingerprint::matches` in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs) compares the stored fingerprint against a matrix rebuilt via `make_rotation_matrix`.
- Differences within `PROBE_TOLERANCE` (`1e-4`) are acceptable; anything larger triggers a rotation drift error.
- The leading causes are mismatched `faer` versions, differing SIMD targets, RNG changes, or altered seeds.
- Use `turbovec::io::load_header` to inspect the header, then recompute the fingerprint with `RotationFingerprint::compute` to see exactly which probes diverge.
- Recreate the index on the target environment when genuine drift is confirmed.

## Frequently Asked Questions

### What is a rotation drift error in Turbovec?

A rotation drift error occurs when the orthogonal matrix rebuilt during `Index::open` does not match the fingerprint stored inside the `.tv` or `.tvim` file header. The mismatch is detected by `RotationFingerprint::matches` in [`turbovec/src/rotation.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/rotation.rs), which rejects the index to prevent silent corruption of vector search results.

### Why does Turbovec not store the full rotation matrix in the index file?

Storing the full matrix would significantly inflate file size. Instead, the crate writes a compact fingerprint—a 64-bit FNV-1a hash and 64 probe samples—then rebuilds the matrix deterministically from `ROTATION_SEED` during load. This design keeps indexes small while still verifying that the reconstruction matches the original environment.

### Can I safely ignore a rotation drift error if my searches still work?

No. Ignoring the error risks using a radically different rotation, which destroys the geometric relationships Turbovec relies on for approximate nearest-neighbor search. The probe tolerance of `1e-4` is already calibrated to ignore harmless numerical noise; any mismatch larger than that indicates a real environmental discrepancy that can silently degrade result quality.

### How do I prevent rotation drift when moving a Turbovec index to another machine?

Ensure the destination machine uses the exact same `faer`, `rand_chacha`, and `rand_distr` versions, the same `ROTATION_SEED` defined in [`turbovec/src/lib.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/lib.rs), and the same target features (SIMD flags) as the machine that created the index. If perfect parity is impossible, recreate the index on the new machine so the header fingerprint matches the local rotation matrix.