How to Debug Rotation Drift Errors During Turbovec Index Loading: A Complete Guide

A “rotation drift” error means the rebuilt orthogonal matrix does not match the fingerprint stored in the Turbovec index header, almost always caused by differences in the faer linear-algebra library, SIMD compile targets, or build environments.

When working with the RyanCodrai/turbovec crate, loading a previously saved index triggers a strict verification step that compares a freshly rebuilt rotation matrix against a compact fingerprint embedded in the v4 file header. If you encounter a rotation drift error during Turbovec index loading, the crate refuses to proceed rather than risk silently corrupting nearest-neighbor search results. Understanding the fingerprint format, tolerance thresholds, and rebuild logic in turbovec/src/rotation.rs lets you pinpoint whether the mismatch is benign numerical noise or a genuine environment mismatch.

Why Rotation Drift Occurs in Turbovec

Turbovec does not store the full rotation matrix inside .tv or .tvim files; deterministic reconstruction keeps indexes compact. Instead, the header stores a rotation fingerprint.

The Fingerprint Format in turbovec/src/rotation.rs

The fingerprint is defined in turbovec/src/rotation.rs and consists of two pieces:

• An FNV-1a 64-bit hash of the exact bit-wise rotation matrix.
• 64 sampled probe values taken from fixed matrix positions.

These are governed by two constants declared at lines 56–68:

/// Number of rotation probe values stored in a v4 file header.
pub(crate) const N_PROBES: usize = 64;
/// Absolute tolerance for comparing stored rotation probes against the
/// rebuilt rotation.
pub(crate) const PROBE_TOLERANCE: f32 = 1e-4;

During load, the matrix is rebuilt from the deterministic ROTATION_SEED and the loader invokes RotationFingerprint::matches to decide whether the index is safe to use.

Sources of Non-Determinism in Matrix Reconstruction

The orthogonal matrix is generated via a QR decomposition through faer::qr. According to the RyanCodrai/turbovec source code, QR is non-deterministic across CPU architectures, thread counts, or faer versions. While microscopic single-ULP differences are expected and absorbed by PROBE_TOLERANCE, larger deviations trigger a drift error:

• Different faer version or SIMD target — matrix elements can shift by roughly 1e-2.
• Changed random-number generator or seed — the entire matrix changes.
• Sign-convention changes in QR output — systematic sign flips.
• Corrupted file header — hash mismatch combined with out-of-range probes.

Step-by-Step Debugging Workflow for Rotation Drift

1. Reproduce the Error and Capture the Header

Start by attempting to load the index exactly as your production code does:

let idx = Index::open("my_index.tvim")?;

If this returns a RotationDrift variant, use the low-level helper turbovec::io::load_header—defined in turbovec/src/io.rs—to inspect the stored fingerprint without rebuilding the full rotation:

use turbovec::io::load_header;
let (header, _) = load_header("my_index.tvim")?;
println!("Stored hash: 0x{:016x}", header.rotation_fingerprint.hash);
println!("Stored probes: {:?}", &header.rotation_fingerprint.probes[..5]);

2. Rebuild the Fingerprint in the Same Process

Retrieve the expected dimension from the header, then regenerate the matrix and fingerprint using the exact functions the loader calls:

let dim = header.dim;
let rebuilt_rot = turbovec::rotation::make_rotation_matrix(dim);
let rebuilt_fp = turbovec::rotation::RotationFingerprint::compute(&rebuilt_rot, dim);

make_rotation_matrix is implemented in turbovec/src/rotation.rs at lines 13–51, and RotationFingerprint::compute lives at lines 6–23.

3. Compare Hash and Probe Differences

Evaluate the delta exactly as RotationFingerprint::matches does at lines 31–38:

if header.rotation_fingerprint.hash != rebuilt_fp.hash {
    println!("Hash mismatch!");
}
for (i, (stored, fresh)) in header.rotation_fingerprint.probes
        .iter()
        .zip(rebuilt_fp.probes.iter())
        .enumerate()
{
    let diff = (stored - fresh).abs();
    if diff > turbovec::rotation::PROBE_TOLERANCE {
        println!("Probe {} differs: {diff:e} > tolerance", i);
    }
}

If every probe difference is ≤ 1e-4, the drift is benign. If any probe exceeds PROBE_TOLERANCE, the loader is correct to reject the index.

4. Validate the Build Environment and Seed

Confirm that the environment matches the one used during index creation:

• faer version: run cargo tree | grep faer.
• rand_chacha and rand_distr versions.
• The ROTATION_SEED constant in turbovec/src/lib.rs (default is 0xdead_beef_cafe_f00d).

Inconsistent environments are the most common root cause of genuine rotation drift.

5. Fix Benign or Genuine Drift

• Benign drift — When all probe differences are within PROBE_TOLERANCE, matches returns true and the loader accepts the index automatically. If you need to force acceptance on a different machine temporarily, you can raise PROBE_TOLERANCE in a local copy of turbovec/src/rotation.rs, recompile, and reload. Do not ship this change, because it weakens the safety guarantee against silent search corruption.
• Genuine drift — When probes exceed tolerance, recreate the index on the current machine or migrate the source data so the stored fingerprint aligns with the local rotation.

6. Add Regression Tests

The test suite in turbovec/tests/io_v4.rs (around lines 260–280) deliberately injects drift to verify detection. Add a similar case to your own tests:

let mut corrupted = original_fingerprint;
for p in &mut corrupted.probes {
    *p *= 1.01; // ~1% change → clearly > 1e-4
}
assert!(!original_fingerprint.matches(&corrupted));

Full Diagnostic Rust Example

Combine the steps into a single utility you can run against any failing index:

use turbovec::{Index, rotation::RotationFingerprint};

fn main() -> turbovec::Result<()> {
    let path = "my_index.tvim";

    match Index::open(path) {
        Ok(i) => {
            println!("Index loaded successfully.");
            let results = i.search(&[0.1_f32, 0.2, 0.3], 10)?;
            println!("{:?}", results);
        }
        Err(e) if e.to_string().contains("rotation drift") => {
            eprintln!("Rotation drift detected; running diagnostics…");
            diagnose(path);
            return Err(e);
        }
        Err(e) => return Err(e),
    }
    Ok(())
}

fn diagnose(path: &str) {
    let (header, _) = turbovec::io::load_header(path).expect("failed to read header");
    println!("Stored dim = {}", header.dim);
    println!("Stored hash = 0x{:016x}", header.rotation_fingerprint.hash);

    let dim = header.dim;
    let rebuilt = turbovec::rotation::make_rotation_matrix(dim);
    let rebuilt_fp = RotationFingerprint::compute(&rebuilt, dim);

    let mut max_diff = 0.0;
    for (i, (s, r)) in header.rotation_fingerprint.probes
            .iter()
            .zip(rebuilt_fp.probes.iter())
            .enumerate()
    {
        let diff = (s - r).abs();
        if diff > max_diff {
            max_diff = diff;
        }
        if diff > turbovec::rotation::PROBE_TOLERANCE {
            println!("Probe {i} diff = {diff:e} (exceeds tolerance)");
        }
    }
    println!("Max probe difference = {max_diff:e}");
}

How Rotation Fingerprint Verification Works in turbovec/src/rotation.rs

The verification logic is intentionally strict. RotationFingerprint::matches, defined at lines 31–38, requires either an exact hash match or full probe agreement within tolerance:

pub fn matches(&self, rebuilt: &Self) -> bool {
    if self.hash == rebuilt.hash {
        return true;                     // exact match → no drift
    }
    self.probes
        .iter()
        .zip(rebuilt.probes.iter())
        .all(|(&stored, &fresh)| (stored - fresh).abs() <= PROBE_TOLERANCE)
}

Index::open in turbovec/src/lib.rs (around line 680) invokes this check during the load sequence. If both the hash and the probes fail, the function returns false and the loader raises the rotation drift error.

Summary

• Turbovec stores an FNV-1a hash plus 64 probe values in the v4 header instead of the full rotation matrix.
• During loading, RotationFingerprint::matches in turbovec/src/rotation.rs compares the stored fingerprint against a matrix rebuilt via make_rotation_matrix.
• Differences within PROBE_TOLERANCE (1e-4) are acceptable; anything larger triggers a rotation drift error.
• The leading causes are mismatched faer versions, differing SIMD targets, RNG changes, or altered seeds.
• Use turbovec::io::load_header to inspect the header, then recompute the fingerprint with RotationFingerprint::compute to see exactly which probes diverge.
• Recreate the index on the target environment when genuine drift is confirmed.

Frequently Asked Questions

What is a rotation drift error in Turbovec?

A rotation drift error occurs when the orthogonal matrix rebuilt during Index::open does not match the fingerprint stored inside the .tv or .tvim file header. The mismatch is detected by RotationFingerprint::matches in turbovec/src/rotation.rs, which rejects the index to prevent silent corruption of vector search results.

Why does Turbovec not store the full rotation matrix in the index file?

Storing the full matrix would significantly inflate file size. Instead, the crate writes a compact fingerprint—a 64-bit FNV-1a hash and 64 probe samples—then rebuilds the matrix deterministically from ROTATION_SEED during load. This design keeps indexes small while still verifying that the reconstruction matches the original environment.

Can I safely ignore a rotation drift error if my searches still work?

No. Ignoring the error risks using a radically different rotation, which destroys the geometric relationships Turbovec relies on for approximate nearest-neighbor search. The probe tolerance of 1e-4 is already calibrated to ignore harmless numerical noise; any mismatch larger than that indicates a real environmental discrepancy that can silently degrade result quality.

How do I prevent rotation drift when moving a Turbovec index to another machine?

Ensure the destination machine uses the exact same faer, rand_chacha, and rand_distr versions, the same ROTATION_SEED defined in turbovec/src/lib.rs, and the same target features (SIMD flags) as the machine that created the index. If perfect parity is impossible, recreate the index on the new machine so the header fingerprint matches the local rotation matrix.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}