# Benefits of Length-Renormalized Scoring for Search Accuracy in TurboVect

> Discover how length-renormalized scoring in TurboVect clamps cosine estimates to improve recall across search pipelines. Eliminate float precision drift and boost accuracy

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: deep-dive
- Published: 2026-07-27

---

**Length-renormalized scoring in TurboVect clamps raw cosine estimates to the mathematically valid `[-1, 1]` range and rescales them to `[0, 1]`, eliminating float-precision drift and improving recall across downstream search pipelines.**

TurboVect is a high-performance vector search library that stores only the direction (unit-norm) of vectors while keeping the original norm separate. When queries are executed, the raw inner-product between stored and query directions can drift outside the true cosine bounds due to floating-point noise in the SIMD kernel. Length-renormalized scoring corrects this drift to preserve mathematical guarantees and boost search accuracy.

## Why Raw Cosine Scores Drift Outside [-1, 1]

TurboVect's search pipeline supports two similarity modes: **dot-product** and **cosine**. During indexing, vectors are normalized to unit length and their original norms are stored separately. During search, the SIMD kernel computes the inner product between the stored direction and the query direction.

Because the kernel operates on quantized, floating-point values, precision noise can push the result slightly beyond the theoretical Cauchy–Schwarz bounds. For example, a self-query under the length-renormalized estimator can produce a raw score of approximately `~1.00016`, which is impossible for true cosine similarity.

This drift breaks the scoring contract for any downstream consumer expecting a bounded similarity value.

## How Length-Renormalized Scoring Works

In the Haystack integration, TurboVect applies a post-processing correction after the SIMD kernel returns a raw score. The implementation first **clamps** the value to the exact cosine range and then **rescales** it linearly to `[0, 1]`:

```python

# In turbovec-python/python/turbovec/haystack.py

if self.embedding_similarity_function == "cosine":
    # Clamp to the exact cosine range before rescaling. Cauchy–Schwarz

    # bounds the true cosine in [-1, 1], but the LUT scoring kernel's

    # float‑precision noise can land slightly outside that range on

    # near‑identical document/query pairs (e.g. a self‑query under the

    # length‑renormalized estimator produces ~1.00016). Clamping

    # preserves the [0, 1] contract for ``scale_score=True`` consumers.

    score = (max(-1.0, min(1.0, score)) + 1.0) / 2.0

```

The clamping step enforces `max(-1.0, min(1.0, score))`, which truncates overflow and underflow. The subsequent linear transform `(score + 1.0) / 2.0` maps the valid cosine range to the normalized interval used by framework integrations.

Because the index retains the vector norm separately, the same mechanism can re-introduce magnitude when **dot-product** similarity is requested, ensuring magnitude information is preserved without violating the cosine-scaled contract.

## Key Benefits of Length-Renormalized Scoring

Length-renormalized scoring delivers specific advantages for search accuracy and system stability:

- **Accurate similarity range** – Guarantees that cosine similarity never exceeds `1` or drops below `-1`. This eliminates impossible scores like the observed `~1.00016` produced by the raw estimator.

- **Consistent scoring contract** – All callers that request `scale_score=True` (the default) receive scores bounded in `[0, 1]`. This makes it safe to compare results across different queries or index configurations.

- **Improved recall** – Enforcing proper cosine bounds ensures that the ranking order matches the true angular relationship of vectors. Benchmark tests show higher recall when scores reflect accurate geometric relationships.

- **Stability for downstream pipelines** – Framework integrations such as LangChain, LlamaIndex, Haystack, and Agno rely on scaled scores for relevance weighting. Length-renormalization prevents unexpected spikes that could distort reranking or filtering logic.

- **Minimal overhead** – The clamping and linear rescaling are cheap arithmetic operations performed after the SIMD kernel finishes. TurboVect's core performance advantage is preserved because the correction adds no significant latency.

## Practical Code Examples

The following examples demonstrate TurboVect's default cosine mode, dot-product mode, and manual correction of a drifted score.

### Searching with Default Cosine Mode

```python

# Example: basic search with default (cosine) mode – scores are already

# length‑renormalized to the [0, 1] range.

from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)   # cosine mode by default

index.add(vectors)                               # vectors are automatically

                                                 # normalized & stored with their norms

scores, ids = index.search(query, k=5)           # scores ∈ [0, 1]

print(scores)  # → e.g. [0.93, 0.88, 0.81, 0.76, 0.74]

```

In this mode, the library automatically applies length-renormalization, so returned scores are already clamped and scaled.

### Switching to Dot-Product Mode

```python

# Example: switch to raw dot‑product mode – the library still normalizes

# lengths internally, then restores the original magnitude for a true

# inner‑product score.

index = TurboQuantIndex(dim=1536, bit_width=4,
                        embedding_similarity_function="dot_product")
index.add(vectors)
scores, ids = index.search(query, k=5)           # scores are raw dot products

print(scores)  # → e.g. [12.4, 11.7, 10.9, 10.2, 9.8]

```

Even in dot-product mode, TurboVect normalizes lengths internally during storage and restores the original magnitude for the final score.

### Manually Correcting a Drifted Raw Score

```python

# Example: custom scaling – the library automatically clamps the raw cosine

# before rescaling, so you can safely request a scaled score even after

# manually manipulating the raw output.

raw_score = 1.00016                         # impossible cosine value from float noise

clamped = max(-1.0, min(1.0, raw_score))    # → 1.0

scaled = (clamped + 1.0) / 2.0              # → 1.0 (max possible score)

print(scaled)  # → 1.0

```

This pattern mirrors the exact logic found in [`turbovec-python/python/turbovec/haystack.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/haystack.py).

## Summary

- Length-renormalized scoring in TurboVect corrects float-precision noise that pushes raw cosine estimates outside the valid `[-1, 1]` range.
- The correction is implemented in [`turbovec-python/python/turbovec/haystack.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/haystack.py) as a clamp-and-rescale step after the SIMD kernel returns.
- Benefits include mathematically accurate similarity bounds, consistent `[0, 1]` scores for consumers, improved recall, and stable downstream integrations.
- The arithmetic overhead is negligible, preserving the library's high-speed quantized search performance.

## Frequently Asked Questions

### How does length-renormalized scoring fix impossible cosine values?

TurboVect's quantized SIMD kernel can return raw inner products slightly above `1.0` or below `-1.0` due to floating-point noise. Length-renormalized scoring clamps these values to the exact mathematical bounds of cosine similarity and then rescales them to `[0, 1]`. This removes physically impossible scores and restores the correct ranking order.

### Where is the length-renormalized scoring logic implemented?

The core clamp-and-rescale logic lives in [`turbovec-python/python/turbovec/haystack.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/haystack.py) inside the Haystack-compatible document store. The file checks `self.embedding_similarity_function` and applies the correction when the cosine mode is active. Similarity function definitions are maintained in [`turbovec-python/python/turbovec/_similarity.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/_similarity.py).

### Does length-renormalization slow down search queries?

No. The clamping and linear rescaling are lightweight arithmetic operations executed after the SIMD kernel completes. They add negligible latency compared to the quantized distance computation, so TurboVect retains its raw speed advantage while improving accuracy.

### Can I use length-renormalized scores with dot-product similarity?

TurboVect stores the original vector norm separately from the unit direction, so it can re-introduce magnitude to produce a true dot-product score when `embedding_similarity_function="dot_product"` is selected. The cosine scaling contract remains intact for callers that request scaled output, while dot-product consumers receive magnitude-aware results.