# turbovec | Ryan Codrai | Knowledge Base | Instagit

A vector index built on TurboQuant, written in Rust with Python bindings

GitHub Stars: 8.2k

Repository: https://github.com/RyanCodrai/turbovec

---

## Articles

### [How to Integrate Turbovec with LangChain for Vector Storage](/RyanCodrai/turbovec/how-to-integrate-turbovec-with-langchain-for-vector-storage)

Integrate Turbovec with LangChain for persistent, quantized vector storage. Replace in-memory stores easily with TurboQuantVectorStore without code changes. Boost performance and scalability.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Use Turbovec for In-Memory Serialization and Deserialization](/RyanCodrai/turbovec/how-to-use-turbovec-for-in-memory-serialization-and-deserialization)

Learn to use turbovec for fast in-memory serialization and deserialization of quantized vector indexes. Achieve deterministic, disk-free round-trip data handling.

- Tags: how-to-guide
- Published: 2026-08-22

### [How turbovec Handles Large Batch Operations and Ctrl-C Interruption: Architecture Deep Dive](/RyanCodrai/turbovec/how-does-turbovec-handle-large-batch-operations-and-support-for-ctrl-c-interruption)

Explore how turbovec manages large batch operations and Ctrl-C interruption. Discover its architecture for efficient parallel query processing and graceful aborts.

- Tags: architecture
- Published: 2026-08-22

### [How to Perform Hybrid Retrieval with Filtered Search in Turbovec Using an Allowlist](/RyanCodrai/turbovec/how-to-perform-hybrid-retrieval-with-filtered-search-in-turbovec-using-allowlist)

Learn to emulate hybrid retrieval in Turbovec by filtering metadata to an allowlist and performing vector search on the constrained set. Enhance your search accuracy now.

- Tags: how-to-guide
- Published: 2026-08-22

### [SIMD Kernel Optimizations in Turbovec: x86 AVX2, AVX‑512, and ARM NEON Implementations](/RyanCodrai/turbovec/what-simd-kernel-optimizations-does-turbovec-use-on-x86-and-arm-architectures)

Explore Turbovec's SIMD kernel optimizations using x86 AVX2, AVX-512, and ARM NEON for accelerated 4-bit vector search. Achieve bit-identical scoring across architectures.

- Tags: internals
- Published: 2026-08-22

### [Effective TQ+ Calibration Rules in Turbovec: A Complete Guide](/RyanCodrai/turbovec/what-are-the-rules-for-effective-tq-calibration-in-turbovec)

Master effective TQ+ calibration rules in Turbovec. Learn the two-step process, sampling needs, and why early calibration is crucial for accurate results. Avoid common pitfalls.

- Tags: how-to-guide
- Published: 2026-08-22

### [When and How to Apply TQ+ Calibration in Turbovec for Maximum Recall](/RyanCodrai/turbovec/when-and-how-should-tq-calibration-be-applied-in-turbovec-for-maximum-recall)

Boost TurboVect recall by 2.5 points on average with TQ+ calibration. Learn when and how to apply this per-coordinate shift and scale transformation for optimal results.

- Tags: best-practices
- Published: 2026-08-22

### [How to Configure Threading for Parallel Search in Turbovec Using `RAYON_NUM_THREADS`](/RyanCodrai/turbovec/how-to-configure-threading-for-parallel-search-in-turbovec-using-rayon-num-threads)

Optimize Turbovec search performance by configuring threading with RAYON_NUM_THREADS. Learn how to set the environment variable for efficient parallel processing and maximize your hardware utilization. Control thread pool size ...

- Tags: how-to-guide
- Published: 2026-08-22

### [Turbovec Bit Widths: How 2, 3, and 4-Bit Quantization Impact Compression and Performance](/RyanCodrai/turbovec/what-are-the-available-bit-widths-in-turbovec-and-how-do-they-affect-performance-and-compression)

Explore Turbovec's 2, 3, and 4-bit quantization options to understand their impact on compression ratios and performance. Optimize your vector search with precise bit width selection.

- Tags: performance
- Published: 2026-08-22

### [How to Perform Incremental Saves with Turbovec Using `sync()`](/RyanCodrai/turbovec/how-to-perform-incremental-saves-with-turbovec-using-sync)

Learn how to perform incremental saves with Turbovec's sync() method. Efficiently update your index by writing only changed data, avoiding full rewrites.

- Tags: how-to-guide
- Published: 2026-08-22

### [Turbovec Index File Formats Explained: `.tv` and `.tvim` Binary Structure](/RyanCodrai/turbovec/what-is-the-file-format-structure-for-turbovec-indexes-tv-and-tvim)

Explore the Turbovec index file formats .tv and .tvim. Understand the binary structure for compact snapshots and persistent indexes. Learn how Turbovec stores vector data.

- Tags: internals
- Published: 2026-08-22

### [How Turbovec Handles Deletions in the TurboQuantIndex and IdMapIndex](/RyanCodrai/turbovec/how-does-turbovec-handle-deletions-in-turboquantindex-and-idmapindex)

Learn how turbovec efficiently handles deletions in TurboQuantIndex and IdMapIndex using swap_remove and dual-swap for O(1) synchronized updates without element shifting.

- Tags: internals
- Published: 2026-08-22

### [TurboQuantIndex vs IdMapIndex: What’s the Difference in Turbovec?](/RyanCodrai/turbovec/what-is-the-difference-between-turboquantindex-and-idmapindex-in-turbovec)

Discover the difference between TurboQuantIndex and IdMapIndex in turbovec. Learn how IdMapIndex adds stable external ID mapping to quantized vectors for persistent user-defined identifiers.

- Tags: deep-dive
- Published: 2026-08-22

### [TurboVec Index Types Explained: TurboQuantIndex vs IdMapIndex Use Cases](/RyanCodrai/turbovec/what-are-the-different-index-types-in-turbovec-and-their-use-cases)

Explore TurboVec index types: TurboQuantIndex for speed and IdMapIndex for stable IDs. Learn use cases for eager and lazy constructors.

- Tags: deep-dive
- Published: 2026-08-22

### [How Turbovec's Search Kernel Works with Rotated Queries: SIMD Lookup Table Architecture](/RyanCodrai/turbovec/how-does-turbovecs-search-kernel-work-with-rotated-queries)

Discover how Turbovec's search kernel handles rotated queries using SIMD lookup tables. Learn about its efficient pre-computation and zero runtime rotation overhead for faster vector search.

- Tags: internals
- Published: 2026-08-22

### [Length-Renormalized Scoring in Turbovec: How It Unbiaseds Similarity Search](/RyanCodrai/turbovec/what-is-length-renormalized-scoring-in-turbovec-and-why-is-it-used)

Discover length-renormalized scoring in Turbovec. This technique eliminates bias from scalar quantization without extra storage or runtime overhead, ensuring accurate similarity search.

- Tags: deep-dive
- Published: 2026-08-22

### [How Turbovec Vectors Are Bit-Packed for Compression](/RyanCodrai/turbovec/how-are-turbovec-vectors-bit-packed-for-compression)

Discover how Turbovec vectors achieve compression by quantizing f32 dimensions to 2-4 bit integer codes. Learn about bit-packing and SIMD blocks for efficient search.

- Tags: deep-dive
- Published: 2026-08-22

### [Lloyd-Max Scalar Quantization in Turbovec: Optimal Vector Compression Explained](/RyanCodrai/turbovec/what-is-lloyd-max-scalar-quantization-in-the-context-of-turbovec)

Discover Lloyd-Max scalar quantization in Turbovec. Learn how it compresses high-dimensional vectors using optimal reconstruction levels for minimal mean-squared error and efficient 2-4 bit compression.

- Tags: deep-dive
- Published: 2026-08-22

### [How Per-Coordinate Calibration (TQ+) Works in Turbovec: A Complete Guide](/RyanCodrai/turbovec/how-does-per-coordinate-calibration-tq-work-in-turbovec)

Learn how Per-Coordinate Calibration TQ+ works in Turbovec. Understand the shift-and-scale transformation for improved recall in TurboQuant. A complete guide.

- Tags: deep-dive
- Published: 2026-08-22

### [What Is the Purpose of the Random Rotation in Turbovec? Explained](/RyanCodrai/turbovec/what-is-the-purpose-of-random-rotation-in-turbovec)

Discover why Turbovec uses random rotation to break coordinate correlations, enhance recall, and ensure reproducible results across all platforms.

- Tags: deep-dive
- Published: 2026-08-22

### [How Turbovec Normalizes Vectors: L2 Normalization and Calibration Scaling](/RyanCodrai/turbovec/how-does-turbovec-normalize-vectors)

Discover how Turbovec normalizes vectors using L2-normalization and calibration scaling. Optimize TQ+ quantization with learned parameters for improved performance. Learn more.

- Tags: deep-dive
- Published: 2026-08-22

### [The Six Stages of Turbovec's Compression Pipeline](/RyanCodrai/turbovec/what-are-the-six-stages-of-turbovecs-compression-pipeline)

Explore the six stages of Turbovec's compression pipeline: normalize, rotate, calibrate, quantize, bit-pack, and scale. Reduce storage up to 16x while preserving unbiased inner-product search.

- Tags: internals
- Published: 2026-08-22

### [The TurboQuant Algorithm Explained: How turbovec Achieves 4-Bit Vector Compression](/RyanCodrai/turbovec/what-is-the-turboquant-algorithm-used-in-turbovec)

Explore the TurboQuant algorithm, turbovec's core engine for 2-4 bit vector compression. Achieve near-optimal similarity search quality with zero training data.

- Tags: deep-dive
- Published: 2026-08-22

### [How Lloyd-Max Scalar Quantization Achieves Near-Optimal Distortion in TurboVec](/RyanCodrai/turbovec/turbovec-lloyd-max-scalar-quantization-distortion)

Discover how Lloyd-Max scalar quantization in TurboVec achieves near-optimal distortion through pre-computed boundaries and TQ+ calibration. Learn the technical details.

- Tags: deep-dive
- Published: 2026-07-27

### [How to Debug Rotation Drift Errors During Turbovec Index Loading: A Complete Guide](/RyanCodrai/turbovec/turbovec-debug-rotation-drift-errors-loading-indices)

Debug rotation drift errors during Turbovec index loading. Understand common causes like faer library, SIMD targets, and environment differences with this comprehensive guide.

- Tags: how-to-guide
- Published: 2026-07-27

### [How Turbovec Implements Concurrent Read Locking Without Holding the GIL](/RyanCodrai/turbovec/turbovec-concurrent-read-locking-no-gil)

Discover how Turbovec uses RwLock and py.detach closures for concurrent read locking, ensuring vector search workloads never hold the Python GIL. Optimize your Python performance!

- Tags: internals
- Published: 2026-07-27

### [turbovec MAX_DIM Limits and Dimension Requirements: The 16384 Cap and Multiple-of-8 Rule](/RyanCodrai/turbovec/turbovec-max_dim-limits-dimension-requirements)

Understand turbovec MAX_DIM limits and dimension requirements. Learn about the 16384 cap and multiple-of-8 rule for safe vector construction and storage.

- Tags: api-reference
- Published: 2026-07-27

### [How to Use Turbovec with the Haystack DocumentStore: Setup, Retrieval, and Persistence](/RyanCodrai/turbovec/turbovec-integrate-haystack-document-store)

Learn to use Turbovec with Haystack DocumentStore for fast embedding retrieval and storage. Explore setup, retrieval, and persistence with TurboQuantDocumentStore.

- Tags: how-to-guide
- Published: 2026-07-27

### [Why Turbovec Outperforms FAISS on ARM but Lags on x86 for 2‑Bit Quantization](/RyanCodrai/turbovec/turbovec-performance-arm-vs-x86-2bit-faiss)

Discover why Turbovec excels on ARM but trails FAISS on x86 for 2-bit quantization. Explore NEON vs AVX-512 performance differences and optimize your vector search.

- Tags: performance
- Published: 2026-07-27

### [Turbovec .tv and .tvim File Format Structure: Quantized Vector Storage Explained](/RyanCodrai/turbovec/turbovec-tv-tvim-file-format-quantized-vectors)

Explore the .tv and .tvim file format structure for quantized vector storage. Understand how Turbovec stores packed quantization codes, scales, and calibration data efficiently.

- Tags: internals
- Published: 2026-07-27

### [How to Handle Float64 Arrays in Turbovec: Casting to Float32](/RyanCodrai/turbovec/turbovec-handle-float64-arrays-cast-float32)

Learn how to handle float64 arrays in Turbovec by casting to float32. Avoid errors and ensure smooth data processing with this essential guide.

- Tags: how-to-guide
- Published: 2026-07-27

### [swap_remove in turbovec: TurboQuantIndex vs IdMapIndex Explained](/RyanCodrai/turbovec/turbovec-swap_remove-turboquantindex-vs-idmapindex)

Understand swap_remove in turbovec. Learn when to use TurboQuantIndex over IdMapIndex for O(1) removal and optimal performance based on your access and ID needs.

- Tags: internals
- Published: 2026-07-27

### [How to Configure Multi-Threading in Turbovec Using `RAYON_NUM_THREADS`: Thread Caps and Runtime Warnings Explained](/RyanCodrai/turbovec/turbovec-configure-multithreading-rayon_num_threads)

Configure multi-threading in Turbovec by setting RAYON_NUM_THREADS. Learn about thread caps and runtime warnings to optimize performance efficiently.

- Tags: how-to-guide
- Published: 2026-07-27

### [Benefits of Length-Renormalized Scoring for Search Accuracy in TurboVect](/RyanCodrai/turbovec/turbovec-length-renormalized-scoring-search-accuracy)

Discover how length-renormalized scoring in TurboVect clamps cosine estimates to improve recall across search pipelines. Eliminate float precision drift and boost accuracy

- Tags: deep-dive
- Published: 2026-07-27

### [How to Integrate Turbovec with LlamaIndex for Vector Storage](/RyanCodrai/turbovec/turbovec-integrate-llamaindex-vector-storage)

Integrate Turbovec with LlamaIndex for efficient vector storage. Configure quantization and similarity modes using TurboQuantVectorStore for optimized performance.

- Tags: how-to-guide
- Published: 2026-07-27

### [How to Integrate Turbovec with LangChain as a Vector Store: Complete Guide](/RyanCodrai/turbovec/turbovec-integrate-langchain-vector-store)

Integrate Turbovec with LangChain as a vector store effortlessly. Use TurboQuantVectorStore to replace InMemoryVectorStore with a quantized, persistent index without changing your chain code.

- Tags: tutorial
- Published: 2026-07-27

### [How Turbovec's Rotation Fingerprint Prevents Recall Loss When Loading Indices](/RyanCodrai/turbovec/turbovec-rotation-fingerprint-prevent-recall-loss-load)

Discover how Turbovec's rotation fingerprint prevents recall loss by detecting bit-level mismatches in orthogonal rotation matrices during index loading, ensuring search accuracy.

- Tags: internals
- Published: 2026-07-27

### [How to Serialize and Deserialize Turbovec Indices Using `tobytes` and `frombytes`](/RyanCodrai/turbovec/turbovec-serialize-deserialize-indices-to_bytes-from_bytes)

Serialize and deserialize Turbovec indices with tobytes and frombytes. Save and load your in-memory index efficiently using this method.

- Tags: how-to-guide
- Published: 2026-07-27

### [How SIMD Kernel Optimizations Work for NEON and AVX-512 in Turbovec](/RyanCodrai/turbovec/turbovec-simd-kernel-optimization-neon-avx512)

Discover how Turbovec accelerates vector search with SIMD kernel optimizations for NEON and AVX-512. Learn about parallel computation and architecture-specific implementations for high throughput.

- Tags: internals
- Published: 2026-07-27

### [How to Use IdMapIndex for Stable External Vector IDs with O(1) Removal](/RyanCodrai/turbovec/turbovec-idmapindex-stable-external-ids-o1-removal)

Learn how to use IdMapIndex for stable external vector IDs. Achieve O(1) removal with TurboQuantIndex and bidirectional hash maps for constant-time operations.

- Tags: how-to-guide
- Published: 2026-07-27

### [Turbovec Memory Compression Ratio Compared to Float32: A Complete Analysis](/RyanCodrai/turbovec/turbovec-memory-compression-ratio-vs-float32)

Discover Turbovec's impressive memory compression ratios, achieving up to 16x reduction compared to float32. Analyze its performance and space-saving benefits.

- Tags: analysis
- Published: 2026-07-27

### [How to Implement Filtered or Hybrid Search with an Allowlist in IdMapIndex](/RyanCodrai/turbovec/turbovec-filtered-hybrid-search-allowlist-idmapindex)

Learn to implement filtered or hybrid search with an allowlist in IdMapIndex using TurboQuantIndex. Restrict vector searches to specific IDs for efficient SIMD-accelerated nearest-neighbor search.

- Tags: how-to-guide
- Published: 2026-07-27

### [How Lazy Dimension Initialization Works in TurboQuantIndex](/RyanCodrai/turbovec/turbovec-lazy-dimension-initialization-turboquantindex)

Discover how TurboQuantIndex implements lazy dimension initialization, delaying dimension assignment until the first batch insertion for optimized performance.

- Tags: internals
- Published: 2026-07-27

### [turbovec Bit Width Options: Supported Values and Use Cases](/RyanCodrai/turbovec/turbovec-supported-bit-widths-usage)

Explore turbovec bit width options. Discover supported values 2, 3, and 4 bits to maximize compression, balance size, and achieve superior recall with SIMD acceleration.

- Tags: deep-dive
- Published: 2026-07-27

### [How TurboQuant Compares to FAISS IndexPQ: A Deep Dive into Vector Quantization Performance](/RyanCodrai/turbovec/turbovec-turboquant-vs-faiss-indexpq)

Discover how TurboQuant outperforms FAISS IndexPQ in vector quantization. Get 2-3x faster inserts and 1.5-2x faster searches with built-in cosine similarity and Python integration.

- Tags: deep-dive
- Published: 2026-07-27

### [Migrating from FAISS to turbovec for Vector Search: A Complete Infrastructure Guide](/RyanCodrai/turbovec/migrating-from-faiss-to-turbovec-for-vector-search-infrastructure)

Migrate from FAISS to turbovec for vector search. Reduce RAM by 8x, eliminate rebuilds, and get direct SIMD filtered retrieval. Learn more in this infrastructure guide.

- Tags: migration-guide
- Published: 2026-07-18

### [Security Considerations for Loading Untrusted .tv and .tvim Files in TurboVec](/RyanCodrai/turbovec/security-considerations-for-loading-untrusted-tvtvim-files)

Safely load untrusted .tv and .tvim files in TurboVec with defense-in-depth validation including magic-byte checks version enforcement and overflow-protected allocation

- Tags: security
- Published: 2026-07-18

### [How turbovec Uses the Lloyd-Max Algorithm for Scalar Quantization](/RyanCodrai/turbovec/how-is-lloyd-max-algorithm-used-for-scalar-quantization)

Discover how turbovec leverages the Lloyd-Max algorithm for scalar quantization. Achieve deterministic vector compression without training by optimizing boundaries and centroids.

- Tags: deep-dive
- Published: 2026-07-18

### [Turbovec Empty Allowlist Handling and Selective Filtering Explained](/RyanCodrai/turbovec/turbovec-handling-of-empty-allowlists-and-selective-filtering)

Understand Turbovec's empty allowlist error handling and how selective filtering efficiently deduplicates IDs using boolean masks for optimized results. Learn how to leverage Turbovec's features.

- Tags: deep-dive
- Published: 2026-07-18

### [How the TurboVec `prepare()` Method Optimizes First Search Latency](/RyanCodrai/turbovec/how-does-the-prepare-method-optimize-first-search-latency)

Discover how TurboVec's prepare() method optimizes first search latency by pre-materializing internal caches, eliminating allocation and computation costs for faster queries.

- Tags: performance
- Published: 2026-07-18

### [Handling NaN/Inf Values in Vectors During Add and Search in TurboVec](/RyanCodrai/turbovec/handling-nan-inf-values-in-vectors-during-add-and-search-in-turbovec)

Learn how TurboVec prevents index corruption by handling NaN/Inf values in vectors during add and search. Discover the first invalid coord validation helper.

- Tags: how-to-guide
- Published: 2026-06-16

### [Migrating from FAISS IndexPQ to turbovec: A Complete Guide](/RyanCodrai/turbovec/migrating-from-faiss-indexpq-to-turbovec)

Migrate from FAISS IndexPQ to turbovec. Enjoy 8×–16× compression with 2-bit or 4-bit quantization, no training, and allowlist filtering. Compatible Python and Rust APIs. Learn more!

- Tags: migration-guide
- Published: 2026-06-16

### [Understanding swap_remove Semantics and Order Preservation in Turbovec](/RyanCodrai/turbovec/understanding-swap_remove-semantics-and-order-preservation-in-turbovec)

Understand turbovec swap_remove semantics. Learn how O(1) removal sacrifices order preservation by swapping with the last element. Discover the former index of the moved vector.

- Tags: internals
- Published: 2026-06-16

### [TurboVec Dimension Constraints: Multiple of 8 and Maximum 65536](/RyanCodrai/turbovec/what-are-the-dimension-constraints-for-turbovec-dim-multiple-of-8-and-65536)

Discover TurboVec dimension constraints: learn why dimensions must be multiples of 8 and capped at 65536 to prevent errors with RyanCodrai/turbovec.

- Tags: internals
- Published: 2026-06-16

### [Security Considerations for Loading Untrusted .tv Files in Turbovec](/RyanCodrai/turbovec/security-considerations-for-loading-untrusted-.tv-files-in-turbovec)

Learn how Turbovec secures untrusted .tv files by validating every field before memory allocation, preventing DoS and memory corruption attacks. Protect your system now.

- Tags: security-best-practices
- Published: 2026-06-16

### [Using prepare() to Warm Up Search Caches in TurboVec](/RyanCodrai/turbovec/using-prepare-to-warm-up-search-caches-in-turbovec)

Optimize TurboVec search performance by calling TurboQuantIndex::prepare() after index setup. Eagerly initialize search caches and eliminate first-query latency.

- Tags: how-to-guide
- Published: 2026-06-16

### [Lloyd-Max Algorithm for Optimal Scalar Quantization Buckets in Turbovec](/RyanCodrai/turbovec/what-is-the-lloyd-max-algorithm-for-optimal-scalar-quantization-buckets-in-turbovec)

Learn how the Lloyd-Max algorithm in turbovec optimizes scalar quantization buckets for Beta-distributed embeddings, minimizing errors with iterative centroid updates.

- Tags: deep-dive
- Published: 2026-06-16

### [How Random Orthogonal Rotation Enables Data-Oblivious Quantization in TurboQuantIndex](/RyanCodrai/turbovec/how-does-random-orthogonal-rotation-enable-data-oblivious-quantization-in-turboquantindex)

Discover how random orthogonal rotation in TurboQuantIndex achieves data-oblivious quantization using a pre-computed codebook, eliminating the need to inspect raw data. Improve your vector indexing efficiency.

- Tags: deep-dive
- Published: 2026-06-16

### [Why TurboQuant Requires No Separate Training Phase: Data-Oblivious Vector Quantization Explained](/RyanCodrai/turbovec/why-does-turboquant-require-no-separate-training-phase)

Discover why TurboQuant needs no separate training phase. Learn how data-oblivious vector quantization mathematically derives parameters and calibrates automatically, unlike traditional methods.

- Tags: deep-dive
- Published: 2026-06-16

### [Turbovec vs FAISS FastScan Performance Benchmarks: A Deep Dive into SIMD-Accelerated Vector Search](/RyanCodrai/turbovec/turbovec-vs-faiss-fastscan-performance-benchmarks)

Discover Turbovec vs FAISS FastScan performance benchmarks. Turbovec offers a speed advantage on ARM and x86 with SIMD acceleration and maintains data layout compatibility.

- Tags: performance
- Published: 2026-06-16

### [Integrating turbovec with LangChain, LlamaIndex, Haystack, or Agno: A Complete Guide](/RyanCodrai/turbovec/integrating-turbovec-with-langchain-llamaindex-haystack-or-agno)

Integrate Turbovec with LangChain LlamaIndex Haystack or Agno and quantize embeddings to 2-4 bits. This guide offers full API compatibility for efficient vector search.

- Tags: how-to-guide
- Published: 2026-06-16

### [How Length-Renormalization Corrects Scoring Bias in TurboVec: Implementation Deep Dive](/RyanCodrai/turbovec/how-does-length-renormalization-correct-scoring-bias-in-turbovec)

Discover how TurboVec uses length-renormalization to fix scoring bias. Learn about its encoding and SIMD search kernel implementation for unbiased inner-product estimates.

- Tags: deep-dive
- Published: 2026-06-16

### [Memory Compression Ratios for Different Dimensionality Vectors in TurboVec: 2-Bit vs 4-Bit Benchmarks](/RyanCodrai/turbovec/what-are-the-memory-compression-ratios-for-different-dimensionality-vectors-in-turbovec)

Discover TurboVec memory compression ratios for 2-bit and 4-bit quantization. See performance gains up to 15x for vectors up to 3072 dimensions.

- Tags: performance
- Published: 2026-06-16

### [When to Use swap_remove vs IdMapIndex for Deletions in Turbovec](/RyanCodrai/turbovec/when-should-i-use-swap_remove-vs-idmapindex-for-deletions-in-turbovec)

Understand when to use swap_remove vs IdMapIndex for deletions in turbovec. Choose swap_remove for self managed indices, or IdMapIndex for stable external IDs.

- Tags: best-practices
- Published: 2026-06-16

### [How NEON and AVX-512 SIMD Kernels Accelerate Search in Turbovec](/RyanCodrai/turbovec/how-do-neon-arm-and-avx-512-x86-simd-kernels-accelerate-search-in-turbovec)

Discover how NEON and AVX-512 SIMD kernels accelerate search in Turbovec with parallel nibble extraction for 10-20x speedups. Learn more about optimizing vector search.

- Tags: performance
- Published: 2026-06-16

### [How Lazy Dimension Initialization Works in TurboQuantIndex](/RyanCodrai/turbovec/how-does-lazy-dimension-initialization-work-in-turboquantindex)

Understand lazy dimension initialization in TurboQuantIndex. Learn how dimensions are set on first add, ensuring consistency for efficient data handling in your projects.

- Tags: internals
- Published: 2026-06-16

### [TQ+ Calibration in TurboVec: When and How It Improves Recall](/RyanCodrai/turbovec/what-is-tq-plus-calibration-and-when-does-it-improve-recall-in-turbovec)

Discover TQ+ calibration in turbovec. Learn how this vector transformation aligns distributions to significantly boost recall, especially with large, non-isotropic datasets.

- Tags: deep-dive
- Published: 2026-06-16

### [How TurboQuant Achieves Better Recall Than FAISS IndexPQ: A Technical Deep Dive](/RyanCodrai/turbovec/how-does-turboquant-achieve-better-recall-than-faiss-indexpq)

Discover how TurboQuant outperforms FAISS IndexPQ with two-level quantization, residual refinement, and SIMD-optimized distance computation. Achieve superior recall efficiently.

- Tags: deep-dive
- Published: 2026-06-16

### [Why 4-Bit Quantization Offers Better Performance Consistency Than 2-Bit in TurboVect](/RyanCodrai/turbovec/why-does-4-bit-quantization-offer-better-performance-consistency-than-2-bit)

Discover why 4-bit quantization in TurboVect provides superior performance consistency over 2-bit. Learn how improved instruction scheduling and reduced memory latency enhance stability across architectures.

- Tags: performance
- Published: 2026-06-10

### [How to Migrate FAISS IndexPQ to turbovec: A Complete Migration Guide](/RyanCodrai/turbovec/how-to-migrate-faiss-indexpq-to-turbovec)

Migrate FAISS IndexPQ to turbovec effortlessly. Eliminate training, double compression to 16x, add ID filtering, and keep the familiar API. Get faster, smaller indexes.

- Tags: migration-guide
- Published: 2026-06-10

### [How Random Orthogonal Rotation Stabilizes Coordinate Distributions in Turbovec](/RyanCodrai/turbovec/how-does-random-orthogonal-rotation-stabilize-coordinate-distributions)

Learn how random orthogonal rotation stabilizes coordinate distributions in Turbovec. Discover its role in preserving L2 norms and ensuring consistent statistical properties for quantization.

- Tags: internals
- Published: 2026-06-10

### [How the TQ+ Calibration Trailer Enables Online Calibration in Turbovec](/RyanCodrai/turbovec/how-does-the-tq-plus-calibration-trailer-enable-online-calibration)

Discover how the TQ+ calibration trailer enables online calibration in Turbovec by storing parameters for immediate reuse, eliminating offline retraining and optimizing workflow.

- Tags: internals
- Published: 2026-06-10

### [Turbovec .tv and .tvim File Format: Binary Layout Specification](/RyanCodrai/turbovec/what-is-the-file-format-of-tv-and-tvim-files)

Explore the Turbovec .tv and .tvim file format binary layout. Understand how Turbovec stores quantized vector indexes with packed codes, TQ+ calibration, and slot-to-id lookup.

- Tags: api-reference
- Published: 2026-06-10

### [Lazy Index Initialization in Turbovec: How It Defers Dimension Commitment and Cache Generation](/RyanCodrai/turbovec/what-is-lazy-index-initialization-and-when-is-it-useful)

Learn about lazy index initialization in Turbovec. Defer dimension validation and cache generation until needed, saving resources and speeding up instantiation.

- Tags: internals
- Published: 2026-06-10

### [How the Allowlist Filter Prevents Wasted Computation in Turbovec](/RyanCodrai/turbovec/how-does-the-allowlist-filter-work-to-prevent-wasted-computation)

Discover how Turbovec's allowlist filter prevents wasted computation. It efficiently skips vector blocks, saving valuable processing time and resources.

- Tags: internals
- Published: 2026-06-10

### [TurboVec Vector Deletion: How Swap-Remove Enables Fast O(1) Removal Without Preserving Order](/RyanCodrai/turbovec/vector-deletion-and-order-preservation)

Discover how TurboVec achieves O(1) vector deletion using swap-remove, sacrificing order for speed. Learn about its efficient internal id table updates.

- Tags: internals
- Published: 2026-06-09

### [TurboVec prepare() for Search Cache Warmup: Eager Cache Materialization Explained](/RyanCodrai/turbovec/prepare-warmup-for-search-caches)

Learn how TurboVec's prepare() function speeds up search by eagerly materializing lazy caches, eliminating latency spikes for initial queries.

- Tags: performance
- Published: 2026-06-09

### [How to Estimate the Index Size for Large Datasets Using TurboVec](/RyanCodrai/turbovec/estimating-index-size-for-large-datasets)

Learn how to estimate TurboVec index size for large datasets. Calculate storage costs with vectors, dimensions, bit width, and metadata overhead for efficient memory management.

- Tags: tutorial
- Published: 2026-06-09

### [What is Lloyd-Max Scalar Quantization and How TurboVec Uses It](/RyanCodrai/turbovec/lloyd-max-scalar-quantization)

Discover Lloyd-Max scalar quantization's optimal compression in TurboVec. Learn how this algorithm minimizes error for efficient codebooks.

- Tags: deep-dive
- Published: 2026-06-09

### [TQ+ Per-Coordinate Calibration in TurboVec: Implementation and Usage](/RyanCodrai/turbovec/tq-plus-per-coordinate-calibration)

Discover how TQ+ per-coordinate calibration in TurboVec corrects anisotropic data distributions by fitting shift and scale parameters to each rotated coordinate. Learn the implementation and usage.

- Tags: deep-dive
- Published: 2026-06-09

### [TurboVec swap_remove: O(1) Vector Deletion with Swap-and-Pop Semantics](/RyanCodrai/turbovec/swap-remove-behavior-for-deletions)

Discover TurboVec swap_remove for O(1) vector deletion. Learn how it efficiently removes vectors by swapping with the last element and invalidates the cache for faster lookups.

- Tags: internals
- Published: 2026-06-09

### [Expected Memory Compression Ratios with TurboVec: 2‑Bit vs 4‑Bit Quantization](/RyanCodrai/turbovec/memory-compression-ratio-calculations)

Discover TurboVec memory compression ratios. Achieve up to 16x with 2-bit and 8x with 4-bit quantization for significant memory reduction.

- Tags: performance
- Published: 2026-06-09

### [What Causes the x86 2‑Bit Multi‑Threaded Search Performance Gap in Turbovec?](/RyanCodrai/turbovec/what-causes-performance-gap-on-x86-2-bit-multi-threaded-search)

Understand the x86 2-bit multi-threaded search performance gap in TurboVec. Learn how bit-unpacking, memory saturation, and mutex contention cause the slowdown.

- Tags: performance
- Published: 2026-06-08

### [How TurboVec Uses the Lloyd-Max Algorithm for Optimal Vector Quantization](/RyanCodrai/turbovec/what-is-the-lloyd-max-algorithm-and-its-use-in-quantization)

Discover how TurboVec leverages the Lloyd-Max algorithm for optimal vector quantization. Achieve high-compression, low-distortion encoding with this powerful technique.

- Tags: deep-dive
- Published: 2026-06-08

### [Memory Overhead of the slot_to_id Map in IdMapIndex: Per-Vector Cost Breakdown](/RyanCodrai/turbovec/what-is-the-memory-overhead-of-the-slot_to_id-map)

Discover the precise memory overhead of the slot_to_id map in IdMapIndex. Understand the 8-byte per-vector cost and 32-byte total bidirectional mapping overhead in turbovec.

- Tags: performance
- Published: 2026-06-08

### [How to Migrate from FAISS IndexPQ to turbovec: A Complete Guide](/RyanCodrai/turbovec/how-to-migrate-from-faiss-indexpq-to-turbovec)

Easily migrate from FAISS IndexPQ to turbovec. Learn how to replace FAISS with TurboQuantIndex or IdMapIndex, skip training, and leverage built-in filtering for higher compression and performance.

- Tags: migration-guide
- Published: 2026-06-08

### [Performance Difference Between 2-Bit and 4-Bit Compression in TurboVec](/RyanCodrai/turbovec/what-is-the-performance-difference-between-2-bit-and-4-bit-compression)

Discover the performance difference between 2-bit and 4-bit compression in TurboVec. Learn how 2-bit offers faster vector search with minimal recall loss for your applications.

- Tags: performance
- Published: 2026-06-08

### [How Lazy Index Initialization Works in Turbovec (and When to Use It)](/RyanCodrai/turbovec/how-does-lazy-index-initialization-work-and-when-to-use-it)

Discover lazy index initialization in Turbovec. Defer vector dimension locking until first add_2d and build heavy caches only when needed for efficient vector index creation.

- Tags: internals
- Published: 2026-06-08

### [Turbovec SIMD Intrinsics for ARM vs x86 Search: NEON, AVX2, and AVX-512BW Kernels Explained](/RyanCodrai/turbovec/what-simd-intrinsics-does-turbovec-use-for-arm-vs-x86-search)

Explore Turbovec SIMD intrinsics: ARM NEON versus x86 AVX2 and AVX-512BW. Understand how these kernels accelerate quantized vector similarity search using lookup-table-based dot products.

- Tags: deep-dive
- Published: 2026-06-08

### [Structure of the .tv File Format: Binary Layout and Data Organization](/RyanCodrai/turbovec/what-is-the-tv-file-format-structure-and-data-layout)

Explore the .tv file format structure. Learn about its 14-byte header, metadata, quantization codes, scale factors, and TQ+ calibration trailer, all organized in a binary layout using little-endian byte order.

- Tags: internals
- Published: 2026-06-08

### [How Length-Renormalized Scoring Removes Quantization Bias in TurboVec Vector Search](/RyanCodrai/turbovec/how-does-length-renormalized-scoring-remove-quantization-bias-in-vector-search)

Discover how length-renormalized scoring eliminates quantization bias in TurboVec vector search. Learn how scale factors correct the systematic shrinkage caused by scalar quantization, enhancing search accuracy.

- Tags: deep-dive
- Published: 2026-06-08

### [TQ+ Per-Coordinate Calibration in turbovec: How It Improves Recall](/RyanCodrai/turbovec/what-is-tq-per-coordinate-calibration-and-how-does-it-improve-recall)

Discover TQ+ per-coordinate calibration in turbovec. Learn how this technique precisely aligns embeddings to eliminate quantizer mis-fit and boost recall by up to 1.4 percentage points at R@1.

- Tags: deep-dive
- Published: 2026-06-08