How SIMD Kernel Optimizations Work for NEON and AVX-512 in Turbovec

Turbovec achieves high-throughput vector search by using hand-written SIMD kernels that compute 4-bit-quantized inner-product scores in parallel, with architecture-specific implementations in turbovec/src/search.rs for ARM64 NEON and x86_64 AVX-2/AVX-512 that produce bit-identical results through nibble-wise table lookups and fused multiply-add accumulation.

RyanCodrai/turbovec is a Rust-based vector search library optimized for low-precision quantization. The library implements specialized SIMD kernels that decode 4-bit compressed vectors and compute similarity scores using processor-specific intrinsic functions, enabling single-instruction-multiple-data parallelism on both ARM and x86_64 platforms.

NEON Kernel Implementation on ARM64

The ARM64 implementation resides in the score_4bit_block_neon function within turbovec/src/search.rs (lines 46-59). This kernel processes vectors in blocks of 32, using NEON's 128-bit registers to perform parallel table lookups and accumulation.

Nibble-wise Table Lookup with vqtbl1q_u8

The kernel loads 16-byte blocks of encoded codes using vld1q_u8. Because 4-bit quantization packs two values per byte, the kernel splits each byte into high and low nibbles. The vqtbl1q_u8 intrinsic performs a parallel table lookup, treating the nibble values as indices into 16-entry lookup tables (LUTs) stored in uint8x16_t registers. This converts compressed 4-bit indices into 8-bit distance values in a single instruction.

Accumulation and Widening

The kernel processes four groups of 32 vectors in an unrolled inner loop, accumulating partial sums in uint16x8 registers to prevent overflow. After every FLUSH_EVERY groups, the 16-bit accumulators are widened to 32-bit integers using vaddw_u8, converted to floating-point with vcvtq_f32_u32, and scaled using vfmaq_f32 (fused multiply-add). The accumulator is initialized with a per-query bias, allowing the first flush to compute bias + scale × partial in one operation.

AVX-2 and AVX-512 Implementation on x86_64

The x86_64 implementation provides two entry points in turbovec/src/search.rs: search_multi_query_avx2 (lines 165-170) and search_multi_query_avx512bw. These kernels process four queries simultaneously (NQ=4) using 256-bit or 512-bit vectors.

Shuffle-based LUT Lookup

The AVX kernels load encoded codes using _mm256_loadu_si256. A bit mask (_mm256_set1_epi8(0x0F)) isolates the low-order nibbles via _mm256_and_si256, while a right shift isolates the high-order nibbles. The _mm256_shuffle_epi8 instruction performs the table lookup operation, functionally equivalent to NEON's vqtbl1q_u8, broadcasting the nibble values across the vector lanes to produce distance values.

Fused Multiply-Add Accumulation

Four 256-bit integer accumulators maintain running sums for the four concurrent queries. When the FLUSH_EVERY threshold is reached, the code converts integer accumulators to float32 using _mm256_cvtepi32_ps, applies the per-query scale with _mm256_mul_ps, and accumulates into the bias-initialized floating-point accumulator using _mm256_fmadd_ps. This fused operation matches the NEON implementation's numerical precision.

Core Design Patterns Across Architectures

Both kernel families share five critical optimization strategies that ensure consistent performance and accuracy.

Nibble-wise Table Lookup

Both architectures leverage hardware-accelerated permute instructions to decode 4-bit quantized values. NEON uses vqtbl1q_u8 while AVX uses _mm256_shuffle_epi8, each performing 16 simultaneous LUT lookups to decompress the compressed representation.

Batch-wise Flush Strategy

Accumulating in 16-bit integers (uint16x8 on NEON, __m256i on AVX) prevents overflow during the inner loop. The FLUSH_EVERY parameter controls how many vector groups accumulate before conversion to floating-point, balancing precision with register pressure.

Per-Query Bias Seeding

Floating-point accumulators initialize with the query-specific bias value. This allows the first accumulation to use a single fused multiply-add operation (vfmaq_f32 or _mm256_fmadd_ps) rather than requiring a separate addition instruction later in the pipeline.

Unrolled Inner Loops

The NEON kernel processes four groups of 32 vectors per iteration, while the AVX kernel handles four queries with 256-bit vectors. This unrolling minimizes loop control overhead and maximizes instruction-level parallelism by keeping the load/store units busy while arithmetic units process previous loads.

Top-K Heap Integration

Rather than materializing the full score array, both kernels call architecture-specific flush handlers (avx2_batch_flush_to_fa for AVX, equivalent NEON routines) after each block. These functions update min-heaps tracking the k-nearest neighbors, significantly reducing memory bandwidth requirements during large-scale searches.

Practical Usage and Compilation

The library automatically dispatches to the appropriate kernel based on runtime CPU feature detection. Users interact with high-level functions that abstract the SIMD implementation details.

use turbovec::search::search_multi_query;
use turbovec::codebook::Codebook;
use turbovec::encode::encode_vectors;

// Train a 4-bit codebook with 128 dimensions
let codebook = Codebook::train(&training_vectors, 4, 128)?;

// Encode dataset to compact 4-bit representation
let encoded = encode_vectors(&codebook, &dataset_vectors)?;

// Prepare batch of 4 queries
let queries = vec![query1, query2, query3, query4];
let query_codes = encode_vectors(&codebook, &queries)?;

// Search for k=10 nearest neighbors
let (scores, ids) = search_multi_query(
    &encoded,
    &codebook,
    &query_codes,
    10,
    None, // No slot mask
)?;

To force NEON code generation for ARM testing:

cargo build --release --target aarch64-unknown-linux-gnu

To compile with native optimizations for x86_64:

RUSTFLAGS="-C target-cpu=native" cargo build --release

The repository includes a dedicated test utility in turbovec/examples/kernel_xtest.rs that invokes the raw kernels directly and verifies that NEON and AVX implementations produce identical scores on the same input data.

Summary

• Turbovec implements hand-optimized SIMD kernels in turbovec/src/search.rs for both NEON (score_4bit_block_neon) and AVX-2/AVX-512 (search_multi_query_avx2, search_multi_query_avx512bw).
• Both architectures use nibble-wise table lookups (vqtbl1q_u8 on NEON, _mm256_shuffle_epi8 on AVX) to decode 4-bit quantized vectors in parallel.
• Accumulation occurs in 16-bit integers with periodic batch flushing (FLUSH_EVERY) to prevent overflow before converting to scaled floating-point scores.
• Per-query bias seeding enables efficient fused multiply-add operations during the first batch flush.
• The kernels integrate directly with min-heap routines to update top-K results without materializing full score arrays, minimizing memory traffic.

Frequently Asked Questions

How does 4-bit quantization work in Turbovec?

Turbovec compresses high-dimensional floating-point vectors by training a codebook that represents each subspace with 16 centroids (4 bits). During encoding, each vector component is replaced by the index of its nearest centroid. The SIMD kernels decode these indices on-the-fly using lookup tables stored in SIMD registers, allowing distance computation without full decompression.

Why do NEON and AVX kernels produce identical results?

Both kernels implement the same mathematical pipeline: nibble extraction, table lookup, 16-bit integer accumulation, and scaled floating-point conversion. The FLUSH_EVERY parameter and bias initialization are synchronized across architectures, and both use IEEE-754 single-precision arithmetic for the final scaling steps, ensuring bit-identical outputs on identical inputs.

When does Turbovec use AVX-512 instead of AVX-2?

The library selects search_multi_query_avx512bw when the CPU reports support for AVX-512 BW (Byte and Word) instructions. AVX-512 provides wider 512-bit registers and additional mask registers for handling the top-K heap updates, though the AVX-2 kernel remains highly competitive for smaller batch sizes due to its efficient 256-bit shuffle operations.

Where can I verify the kernel implementations?

The turbovec/src/search.rs file contains the primary kernel implementations, while turbovec/tests/kernel_correctness.rs provides unit tests that cross-validate NEON and AVX outputs. Developers can run turbovec/examples/kernel_xtest.rs to execute the kernels against synthetic data and verify timing and correctness on their specific hardware.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}