# Turbovec SIMD Intrinsics for ARM vs x86 Search: NEON, AVX2, and AVX-512BW Kernels Explained

> Explore Turbovec SIMD intrinsics: ARM NEON versus x86 AVX2 and AVX-512BW. Understand how these kernels accelerate quantized vector similarity search using lookup-table-based dot products.

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: deep-dive
- Published: 2026-06-08

---

**Turbovec leverages ARM NEON intrinsics including `vqtbl1q_u8` and `vfmaq_f32` for 128-bit aarch64 operations, while x86 platforms utilize AVX2 (`_mm256_shuffle_epi8`, `_mm256_fmadd_ps`) or AVX-512BW (`_mm512_shuffle_epi8`, `_mm512_inserti64x4`) to accelerate 4-bit quantized vector similarity search through lookup-table-based dot products.**

The `RyanCodrai/turbovec` repository implements high-throughput approximate nearest neighbor search via hand-optimized SIMD kernels written in Rust. By utilizing `std::arch` intrinsics, turbovec achieves bit-identical scoring across ARM and x86 architectures through a 4-bit quantization scheme that relies on nibble-splitting and lookup table (LUT) accumulation. Understanding these turbovec SIMD intrinsics for ARM vs x86 search reveals how the library maximizes throughput on each platform while maintaining identical numerical results.

## ARM NEON Intrinsics (aarch64)

On ARM 64-bit targets, turbovec implements the `score_4bit_block_neon` function in [`turbovec/src/search.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/search.rs) using 128-bit NEON registers. This kernel processes 32-byte code blocks by splitting each byte into high and low 4-bit nibbles, using table lookup instructions that are uniquely efficient on ARM.

The NEON implementation relies on the following key intrinsics:

- **`vdupq_n_u8`** / **`vdupq_n_f32`** – Broadcast scalar values (such as the `0x0F` nibble mask and per-query scales) across all lanes of a 128-bit vector.
- **`vld1q_u8`** / **`vld1q_f32`** – Load 16-byte or 16-float chunks from the blocked codes and lookup tables.
- **`vqtbl1q_u8`** – Perform byte-wise table lookups in a single instruction, indexing the 16-byte LUT with the 4-bit nibbles extracted from the codes.
- **`vaddw_u8`** – Widen and add 8-bit values into 16-bit accumulators to prevent overflow during dot product accumulation.
- **`vfmaq_f32`** – Execute fused multiply-add operations when converting final 16-bit integer accumulators to 32-bit floats and applying per-vector scaling factors.

This approach loads 32-byte code blocks, splits nibbles using `vandq_u8` and `vshrq_n_u8`, accumulates via `vaddq_u8` and `vaddw_u8`, then finishes with `vfmaq_f32` for the final scaling step.

```rust
#[cfg(target_arch = "aarch64")]
unsafe fn score_4bit_block_neon(
    blocked_codes: &[u8],
    uint8_luts:   &[u8],
    /* … other args … */
) {
    use std::arch::aarch64::*;

    let mask = vdupq_n_u8(0x0F);
    let v_scale = vdupq_n_f32(scale);

    for batch in 0..n_batches {
        // Load LUTs and code bytes (32 bytes each)
        let lut_hi = vld1q_u8(lp);
        let lut_lo = vld1q_u8(lp.add(16));
        let c0 = vld1q_u8(cp);
        let c1 = vld1q_u8(cp.add(16));

        // Table-lookup + nibble split
        let s0 = vaddq_u8(vqtbl1q_u8(lut_lo, vandq_u8(c0, mask)),
                         vqtbl1q_u8(lut_hi, vshrq_n_u8(c0, 4)));
        let lo = vcvtq_f32_u32(vmovl_u16(vget_low_u16(acc[i])));
        fa[i*2] = vfmaq_f32(fa[i*2], v_scale, lo);
    }
}

```

## x86 AVX2 Intrinsics (x86_64)

For x86 platforms without AVX-512 support, turbovec provides the `search_multi_query_avx2` function in [`turbovec/src/search.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/search.rs). This kernel processes 256-bit vectors (32 bytes) per iteration, using AVX2's powerful byte-shuffle capabilities to perform the LUT lookups that ARM handles with `vqtbl1q_u8`.

Critical AVX2 intrinsics include:

- **`_mm256_set1_epi8`** – Broadcast the `0x0F` mask across all 32 lanes of a 256-bit vector.
- **`_mm256_loadu_si256`** – Load unaligned 256-bit chunks of packed codes and lookup tables.
- **`_mm256_and_si256`** and **`_mm256_srli_epi16`** – Extract low and high nibbles by masking and shifting.
- **`_mm256_shuffle_epi8`** – Perform 32 parallel byte lookups into the LUT using the extracted nibbles as indices.
- **`_mm256_add_epi16`** – Accumulate partial dot products into 16-bit integer vectors.
- **`_mm256_cvtepu16_epi32`** and **`_mm256_cvtepi32_ps`** – Widen 16-bit integers to 32-bit floats.
- **`_mm256_fmadd_ps`** – Apply per-query scaling factors using fused multiply-add operations.

The AVX2 kernel follows the same nibble-splitting logic as NEON but processes twice the data width per instruction.

```rust
#[cfg(target_arch = "x86_64")]
#[target_feature(enable = "avx2,fma")]
unsafe fn search_multi_query_avx2(/* … */) {
    use std::arch::x86_64::*;

    let nibble_mask = _mm256_set1_epi8(0x0F);
    for batch in 0..n_batches {
        let mut accus = [[_mm256_setzero_si256(); 4]; 4];
        for g in g_start..g_end {
            let cp = codes_base.add((b * n_byte_groups + g) * BLOCK);
            let codes_v = _mm256_loadu_si256(cp as *const __m256i);
            let clo = _mm256_and_si256(codes_v, nibble_mask);
            let chi = _mm256_and_si256(_mm256_srli_epi16(codes_v, 4), nibble_mask);

            for qi in 0..4 {
                let lut = _mm256_loadu_si256(luts[qi].as_ptr().add(g * 32) as *const __m256i);
                let res0 = _mm256_shuffle_epi8(lut, clo);
                let res1 = _mm256_shuffle_epi8(lut, chi);
                accus[qi][0] = _mm256_add_epi16(accus[qi][0], res0);
            }
        }
        let f0 = _mm256_cvtepi32_ps(_mm256_cvtepu16_epi32(_mm256_castsi256_si128(dis0)));
        fa[qi][0] = _mm256_fmadd_ps(v_scales[qi], f0, fa[qi][0]);
    }
}

```

## x86 AVX-512BW Intrinsics (x86_64)

The `search_multi_query_avx512bw` function in [`turbovec/src/search.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/search.rs) targets AVX-512BW-capable processors, processing 512-bit vectors by fusing two 256-bit blocks into a single register. This approach halves the number of loop iterations compared to the AVX2 implementation while maintaining identical numerical precision.

Key AVX-512BW intrinsics include:

- **`_mm512_inserti64x4`** and **`_mm512_castsi256_si512`** – Combine two 256-bit code blocks into one 512-bit register (`zmm`).
- **`_mm512_set1_epi8`** – Broadcast the nibble mask across 64 lanes.
- **`_mm512_and_si512`** and **`_mm512_srli_epi16`** – Extract low and high nibbles at 512-bit width.
- **`_mm512_shuffle_epi8`** – Perform 64 parallel byte lookups (AVX-512BW extends the AVX2 shuffle to 512 bits).
- **`_mm512_broadcast_i64x4`** – Broadcast 256-bit LUT data into 512-bit registers for the shuffle operation.
- **`_mm512_add_epi16`** – Accumulate into 512-bit vectors.
- **`_mm512_extracti64x4_epi64`** – Split the 512-bit accumulator back into 256-bit halves for final conversion using AVX2 helpers.

This kernel utilizes the wider vector width to process paired blocks simultaneously, significantly improving instructions-per-cycle throughput on server-grade x86 hardware.

```rust
#[cfg(target_arch = "x86_64")]
#[target_feature(enable = "avx2,fma,avx512f,avx512bw")]
unsafe fn search_multi_query_avx512bw(/* … */) {
    use std::arch::x86_64::*;

    let mask512 = _mm512_set1_epi8(0x0F);
    for batch in 0..n_batches {
        let mut accus = [[_mm512_setzero_si512(); 4]; 4];
        for gp in gp_start..gp_end {
            let codes_a = _mm512_inserti64x4(
                _mm512_castsi256_si512(_mm256_loadu_si256(cp0_a as *const __m256i)),
                _mm256_loadu_si256(cp1_a as *const __m256i),
                1,
            );

            let clo = _mm512_and_si512(codes_a, mask512);
            let chi = _mm512_and_si512(_mm512_srli_epi16(codes_a, 4), mask512);

            for qi in 0..4 {
                let lut_a = _mm512_broadcast_i64x4(
                    _mm256_loadu_si256(luts[qi].as_ptr().add(g0 * 32) as *const __m256i)
                );
                let res0_a = _mm512_shuffle_epi8(lut_a, clo);
                accus[qi][0] = _mm512_add_epi16(accus[qi][0], res0_a);
            }
        }
        let low_half = _mm512_extracti64x4_epi64(accus[0][0], 0);
        avx2_batch_flush_to_fa(low_half, &mut fa[qi]);
    }
}

```

## Core Data Flow and Supporting Files

The SIMD kernels consume a specific data layout defined in [`turbovec/src/pack.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/pack.rs), which implements a FAISS-style perm0-interleaved block format required for efficient LUT-based scoring. Query-specific lookup tables are constructed in [`turbovec/src/encode.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/encode.rs) via the `build_query_neon_lut_from_slice` function, preparing the 16-byte tables that both ARM and x86 kernels index during search.

Turbovec selects the appropriate kernel at compile time using `#[cfg(target_arch = "aarch64")]` for ARM and `#[cfg(target_arch = "x86_64")]` for x86, with runtime feature detection (`is_x86_feature_detected!`) choosing between AVX2 and AVX-512BW paths on x86 platforms.

## Summary

- **ARM NEON** uses 128-bit vectors with `vqtbl1q_u8` for table lookups and `vfmaq_f32` for fused multiply-add scaling in the `score_4bit_block_neon` function.
- **x86 AVX2** processes 256-bit vectors using `_mm256_shuffle_epi8` for parallel byte lookups and `_mm256_fmadd_ps` for final scaling in `search_multi_query_avx2`.
- **x86 AVX-512BW** doubles throughput via 512-bit operations in `search_multi_query_avx512bw`, utilizing `_mm512_inserti64x4` to process paired blocks and `_mm512_shuffle_epi8` for 64-lane parallel lookups.
- All three kernels implement identical 4-bit nibble-splitting logic and accumulate into 16-bit integers before converting to 32-bit floats, ensuring bit-identical results across architectures.
- The implementation resides primarily in [`turbovec/src/search.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/search.rs), with data layout support in [`turbovec/src/pack.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/pack.rs) and LUT generation in [`turbovec/src/encode.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/encode.rs).

## Frequently Asked Questions

### Why does turbovec use different intrinsics for ARM vs x86 search?

**ARM NEON and x86 AVX2/AVX-512BW offer fundamentally different instruction sets for parallel table lookups.** NEON provides `vqtbl1q_u8` for 16-byte table lookups natively, while AVX2 relies on `_mm256_shuffle_epi8` with careful nibble extraction. The library maps the same 4-bit quantization algorithm to each architecture's most efficient instructions rather than forcing a common denominator approach, maximizing throughput on both platforms.

### What is the performance difference between AVX2 and AVX-512BW in turbovec?

**The AVX-512BW kernel processes two 256-bit blocks simultaneously within 512-bit registers, reducing loop iteration count by half compared to AVX2.** By using `_mm512_inserti64x4` to pair blocks and `_mm512_shuffle_epi8` to perform 64 parallel byte lookups, the AVX-512BW implementation achieves higher instructions-per-cycle throughput on compatible hardware, though the actual speedup depends on CPU-specific frequency scaling and thermal constraints.

### How does turbovec maintain bit-identical results across ARM and x86?

**All three kernels follow the same numerical pipeline:** split 4-bit nibbles, index into query-specific LUTs using byte-wise shuffles, accumulate into 16-bit integers to prevent overflow, then convert to 32-bit floats and apply fused multiply-add scaling. By using fixed-point integer accumulation until the final scaling step—implemented via `vfmaq_f32` on ARM and `_mm256_fmadd_ps` or `_mm512_fmadd_ps` on x86—turbovec ensures deterministic, architecture-independent similarity scores.

### Which source file contains the main search implementations?

**[`turbovec/src/search.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/search.rs)** contains all three SIMD kernels: `score_4bit_block_neon` for ARM, `search_multi_query_avx2` for 256-bit x86, and `search_multi_query_avx512bw` for 512-bit x86. Supporting logic for LUT construction resides in [`turbovec/src/encode.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/encode.rs), while the required FAISS-style block interleaving format is defined in [`turbovec/src/pack.rs`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec/src/pack.rs).