VelesDB SQ8 vs Binary Vector Quantization: Trade-offs and Implementation Guide

VelesDB's Binary quantization achieves 32× compression with ~90-95% recall, while SQ8 offers 4× compression with ~99% recall, making the choice dependent on whether memory constraints or accuracy is the priority.

VelesDB is an open-source vector database that provides two lossy compression schemes for high-dimensional vectors: SQ8 (8-bit scalar quantization) and Binary (1-bit quantization). Both methods drastically reduce memory footprint compared to full-precision f32 storage, but they differ significantly in compression ratio, recall accuracy, and computational overhead according to the cyberlife-coder/velesdb source code.

Memory and Compression Trade-offs

SQ8 Compression Ratio

SQ8 reduces vector storage to 4× smaller than full-precision vectors. Each dimension consumes 1 byte, plus an additional 8 bytes per vector to store the min and scale parameters required for de-quantization. This overhead is negligible compared to the overall payload reduction.

As defined in crates/velesdb-wasm/src/lib.rs, the StorageMode::SQ8 variant triggers this compression path during store initialization.

Binary Compression Ratio

Binary quantization achieves 32× compression, storing only 1 bit per dimension. This represents the maximum memory efficiency available in VelesDB, making it suitable for edge devices, mobile applications, and WASM environments where memory is the primary bottleneck.

The StorageMode::Binary enum variant activates this mode, as implemented in the same lib.rs file.

Recall Accuracy Comparison

SQ8 Recall Performance

SQ8 maintains approximately 99% recall, with a typical loss of only ~1% compared to full-precision search. The 256 discrete levels per dimension preserve coarse magnitude information through the min and scale parameters, minimizing information loss during the quantization process.

According to crates/velesdb-wasm/src/lib.rs (lines 108-109), this recall rate is the documented expectation for production deployments.

Binary Recall Performance

Binary quantization exhibits a 5-10% recall loss, typically achieving 90-95% recall. By discarding magnitude entirely and retaining only the sign of each component, this method loses significant discriminative power. However, for many approximate nearest neighbor applications, this trade-off is acceptable when memory constraints are severe.

The recall characteristics are documented in crates/velesdb-wasm/src/lib.rs (lines 108-110).

Runtime Performance and Computation

SQ8 De-quantization Overhead

SQ8 requires on-the-fly de-quantization before distance calculation. The engine applies the formula value = (u8 / scale) + min to each dimension, as implemented in crates/velesdb-wasm/src/vector_ops.rs (lines 67-78).

Despite this overhead, SQ8 queries remain faster than full-precision because the compressed data fits efficiently in CPU cache. The crates/velesdb-core/src/quantization/scalar.rs file contains SIMD-optimized dot-product and Euclidean distance functions that accelerate these operations.

Binary Distance Calculation

Binary quantization uses Hamming distance calculations on packed bits, which is computationally inexpensive. However, the engine must first expand packed bits to 0/1 floats at query time before invoking the standard metric routine, as seen in crates/velesdb-wasm/src/vector_ops.rs (lines 94-112).

While the unpacking loop adds some overhead, Binary is typically the fastest option for very large collections where cache-fit advantages dominate computational costs.

Implementation Examples

Rust Implementation

The VectorStore::new_with_mode constructor in crates/velesdb-wasm/src/lib.rs (lines 31-40) accepts a StorageMode enum to select the compression scheme:

use velesdb_wasm::{VectorStore, StorageMode};

// 768-dimensional vectors with cosine similarity using SQ8
let mut store_sq8 = VectorStore::new_with_mode(768, "cosine", StorageMode::SQ8)
    .expect("SQ8 store creation failed");

// Insert full-precision f32 vector
let vec = vec![0.1_f32; 768];
store_sq8.insert(&vec, None).unwrap();

// Binary mode for maximum compression
let mut store_bin = VectorStore::new_with_mode(768, "cosine", StorageMode::Binary)
    .expect("Binary store creation failed");
store_bin.insert(&vec, None).unwrap();

Python Implementation

The Python wrapper parses storage mode strings via parse_storage_mode in crates/velesdb-python/src/utils.rs (lines 63-66):

import velesdb as vb

# SQ8 quantization

store = vb.VectorStore(dimension=768, metric="cosine", storage_mode="sq8")
store.insert([0.1] * 768)

# Binary quantization

store_bin = vb.VectorStore(dimension=768, metric="cosine", storage_mode="binary")
store_bin.insert([0.1] * 768)

Querying Vectors

Both modes use the same search interface, with the engine automatically selecting the appropriate scoring routine from crates/velesdb-wasm/src/vector_ops.rs (lines 25-35):

let query = vec![0.2_f32; 768];

// SQ8 search with automatic de-quantization
let results_sq8 = store_sq8.search(&query, 10).unwrap();

// Binary search with bit unpacking
let results_bin = store_bin.search(&query, 10).unwrap();

Summary

• SQ8 provides 4× compression with ~99% recall, making it ideal for semantic search and RAG pipelines where accuracy is critical.
• Binary delivers 32× compression with 90-95% recall, suited for memory-constrained environments like edge devices, mobile apps, and WASM deployments.
• Runtime performance favors Binary for very large collections due to cache efficiency, while SQ8 offers better accuracy with modest computational overhead.
• Implementation is straightforward via StorageMode::SQ8 or StorageMode::Binary in Rust, or storage_mode="sq8" / storage_mode="binary" in Python.

Frequently Asked Questions

What is the exact memory footprint difference between SQ8 and Binary in VelesDB?

SQ8 reduces memory usage by 4× compared to full-precision f32 vectors, consuming 1 byte per dimension plus 8 bytes per vector for min/scale metadata. Binary quantization achieves 32× compression, using only 1 bit per dimension. For a 768-dimensional vector, SQ8 requires approximately 776 bytes while Binary requires only 96 bytes (plus negligible overhead).

How much recall accuracy should I expect to lose with each quantization method?

According to the VelesDB source code in crates/velesdb-wasm/src/lib.rs, SQ8 typically maintains approximately 99% recall (1% loss), while Binary quantization achieves 90-95% recall (5-10% loss). The exact impact depends on vector dimensionality and data distribution, with higher dimensions generally tolerating Binary quantization better.

Which quantization mode should I choose for a WebAssembly deployment?

For WASM environments, Binary quantization is typically preferred because it minimizes memory footprint—critical for browser-based applications with limited heap space. However, if your use case requires high-precision semantic search (e.g., RAG applications), SQ8 offers a better accuracy/compression balance while still fitting within typical WASM memory constraints.

Can I switch between SQ8 and Binary quantization after creating a VelesDB store?

No, the storage mode is fixed at initialization when calling VectorStore::new_with_mode or the Python equivalent. The StorageMode enum (defined in crates/velesdb-wasm/src/lib.rs) determines the internal vector representation at creation time. To change quantization schemes, you must create a new store with the desired mode and re-insert your vectors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}