How to Perform Hybrid Retrieval with Filtered Search in Turbovec Using an Allowlist

Turbovec doesn't natively support hybrid keyword-vector search, but you can emulate hybrid retrieval by resolving metadata filters to an allowlist of internal handles and running pure vector search against that constrained set.

Turbovec is a high-performance vector database for Python, and its core engine (as implemented in TurboQuantVectorDb) is optimized for pure vector similarity. While there is no built-in hybrid retrieval mode, the library exposes a powerful filter-allowlist mechanism that lets you combine metadata filtering with vector search for practical hybrid-like workflows. Let's explore exactly how this works, including the underlying source logic and code examples you can run today.

Understanding Turbovec's Hybrid Retrieval Architecture

The first thing to know is that Turbovec explicitly rejects SearchType.hybrid — the constructor guard in TurboQuantVectorDb raises a clear error if you attempt to use it. This design choice means hybrid retrieval isn't a slot you plug into; it's a pattern you build on top of the core engine.

The workaround relies on a filter-allowlist pipeline that powers all high-level integrations in the repository:

• Filter resolution converts your metadata condition (e.g., {"category": "science"}) into a list of internal uint64 handles via the private _resolve_filter_to_handles method.
• Allowlist search passes those handles to the kernel's index.search(qvec, limit, allowlist=allowlist) call.
• Stale allowlist tolerance catches KeyError when a concurrent delete invalidates the allowlist, rebuilds it, and retries up to eight times.
• Fallback post-filtering kicks in if churn is extreme, performing an unfiltered search and filtering the results in Python.

This pipeline exists exactly once — in the _retrieve method of TurboQuantVectorDb — and every high-level wrapper (LlamaIndex, Haystack, Agno's Knowledge.search) delegates to it by passing the filter through the filters= argument.

Prerequisites and Setup for Filtered Vector Search

Before running any hybrid-style retrieval, you need a working Turbovec store with an embedder. The example below uses the Agno backend and OpenAI embedder, but the pattern carries across all integrations:

from turbovec.agno import TurboQuantVectorDb
from agno.knowledge.embedder.openai import OpenAIEmbedder

db = TurboQuantVectorDb(embedder=OpenAIEmbedder())
db.create()  # loads a persisted index or creates a new one

This creates or opens a persisted vector index on disk. Once initialized, you can insert documents along with metadata that will later become your hybrid filters.

Implementing Hybrid Retrieval with Filters and Allowlists

1. Insert Documents with Metadata Filters

Your metadata fields become the keyword dimension of the hybrid workflow. The simplest hybrid use case is "vector search, restricted to one metadata category":

docs = [
    {"content": "Quantum entanglement is spooky.", "metadata": {"category": "science"}},
    {"content": "The pandas ate bamboo.", "metadata": {"category": "nature"}},
    {"content": "Black holes warp spacetime.", "metadata": {"category": "science"}},
]
db.insert(content_hash="hash-001", documents=docs)  # embeds & stores them

2. Query with a Metadata Filter

The public API accepts a filters dict on search(). Under the hood, that dict is resolved to an internal handle allowlist, effectively limiting the vector search to the "keyword-filtered" document subset:

query = "What is a black hole?"
results = db.search(query, limit=5, filters={"category": "science"})

for doc in results:
    print(f"[{doc.id}] {doc.content}")

This is the recommended way to achieve hybrid retrieval, and the same filters= argument works the same way whether you're using LlamaIndex, Haystack, or the bare TurboQuantVectorDb object.

Manual Allowlist Construction for Advanced Control

If you need more control — for example, combining multiple filter conditions or custom handle manipulation — you can construct the allowlist manually. The private helper _resolve_filter_to_handles performs the same conversion used internally:

import numpy as np

# Resolve a filter to internal handles (uint64 ids)

allowlist_handles = db._resolve_filter_to_handles({"category": "science"})

# Build the query vector (must be 2-D)

qvec = db.embedder.get_embedding(query)[None, :]

# Run a raw vector search with the allowlist

scores, handles = db._index.search(
    qvec=qvec,
    k=10,
    allowlist=np.asarray(allowlist_handles, dtype=np.uint64),
)

# Convert handles back to documents

docs = [db._u64_to_doc[h] for h in handles]

Note: This snippet touches private internals. The public search(..., filters=…) is the recommended entry point for production. Use this only if you need low-level control or custom logic.

How Turbovec Handles Stale Allowlists and Edge Cases

One of the most important robustness features is the retry logic inside _retrieve. Because the allowlist is resolved at query time, concurrent inserts or deletes can invalidate it between resolution and search. The implementation wraps the search in a retry loop:

• On KeyError, the program rebuilds the allowlist from the current index and attempts the search again (up to 8 times).
• If the retry limit is exhausted — the signature of extreme churn — the code performs an unfiltered search and post-filters those results by metadata.

This tolerance is why the same filter-allowlist pattern is also used by the Haystack wrapper, as shown in haystack.py where a filter is resolved to a handle allowlist before scoring.

Key Implementation Details from the Turbovec Source

Component	File	Why It Matters
Core store implementation	turbovec-python/python/turbovec/agno.py	Implements _resolve_filter_to_handles, _retrieve, stale-allowlist retry, and fallback logic.
LlamaIndex wrapper	turbovec-python/python/turbovec/llama_index.py	Shows filters forwarding to the allowlist path from higher-level libraries.
Haystack wrapper	turbovec-python/python/turbovec/haystack.py	Proof the same filter → allowlist → vector flow works with Haystack DSL.
Filtering test suite	turbovec-python/tests/test_filtering.py	Unit tests confirming expected allowlist behavior, including retries.
Search type validation	turbovec-python/python/turbovec/agno.py — search_type setter	Raises a clear error when SearchType.hybrid is requested.

Summary

• Turbovec does not implement native hybrid retrieval; its core engine only supports pure vector search.
• The supported hybrid pattern consists of metadata filter → allowlist → vector search, which you get via the public search(..., filters=...) argument.
• Internally, filters are resolved to handle allowlists in _resolve_filter_to_handles, and stale allowlists trigger rebuild-and-retry loops (with post-filtering as a last resort).
• The same flow works across all wrapper libraries in the repository (LlamaIndex, Haystack, Agno Knowledge) because they all delegate to the same _retrieve path.
• Manual allowlist construction via private helpers is possible but discouraged; stick to the public filters= API unless you need absolute low-level control.

Frequently Asked Questions

Can turbovec directly perform hybrid retrieval with keyword search?

No. According to the source code in TurboQuantVectorDb, requesting SearchType.hybrid gene-rate raises a guard error. Turbovec's engine is intentionally pure vector search — you can emulate hybrid behavior by filtering metadata and running an allowlist search, as described here.

How do filters work internally in the turbovec search path?

Filters are passed to the public search() method as a dict. Inside _retrieve, they're converted to a list of internal handles with the private helper _resolve_filter_to_handles. That handle list becomes the allowlist parameter in the kernel call, limiting candidate vectors to those belonging to matching documents — including a retry loop for concurrent updates.

What happens if the filter allowlist is stale or deleted?

The implementation catches KeyError, rebuilds the allowlist from the current filter condition, and re-attempts the search up to 8 times. When churn is so heavy that even retries fail, it falls back to a full unfiltered search and post-filters the results in Python, which may be slower but always yields complete, filtered results.

Which high-level libraries support filters= for hybrid flow?

All wrappers that ship in the turbovec-python package support it — including LlamaIndex, Haystack, and Agno's Knowledge.search. Each simply passes the filters argument through to the same core _retrieve implementation, so you get the identical allowlist-based behavior regardless of which high-level API you choose.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}