# How to Perform Hybrid Retrieval with Filtered Search in Turbovec Using an Allowlist

> Learn to emulate hybrid retrieval in Turbovec by filtering metadata to an allowlist and performing vector search on the constrained set. Enhance your search accuracy now.

- Repository: [Ryan Codrai/turbovec](https://github.com/RyanCodrai/turbovec)
- Tags: how-to-guide
- Published: 2026-08-22

---

**Turbovec doesn't natively support hybrid keyword-vector search, but you can emulate hybrid retrieval by resolving metadata filters to an allowlist of internal handles and running pure vector search against that constrained set.**

Turbovec is a high-performance vector database for Python, and its core engine (as implemented in [`TurboQuantVectorDb`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/agno.py)) is optimized for pure vector similarity. While there is no built-in hybrid retrieval mode, the library exposes a powerful filter-allowlist mechanism that lets you combine metadata filtering with vector search for practical hybrid-like workflows. Let's explore exactly how this works, including the underlying source logic and code examples you can run today.

## Understanding Turbovec's Hybrid Retrieval Architecture

The first thing to know is that **Turbovec explicitly rejects `SearchType.hybrid`** — the constructor guard in `TurboQuantVectorDb` raises a clear error if you attempt to use it. This design choice means hybrid retrieval isn't a slot you plug into; it's a pattern you build on top of the core engine.

The workaround relies on a filter-allowlist pipeline that powers all high-level integrations in the repository:

- **Filter resolution** converts your metadata condition (e.g., `{"category": "science"}`) into a list of internal `uint64` handles via the private `_resolve_filter_to_handles` method.
- **Allowlist search** passes those handles to the kernel's `index.search(qvec, limit, allowlist=allowlist)` call.
- **Stale allowlist tolerance** catches `KeyError` when a concurrent delete invalidates the allowlist, rebuilds it, and retries up to eight times.
- **Fallback post-filtering** kicks in if churn is extreme, performing an unfiltered search and filtering the results in Python.

This pipeline exists exactly once — in the `_retrieve` method of `TurboQuantVectorDb` — and every high-level wrapper (LlamaIndex, Haystack, Agno's `Knowledge.search`) delegates to it by passing the filter through the `filters=` argument.

## Prerequisites and Setup for Filtered Vector Search

Before running any hybrid-style retrieval, you need a working Turbovec store with an embedder. The example below uses the Agno backend and OpenAI embedder, but the pattern carries across all integrations:

```python
from turbovec.agno import TurboQuantVectorDb
from agno.knowledge.embedder.openai import OpenAIEmbedder

db = TurboQuantVectorDb(embedder=OpenAIEmbedder())
db.create()  # loads a persisted index or creates a new one

```

This creates or opens a persisted vector index on disk. Once initialized, you can insert documents along with metadata that will later become your hybrid filters.

## Implementing Hybrid Retrieval with Filters and Allowlists

### 1. Insert Documents with Metadata Filters

Your metadata fields become the keyword dimension of the hybrid workflow. The simplest hybrid use case is "vector search, restricted to one metadata category":

```python
docs = [
    {"content": "Quantum entanglement is spooky.", "metadata": {"category": "science"}},
    {"content": "The pandas ate bamboo.", "metadata": {"category": "nature"}},
    {"content": "Black holes warp spacetime.", "metadata": {"category": "science"}},
]
db.insert(content_hash="hash-001", documents=docs)  # embeds & stores them

```

### 2. Query with a Metadata Filter

The public API accepts a `filters` dict on `search()`. Under the hood, that dict is resolved to an internal handle allowlist, effectively limiting the vector search to the "keyword-filtered" document subset:

```python
query = "What is a black hole?"
results = db.search(query, limit=5, filters={"category": "science"})

for doc in results:
    print(f"[{doc.id}] {doc.content}")

```

**This is the recommended way to achieve hybrid retrieval**, and the same `filters=` argument works the same way whether you're using LlamaIndex, Haystack, or the bare `TurboQuantVectorDb` object.

## Manual Allowlist Construction for Advanced Control

If you need more control — for example, combining multiple filter conditions or custom handle manipulation — you can construct the allowlist manually. The private helper `_resolve_filter_to_handles` performs the same conversion used internally:

```python
import numpy as np

# Resolve a filter to internal handles (uint64 ids)

allowlist_handles = db._resolve_filter_to_handles({"category": "science"})

# Build the query vector (must be 2-D)

qvec = db.embedder.get_embedding(query)[None, :]

# Run a raw vector search with the allowlist

scores, handles = db._index.search(
    qvec=qvec,
    k=10,
    allowlist=np.asarray(allowlist_handles, dtype=np.uint64),
)

# Convert handles back to documents

docs = [db._u64_to_doc[h] for h in handles]

```

**Note:** This snippet touches private internals. The public `search(..., filters=…)` is the recommended entry point for production. Use this only if you need low-level control or custom logic.

## How Turbovec Handles Stale Allowlists and Edge Cases

One of the most important robustness features is the retry logic inside `_retrieve`. Because the allowlist is resolved at query time, concurrent inserts or deletes can invalidate it between resolution and search. The implementation wraps the search in a retry loop:

- On `KeyError`, the program rebuilds the allowlist from the current index and attempts the search again (up to 8 times).
- If the retry limit is exhausted — the signature of extreme churn — the code performs an unfiltered search and post-filters those results by metadata.

This tolerance is why the same filter-allowlist pattern is also used by the Haystack wrapper, as shown in [`haystack.py`](https://github.com/RyanCodrai/turbovec/blob/main/haystack.py) where a filter is resolved to a handle allowlist before scoring.

## Key Implementation Details from the Turbovec Source

| Component | File | Why It Matters |
|----------|------|-----------------|
| Core store implementation | [`turbovec-python/python/turbovec/agno.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/agno.py) | Implements `_resolve_filter_to_handles`, `_retrieve`, stale-allowlist retry, and fallback logic. |
| LlamaIndex wrapper | [`turbovec-python/python/turbovec/llama_index.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/llama_index.py) | Shows `filters` forwarding to the allowlist path from higher-level libraries. |
| Haystack wrapper | [`turbovec-python/python/turbovec/haystack.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/haystack.py) | Proof the same filter → allowlist → vector flow works with Haystack DSL. |
| Filtering test suite | [`turbovec-python/tests/test_filtering.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/tests/test_filtering.py) | Unit tests confirming expected allowlist behavior, including retries. |
| Search type validation | [`turbovec-python/python/turbovec/agno.py`](https://github.com/RyanCodrai/turbovec/blob/main/turbovec-python/python/turbovec/agno.py) — `search_type` setter | Raises a clear error when `SearchType.hybrid` is requested. |

## Summary

- **Turbovec does not implement native hybrid retrieval**; its core engine only supports **pure vector search**.
- The supported hybrid pattern consists of **metadata filter → allowlist → vector search**, which you get via the public `search(..., filters=...)` argument.
- Internally, filters are resolved to handle allowlists in `_resolve_filter_to_handles`, and stale allowlists trigger rebuild-and-retry loops (with post-filtering as a last resort).
- The same flow works across all wrapper libraries in the repository (LlamaIndex, Haystack, Agno `Knowledge`) because **they all delegate to the same `_retrieve` path**.
- Manual allowlist construction via private helpers is possible but discouraged; stick to the public `filters=` API unless you need absolute low-level control.

## Frequently Asked Questions

### Can turbovec directly perform hybrid retrieval with keyword search?

No. According to the source code in `TurboQuantVectorDb`, requesting `SearchType.hybrid` gene-rate raises a guard error. Turbovec's engine is intentionally pure vector search — you can emulate hybrid behavior by filtering metadata and running an allowlist search, as described here.

### How do filters work internally in the turbovec search path?

Filters are passed to the public `search()` method as a dict. Inside `_retrieve`, they're converted to a list of internal handles with the private helper `_resolve_filter_to_handles`. That handle list becomes the `allowlist` parameter in the kernel call, limiting candidate vectors to those belonging to matching documents — including a retry loop for concurrent updates.

### What happens if the filter allowlist is stale or deleted?

The implementation catches `KeyError`, rebuilds the allowlist from the current filter condition, and re-attempts the search up to 8 times. When churn is so heavy that even retries fail, it falls back to a full unfiltered search and post-filters the results in Python, which may be slower but always yields complete, filtered results.

### Which high-level libraries support `filters=` for hybrid flow?

All wrappers that ship in the `turbovec-python` package support it — including LlamaIndex, Haystack, and Agno's `Knowledge.search`. Each simply passes the `filters` argument through to the same core `_retrieve` implementation, so you get the identical allowlist-based behavior regardless of which high-level API you choose.