How OpenSearch Hybrid Search Combines BM25 and Vector Search with RRF Fusion

OpenSearch hybrid search merges lexical BM25 scoring with dense vector similarity using Reciprocal Rank Fusion (RRF) to produce a unified relevance score that leverages both exact keyword matching and semantic understanding.

The jamwithai/production-agentic-rag-course repository implements a production-ready OpenSearch hybrid search system that natively combines traditional text retrieval with neural embeddings. This implementation demonstrates how to configure and execute hybrid queries that fuse BM25 lexical relevance with k-NN vector similarity using OpenSearch's built-in RRF pipeline.

Index Configuration and RRF Pipeline Setup

Hybrid search requires an index that supports both text analysis and vector storage. In src/services/opensearch/index_config_hybrid.py (lines 9-50), the index mapping defines textual fields for BM25 alongside a knn_vector field containing 1024-dimensional embeddings.

The Reciprocal Rank Fusion pipeline is defined in the same file (lines 72-80) as a post-processing ingest pipeline:

{
  "id": "hybrid-rrf-pipeline",
  "description": "Post processor for hybrid RRF search",
  "phase_results_processors": [
    {
      "score-ranker-processor": {
        "combination": {
          "technique": "rrf",
          "rank_constant": 60
        }
      }
    }
  ]
}

This configuration uses score-ranker-processor with the RRF technique and a rank_constant of 60, which determines the smoothing factor in the fusion formula.

Query Construction and Execution

When use_hybrid=True is specified, the system constructs two independent sub-queries. The _search_hybrid_native method in src/services/opensearch/client.py (lines 44-56) wraps these into OpenSearch's native hybrid query object:

1. BM25 query: A multi-match query generated by QueryBuilder that performs lexical matching on text fields
2. k-NN query: A vector similarity search using the query embedding against the knn_vector field

The client sends the request with the search pipeline parameter:

self.client.search(
    index=self.index_name,
    body=search_body,
    params={"search_pipeline": HYBRID_RRF_PIPELINE["id"]},
)

RRF Fusion and Scoring Mechanics

OpenSearch executes both sub-queries independently, producing two ranked result lists. The RRF processor then fuses these rankings using the formula:

score_RRF(d) = Σ 1 / (k + rank_i(d))

Where k is the rank_constant (60) and rank_i(d) is the document's rank in each respective result set (BM25 or k-NN). Documents appearing in both result sets receive higher combined scores, boosting results that perform moderately well in both lexical and semantic spaces.

After fusion, the search_unified method (lines 176-205 in client.py) filters results by applying a min_score threshold before returning the standardized {"total": ..., "hits": [...]} payload.

API Integration and Fallback Behavior

The FastAPI router in src/routers/hybrid_search.py (lines 34-42) exposes the hybrid search functionality via the /api/v1/hybrid-search/ endpoint. If the embedding service fails to generate vectors or use_hybrid is false, the system gracefully falls back to pure BM25 search, ensuring system reliability.

Using the Python Client

from src.services.opensearch.client import OpenSearchClient
from src.config import Settings

settings = Settings()
client = OpenSearchClient(host="http://localhost:9200", settings=settings)

query = "deep learning for graph neural networks"
embedding = await embeddings_service.embed_query(query)

results = client.search_unified(
    query=query,
    query_embedding=embedding,
    size=10,
    use_hybrid=True,
    min_score=0.0,
)

print("Total hits:", results["total"])
for hit in results["hits"]:
    print(f"{hit['title']} (score={hit['score']:.3f})")

Calling the HTTP Endpoint

curl -X POST https://my-api.example.com/api/v1/hybrid-search/ \
  -H "Content-Type: application/json" \
  -d '{
        "query": "graph neural networks",
        "size": 5,
        "use_hybrid": true,
        "min_score": 0.0
      }'

Summary

• Index setup: The system stores both text fields and 1024-dimensional knn_vector embeddings in a unified OpenSearch index defined in index_config_hybrid.py
• RRF pipeline: A post-processing score-ranker-processor with rank_constant: 60 fuses BM25 and k-NN result rankings
• Query execution: The search_unified method in client.py orchestrates hybrid queries by combining QueryBuilder output with raw k-NN vector search
• Score-agnostic fusion: RRF operates on document ranks rather than raw scores, eliminating the need to normalize between BM25 and cosine similarity scales
• Graceful degradation: The API automatically falls back to BM25-only search when embeddings are unavailable

Frequently Asked Questions

What is the purpose of the rank_constant in RRF?

The rank_constant (set to 60 in this implementation) is a smoothing parameter that prevents top-ranked documents from dominating the fused results. Higher values diminish the score difference between ranks, allowing documents ranked lower in one result set to remain competitive if they appear in both sets. This value is configured in src/services/opensearch/index_config_hybrid.py within the hybrid-rrf-pipeline definition.

How does the system handle queries when embeddings cannot be generated?

The FastAPI router in src/routers/hybrid_search.py checks the use_hybrid flag and embedding availability before calling search_unified. If vector generation fails or use_hybrid is false, the client automatically falls back to the standard BM25 search path, ensuring users always receive relevant lexical results even when semantic search is unavailable.

Where is the hybrid query logic implemented in the codebase?

The core hybrid search logic resides in src/services/opensearch/client.py. Specifically, the _search_hybrid_native method (lines 44-56) constructs the native OpenSearch hybrid query object, while search_unified (lines 176-205) orchestrates the entire search flow including pipeline selection and result post-processing.

Why use RRF instead of weighted score averaging?

RRF is score-agnostic, meaning it works only with relative rankings rather than absolute similarity scores. This is critical because BM25 scores (based on term frequency and inverse document frequency) and vector cosine similarity scores exist on completely different scales and distributions. RRF requires no manual weight tuning and naturally handles the heterogeneous score spaces without normalization, as implemented in the score-ranker-processor of the hybrid pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}