Using Hybrid Search with Vector Stores in Semantica AGI: Implementation Guide

Semantica AGI provides a hybrid search architecture that fuses vector-based similarity with metadata filtering and configurable fusion strategies to rank results across any vector store backend.

The semantica-agi/semantica repository implements a production-ready hybrid search layer designed for complex AI applications requiring both semantic understanding and structured data constraints. This system enables developers to query vector stores using natural language while applying precise metadata predicates, then fuse results using algorithms like Reciprocal Rank Fusion (RRF) or weighted averaging.

Core Components of Hybrid Search

Semantica AGI’s hybrid search capability relies on four tightly integrated modules located in semantica/vector_store/hybrid_search.py and semantica/vector_store/hybrid_similarity.py.

HybridSearch orchestrates the complete query pipeline. This class manages embedding generation, vector retrieval, metadata filtering, and result ranking through its search() method defined at line 250 of hybrid_search.py. It supports both in-memory and external vector stores, automatically delegating to native search_vectors APIs when available.

MetadataFilter provides a fluent builder interface for Boolean metadata predicates. Located at line 47 of hybrid_search.py, this class supports operations including eq, gt, contains, and in_list. The matches() method evaluates dictionary objects against all stored conditions, enabling pre-filtering or post-filtering depending on the backend capabilities.

SearchRanker handles result fusion strategies. Implemented at line 40 of hybrid_search.py, this component supports Reciprocal-Rank-Fusion (RRF) and Weighted-Average algorithms. The ranker integrates into HybridSearch via the ranking_strategy configuration parameter.

HybridSimilarityCalculator computes decision-level hybrid similarity scores. Found at line 43 of hybrid_similarity.py, this class blends semantic and structural embeddings using configurable weights and supports multiple distance metrics including cosine, Pearson, Euclidean, and dot-product similarity.

How Hybrid Search Works

The architectural flow in semantica/vector_store/hybrid_search.py follows a six-stage pipeline:

1. Query Normalization: When HybridSearch.search() receives a string query, it lazily loads an EmbeddingGenerator (lines 26-38) to convert text into dense vectors before retrieval.

2. Data Source Resolution: The system detects whether to read raw vectors directly from in-memory backends (accessing the vectors dictionary) or delegate to remote stores via search_vectors APIs (lines 48-66).

3. Metadata Filtering: The optional MetadataFilter applies Boolean predicates either before vector similarity computation (for local data) or after backend retrieval (for remote stores), as implemented in lines 68-78.

4. Vector Similarity: For local datasets, _vector_search() computes cosine similarity using NumPy operations (lines 86-115). Remote backend results normalize into a common schema (lines 78-94).

5. Result Fusion: The SearchRanker combines result sets using the configured fusion algorithm (lines 40-48), handling both two-stage pipelines and multi-source retrieval scenarios.

6. Decision-Level Scoring: Workflows requiring structural understanding utilize HybridSimilarityCalculator.calculate_hybrid_similarity() (lines 108-144) to weight semantic embeddings against structural embeddings based on domain-specific importance.

Implementing Hybrid Search in Practice

Basic Hybrid Search with Metadata Filtering

The following example demonstrates initializing HybridSearch, constructing a metadata filter, and executing a query against the vector store:

from semantica.vector_store import HybridSearch, MetadataFilter

# Initialize (optionally inject a custom vector store)

search = HybridSearch()

# Build a filter: category == "science" AND year > 2020

metadata_filter = (
    MetadataFilter()
    .eq("category", "science")
    .gt("year", 2020)
)

# Perform the search (query can be a raw vector or text)

results = search.search(
    query="Quantum entanglement in photonic systems",
    k=10,
    metadata_filter=metadata_filter,
)

print("Top hybrid results:", results)

This implementation references HybridSearch construction (lines 62-64), MetadataFilter usage (lines 69-78), and the search() method entry point (lines 76-84) in hybrid_search.py.

Calculating Hybrid Similarity Scores

For decision-tracking workflows requiring explicit similarity computation between entities, use HybridSimilarityCalculator:

from semantica.vector_store import HybridSimilarityCalculator
import numpy as np

# Dummy embeddings (semantic & structural)

sem_vec1 = np.random.rand(768)
struct_vec1 = np.random.rand(256)
sem_vec2 = np.random.rand(768)
struct_vec2 = np.random.rand(256)

calc = HybridSimilarityCalculator(semantic_weight=0.6, structural_weight=0.4)

hybrid_score = calc.calculate_hybrid_similarity(
    semantic_vec1=sem_vec1,
    structural_vec1=struct_vec1,
    semantic_vec2=sem_vec2,
    structural_vec2=struct_vec2,
)

print("Hybrid similarity:", hybrid_score)

The calculator initialization (lines 68-74) and calculate_hybrid_similarity() method (lines 108-144) reside in hybrid_similarity.py, with automatic validation ensuring weight sums remain valid.

Finding Similar Decisions with Metadata Filters

Batch processing with metadata constraints utilizes the high-level find_most_similar_decisions() wrapper:

from semantica.vector_store import HybridSimilarityCalculator

calc = HybridSimilarityCalculator()
query_sem = np.random.rand(768)
query_struct = np.random.rand(256)

# Candidate embeddings (list of (semantic, structural) tuples)

candidates = [
    (np.random.rand(768), np.random.rand(256)),
    (np.random.rand(768), np.random.rand(256)),
]

candidate_meta = [
    {"type": "policy", "priority": 5},
    {"type": "research", "priority": 2},
]

top_hits = calc.find_most_similar_decisions(
    query_semantic=query_sem,
    query_structural=query_struct,
    candidate_embeddings=candidates,
    candidate_metadata=candidate_meta,
    top_k=5,
    filters={"type": "research"},
)

for hit in top_hits:
    print(hit["similarity"], hit["metadata"])

This method (lines 79-104 in hybrid_similarity.py) internally calls _apply_filters() (lines 48-84) to enforce metadata constraints before calculating hybrid similarity scores for the candidate pool.

Customization and Extensibility

Semantica AGI’s hybrid search supports extensive customization without requiring core modifications.

Ranking Strategy Configuration: Pass a different ranking_strategy when constructing HybridSearch to switch between "reciprocal_rank_fusion" (default) and "weighted_average" fusion algorithms.

Distance Metrics: HybridSimilarityCalculator accepts semantic_metric and structural_metric arguments supporting "cosine", "pearson", "euclidean", and "dot_product" values (lines 70-74 of hybrid_similarity.py).

Context-Aware Enhancement: The optional calculate_context_aware_similarity() method (lines 67-88) enriches hybrid scores with graph-based context similarity for relationship-heavy datasets.

Vector Store Integration: The system exports a global instance via _get_hybrid_search() in semantica/vector_store/methods.py, while the public API in semantica/vector_store/__init__.py exposes HybridSearch and HybridSimilarityCalculator for direct import.

Summary

• HybridSearch in semantica/vector_store/hybrid_search.py orchestrates the complete search pipeline, supporting both text and vector queries with optional metadata filtering.
• MetadataFilter provides a fluent interface for Boolean predicates including equality, range, and membership checks.
• SearchRanker implements both Reciprocal Rank Fusion and weighted averaging strategies for result combination.
• HybridSimilarityCalculator enables decision-level similarity scoring with configurable weights for semantic versus structural embeddings.
• The architecture supports pluggable vector stores (FAISS, Pinecone, pgVector) while maintaining consistent metadata filtering across local and remote backends.

Frequently Asked Questions

What is hybrid search in Semantica AGI?

Hybrid search in Semantica AGI refers to the combination of vector-based semantic similarity with structured metadata filtering and advanced ranking algorithms. Unlike pure vector search, this approach allows developers to constrain results using Boolean metadata predicates while maintaining semantic relevance through embedding similarity.

How does metadata filtering work?

The MetadataFilter class in hybrid_search.py (line 47) provides a builder pattern for constructing complex filters. Filters apply either pre-search (for in-memory stores) or post-search (for remote APIs) depending on the backend capabilities. The matches() method evaluates individual records against all configured predicates.

What ranking strategies are available?

The SearchRanker class supports two primary strategies: Reciprocal Rank Fusion (RRF), which scores documents based on their reciprocal rank across multiple result sets, and Weighted Average, which combines scores using configurable weights. Set the strategy via the ranking_strategy parameter when initializing HybridSearch.

Can I use custom vector stores?

Yes. HybridSearch accepts any vector store implementation through its constructor. For in-memory backends, the system reads directly from the vectors dictionary. For external stores (Pinecone, pgVector, etc.), it delegates to the store's native search_vectors API while normalizing results into a common schema for consistent ranking.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}