How to Implement Hybrid Semantic Search with Semantica's VectorStore

Semantica's VectorStore combines semantic (text-based) and structural (graph-based) embeddings through HybridSimilarityCalculator and HybridSearch to deliver precision retrieval with configurable weighting and metadata filtering.

The semantica-agi/semantica framework enables hybrid semantic search by fusing vector similarity with structural knowledge graph signals. This article walks through the core components, implementation patterns, and code examples drawn directly from the source.

Core Architecture: Two Complementary Modules

Semantica's hybrid search capability splits across two specialized classes in semantica/vector_store/:

Component	Key File	Responsibility
HybridSimilarityCalculator	hybrid_similarity.py	Computes weighted combination of semantic and structural similarity scores
HybridSearch	hybrid_search.py	Orchestrates full search pipeline with filtering, backends, and result fusion

HybridSimilarityCalculator: The Scoring Engine

The HybridSimilarityCalculator class in semantica/vector_store/hybrid_similarity.py produces a single hybrid score from dual embeddings. Key characteristics:

• Weights must sum to 1.0: semantic_weight + structural_weight = 1.0
• Four supported metrics: cosine, pearson, euclidean, dot_product
• Dimension handling: Automatically pads mismatched vector dimensions
• Batch processing: Efficient bulk similarity computations
• Runtime adaptability: Call update_weights() to shift domain priorities

Default weighting is 0.7 semantic / 0.3 structural, favoring text meaning while preserving structural relationships.

HybridSearch: The Orchestration Layer

The HybridSearch class in semantica/vector_store/hybrid_search.py manages end-to-end retrieval:

• Accepts raw query strings (auto-embedded) or pre-computed vectors
• Delegates to vector_store.search_vectors when a backend (FAISS, Qdrant, Pinecone) is attached
• Falls back to in-memory cosine similarity for backend-less operation
• Applies MetadataFilter chains for pre-ranking pruning
• Uses SearchRanker for Reciprocal Rank Fusion (RRF) or weighted-average merging

The Five-Stage Search Pipeline

Understanding how HybridSearch.search() processes queries:

1. Embedding Generation — String queries trigger lazy EmbeddingGenerator instantiation from semantica.embeddings
2. Vector Retrieval — Backend delegation or in-memory fallback via _vector_search()
3. Metadata Filtering — MetadataFilter chains (.eq(), .gt(), .in_list(), etc.) discard non-matching candidates
4. Hybrid Scoring — calculate_hybrid_similarity() merges semantic and structural similarities per candidate
5. Result Fusion — SearchRanker.rank() applies RRF or weighted strategies for multi-source searches

Implementation Examples

Basic Hybrid Similarity Computation

Compute hybrid scores between two decisions with explicit weighting:

from semantica.vector_store import HybridSimilarityCalculator
import numpy as np

calc = HybridSimilarityCalculator(semantic_weight=0.6, structural_weight=0.4)

semantic_a = np.array([0.12, 0.33, 0.55])
structural_a = np.array([0.78, 0.21, 0.09])

semantic_b = np.array([0.15, 0.30, 0.60])
structural_b = np.array([0.80, 0.20, 0.10])

hybrid_score = calc.calculate_hybrid_similarity(
    semantic_a, structural_a, semantic_b, structural_b
)
print("Hybrid similarity:", hybrid_score)  # Single float 0.0-1.0

In-Memory Hybrid Search with Metadata Filtering

Search without external backends using HybridSearch directly:

from semantica.vector_store import HybridSearch, MetadataFilter
import numpy as np

# Prepare dummy dataset

vectors = [np.random.rand(128) for _ in range(100)]
metadata = [
    {"category": "finance", "risk": "low"} if i % 2 == 0 
    else {"category": "health", "risk": "high"} 
    for i in range(100)
]
ids = [f"id_{i}" for i in range(100)]

# Chain metadata filters: category == "finance" AND risk ≤ 0.5

meta_filter = MetadataFilter().eq("category", "finance").lte("risk", 0.5)

search = HybridSearch()
results = search.search(
    query="loan approval request",  # Auto-embedded via EmbeddingGenerator

    vectors=vectors,
    metadata=metadata,
    vector_ids=ids,
    k=5,
    metadata_filter=meta_filter,
)

for r in results:
    print(r["id"], r["score"], r["metadata"])

Multi-Source Search with Result Fusion

Aggregate results from multiple vector sources using SearchRanker:

from semantica.vector_store import HybridSearch, SearchRanker, MetadataFilter
import numpy as np

source_a = {"vectors": vectors[:50], "metadata": metadata[:50], "ids": ids[:50]}
source_b = {"vectors": vectors[50:], "metadata": metadata[50:], "ids": ids[50:]}

# Per-source metadata filters

source_a["filter"] = MetadataFilter().eq("category", "finance")
source_b["filter"] = MetadataFilter().eq("category", "health")

search = HybridSearch()
fused_results = search.multi_source_search(
    query_vector=np.random.rand(128),  # Pre-computed embedding

    sources=[source_a, source_b],
    k=10,
    ranking_strategy="reciprocal_rank_fusion",  # or "weighted_average"

)

print("Fused top-10 results:")
for r in fused_results:
    print(r["id"], r["score"], r["metadata"])

Advanced Capabilities

Context-Aware Scoring

Extend hybrid similarity with graph neighborhood signals using calculate_context_aware_similarity(). This boosts candidates that are proximal in provenance graphs—useful for decision lineage retrieval.

Custom Backend Integration

The abstract VectorStore base class in semantica/vector_store/vector_store.py defines the contract for pluggable backends. Implement search_vectors() to connect FAISS, Qdrant, Pinecone, or custom stores.

Dynamic Weight Adaptation

Adjust semantic/structural balance at runtime without reconstruction:

calc = HybridSimilarityCalculator(semantic_weight=0.7, structural_weight=0.3)

# Later: shift toward structural signals for schema-heavy queries

calc.update_weights(new_sem_weight=0.4, new_struct_weight=0.6)

Custom Similarity Metrics

Extend _calculate_similarity() in HybridSimilarityCalculator and register new metrics in valid_metrics for domain-specific distance functions.

Key Source Files Reference

File	Purpose	Direct Link
semantica/vector_store/hybrid_similarity.py	HybridSimilarityCalculator implementation	View source
semantica/vector_store/hybrid_search.py	HybridSearch, MetadataFilter, SearchRanker	View source
semantica/vector_store/vector_store.py	Backend abstraction (VectorStore base)	View source
semantica/embeddings/__init__.py	EmbeddingGenerator for text-to-vector	View source

Observability for long-running operations is handled via semantica/utils/progress_tracker.py.

Summary

• Hybrid semantic search in Semantica fuses text embeddings with graph-structured signals through configurable weighting
• HybridSimilarityCalculator performs the core math; HybridSearch orchestrates the full pipeline
• MetadataFilter enables precise pre-filtering with chainable conditions (.eq(), .gt(), .in_list(), etc.)
• Backend delegation supports FAISS, Qdrant, Pinecone via the VectorStore abstraction; automatically falls back to in-memory cosine similarity
• Multi-source fusion via SearchRanker with RRF or weighted-average strategies
• Runtime adaptation possible through update_weights() for domain-specific retrieval tuning

Frequently Asked Questions

What is the default weighting between semantic and structural similarity?

The default weighting is 0.7 semantic / 0.3 structural, emphasizing text meaning while preserving graph relationships. Override via constructor or update_weights() at any time.

Can Semantica's hybrid search work without external vector databases?

Yes. HybridSearch automatically falls back to in-memory cosine similarity when no vector_store backend is supplied. This is implemented in _vector_search() within hybrid_search.py.

How does metadata filtering interact with vector similarity?

MetadataFilter applies before similarity ranking—candidates failing metadata constraints are pruned entirely. This reduces computational overhead and improves precision for structured queries.

What fusion strategies are available for multi-source searches?

SearchRanker supports Reciprocal Rank Fusion (RRF) by default, which normalizes ranks across sources without score calibration. Alternatively, specify "weighted_average" for score-based merging in multi_source_search().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}