Semantica Vector Store Abstraction Layer: A Deep Dive into Its Modular Architecture

Semantica's vector store abstraction layer is a three-pillar architecture comprising a unified Python API front-end, interchangeable backend adapters for eight vector databases, and an extensibility framework supporting custom methods and hybrid semantic-structural search.

The semantica-agi/semantica repository implements a production-ready vector storage system that decouples application code from specific database implementations. This article examines the complete architecture based on the actual source code, covering the VectorStore facade, adapter pattern implementation, and cross-cutting extensibility features.

---

Unified Front-End: The VectorStore API

The ** VectorStore class** in semantica/vector_store/vector_store.py (lines #100-#650) serves as the sole public interface. It exposes backend-agnostic methods including store_vectors, search_vectors, update_vectors, delete_vectors, count, and decision-specific operations like search_decisions and filter_decisions.

Backend selection occurs at initialization through the backend parameter validated against SUPPORTED_BACKENDS. The private method _init_backend_store (lines #65-#84) lazily instantiates the appropriate adapter class:

from semantica.vector_store import VectorStore

# FAISS for local, high-performance indexing

store = VectorStore(backend="faiss", config={"dimension": 768})

# Or switch to remote production backend without changing code

store = VectorStore(
    backend="pinecone",
    config={
        "api_key": "YOUR_KEY",
        "environment": "us-west1-gcp",
        "dimension": 768
    }
)

For the in-memory backend, the class builds local storage structures directly: self.vectors and self.metadata dictionaries paired with VectorIndexer/VectorRetriever components for NumPy-based similarity computation.

All operations integrate with get_progress_tracker for real-time feedback during bulk operations, and missing backend methods raise explicit NotImplementedError exceptions rather than failing silently.

---

Backend Adapters: The Plug-In Implementation Layer

Each supported vector database implements a common contract through dedicated adapter modules under semantica/vector_store/. The minimal required interface includes:

add(vectors: List[np.ndarray], metadata: Optional[List[Dict]], **options) -> List[str]
search(query_vector: np.ndarray, top_k: int, **options) -> List[SearchResult]
filter_by_metadata(filters: Dict, limit: int) -> List[Dict]
count() -> int

FAISS Adapter (Local Performance)

The ** FAISSStore**, FAISSIndex, and FAISSSearch classes in faiss_store.py (lines #46-#552) provide a complete local vector search solution:

• FAISSIndex wraps the native FAISS index with metadata management
• FAISSSearch encapsulates similarity search logic
• FAISSStore presents the public adapter interface

# Data flow through the FAISS adapter

vectors = [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]]
metadata = [{"type": "doc1"}, {"type": "doc2"}]

# VectorStore.store_vectors [#94-#107] → FAISSStore.add_vectors [#98-#104]

ids = store.store_vectors(vectors, metadata=metadata)

# VectorStore.search_vectors [#96-#108] → FAISSStore.search_similar [#73-#84]

hits = store.search_vectors(query_vector=[0.1, 0.2, 0.25], k=2)

Remote Database Adapters

Adapter	Module	Key Configuration
Weaviate	weaviate_store.py	gRPC/REST endpoint, API key
Qdrant	qdrant_store.py	Host, port, collection name
Pinecone	pinecone_store.py	API key, environment, project
Milvus	milvus_store.py	URI, token, collection params

SQL-Based Adapters

• ** PgVectorStore** (pgvector_store.py) — Requires PostgreSQL with the pgVector extension; connection established via connection_string
• ** SQLiteVecStore** (sqlite_vec_store.py) — Lightweight disk-based persistence using db_path configuration

All persistent adapters expose save_index and load_index methods for state preservation across process restarts.

---

Extensibility Framework: Cross-Cutting Concerns

Method Registry for Custom Operations

The ** MethodRegistry class** in registry.py (lines #45-#156) enables runtime registration of custom storage, search, and indexing functions:

from semantica.vector_store.registry import method_registry

def custom_store(vectors, ids=None, metadata=None, **opts):
    """Custom storage implementation"""
    return ids or [f"cvec_{i}" for i in range(len(vectors))]

# Register under the "store" task namespace

method_registry.register(
    "store",
    "my_custom",
    custom_store,
    description="Demo store implementation"
)

# Discover registered methods

print(method_registry.list_all("store"))

The global method_registry instance allows plugins to extend functionality without core code modification.

Hybrid Similarity and Decision Pipelines

** HybridSimilarityCalculator** (imported in vector_store.py) merges semantic vector similarity with structural graph embeddings using Node2Vec, path-finding, and community detection algorithms.

The ** DecisionEmbeddingPipeline** enables domain-specific decision tracking:

# Initialize with graph store for structural embeddings

store.initialize_decision_pipeline(graph_store=my_graph)

# Store structured decision data

vec_id = store.store_decision(
    scenario="Credit approval",
    reasoning="Applicant has high income",
    outcome="approved",
    confidence=0.9,
    entities=["applicant", "loan"],
    category="finance"
)

# Hybrid search combining semantic and structural signals

similar = store.search_decisions(
    query="Loan approval with strong repayment ability",
    semantic_weight=0.6,
    structural_weight=0.4,
    limit=5
)

The pipeline initialization occurs in VectorStore.initialize_decision_pipeline (lines #56-#73), with search delegated to DecisionEmbeddingPipeline.find_similar_decisions (lines #511-#528).

Supporting Infrastructure

• ** EmbeddingGenerator** — Lazy initialization of text embedding models with random vector fallback
• ** ProgressTracker** — Observable operation progress for bulk workloads
• ** ProcessingError, ValidationError** — Structured exception hierarchy

---

Complete Data Flow Example

from semantica.vector_store import VectorStore

# 1. Instantiate with any backend

store = VectorStore(backend="faiss", config={"dimension": 768})

# 2. Store vectors with metadata

vectors = [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]]
metadata = [{"type": "doc1"}, {"type": "doc2"}]
ids = store.store_vectors(vectors, metadata=metadata)

# 3. Search with automatic backend delegation

query = [0.1, 0.2, 0.25]
results = store.search_vectors(query_vector=query, k=2)

# 4. Enable decision tracking with hybrid search

store.initialize_decision_pipeline(graph_store=my_graph)
decision_results = store.search_decisions(
    query="Related approval scenarios",
    semantic_weight=0.7,
    structural_weight=0.3
)

---

Summary

• Unified API — VectorStore class provides backend-agnostic interface with automatic adapter selection via _init_backend_store
• Eight Supported Backends — FAISS (local), Weaviate, Qdrant, Pinecone, Milvus (remote), pgVector, SQLiteVecStore (SQL-based), and in-memory fallback
• Common Adapter Contract — All backends implement add, search, filter_by_metadata, and count with optional save_index/load_index
• Runtime Extensibility — MethodRegistry enables custom method registration without code changes
• Hybrid Capabilities — Native integration of semantic-structural similarity and decision-specific embeddings via DecisionEmbeddingPipeline

---

Frequently Asked Questions

How do I switch between vector database backends in Semantica?

Change the backend parameter in VectorStore initialization and provide the appropriate config dictionary. The same store_vectors and search_vectors calls work unchanged across all eight supported backends. Backend-specific settings (API keys, connection strings, paths) are validated in _init_backend_store.

What happens if a backend doesn't support a specific operation?

The VectorStore facade attempts to forward the call to the backend adapter; if the method is absent, it raises NotImplementedError with a clear message. This design prevents silent failures and makes backend capability gaps explicit.

Can I use Semantica's vector store without installing heavy database dependencies?

Yes. The in-memory backend (backend="inmemory") provides full functionality using pure Python dictionaries and NumPy, requiring no external services. The SQLite backend offers lightweight disk persistence with minimal dependencies.

How does the hybrid similarity feature work in practice?

The HybridSimilarityCalculator combines scores from two sources: (1) vector cosine similarity from the embedding space, and (2) structural similarity derived from graph embeddings (Node2Vec) and path analysis. Weights are configurable per query, enabling tuning between semantic relevance and relational proximity in decision search scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}