How Semantica Supports Different Vector Store Backends: A Complete Architecture Guide

Semantica supports multiple vector store backends through a unified VectorStore class that delegates to backend-specific adapters, allowing seamless switching between FAISS, Weaviate, Qdrant, Pinecone, Milvus, PgVector, SQLite, and in-memory storage without changing application code.

Semantica’s vector storage layer is designed for flexibility, enabling developers to optimize for latency, persistence, or scalability depending on deployment requirements. The semantica-agi/semantica repository implements this through a clean abstraction that separates the high-level API from concrete storage implementations.

The VectorStore Abstraction Layer

At the core of Semantica’s multi-backend support is the VectorStore class defined in semantica/vector_store/vector_store.py. This class provides a consistent interface for vector operations including store_vectors(), search_vectors(), add_documents(), update(), and delete(), regardless of the underlying engine.

Backend Selection and Validation

When instantiating a vector store, you specify the desired backend via the backend parameter:

from semantica.vector_store import VectorStore

store = VectorStore(backend="faiss", dimension=768)

The constructor validates the backend name against VectorStore.SUPPORTED_BACKENDS, which includes FAISS, Weaviate, Qdrant, Milvus, Pinecone, PgVector, SQLite, and in-memory options. Upon validation, the instance calls _init_backend_store() (lines 14-33 in vector_store.py) to initialize the appropriate adapter.

Supported Vector Store Backends

Semantica ships with dedicated adapter classes for each major vector database technology. Each adapter implements the minimal required operations (add, search, update, delete) while handling backend-specific connection management and query translation.

FAISS (Local File-Based)

The FAISSStore adapter in semantica/vector_store/faiss_store.py provides high-performance local vector indexing without external dependencies. Ideal for prototyping and edge deployments, this adapter persists indices to disk and supports metadata filtering through FAISS’s ID mapping capabilities.

Weaviate (Cloud-Native)

For graph-vector hybrid applications, the WeaviateStore adapter in semantica/vector_store/weaviate_store.py connects to Weaviate instances. It supports automatic embedding generation, hybrid search (combining vector similarity with BM25), and rich metadata filtering through GraphQL interfaces.

Qdrant (Filtering-Enabled)

The QdrantStore adapter in semantica/vector_store/qdrant_store.py targets the Qdrant vector database, excelling at complex filtering scenarios. This backend supports payload-based filtering alongside vector search, making it suitable for applications requiring attribute-based segmentation of search results.

Pinecone (Managed Service)

For serverless deployments, the PineconeStore adapter in semantica/vector_store/pinecone_store.py interfaces with Pinecone’s managed vector service. It handles index management, namespace operations, and metadata queries while abstracting infrastructure scaling concerns.

Milvus (Distributed)

The MilvusStore adapter in semantica/vector_store/milvus_store.py supports Milvus and Zilliz Cloud for distributed vector computations. This backend handles collection management, partition strategies, and complex similarity metrics across large-scale deployments.

PgVector (PostgreSQL Extension)

For teams preferring SQL-based workflows, the PgVectorStore adapter in semantica/vector_store/pgvector_store.py leverages the PgVector PostgreSQL extension. This enables ACID-compliant vector storage within existing PostgreSQL databases, supporting transactions and complex joins between vector and relational data.

SQLite (Lightweight)

The SQLiteVecStore adapter in semantica/vector_store/sqlite_vec_store.py provides a zero-dependency, file-based solution using SQLite with vector extensions. Perfect for mobile applications or single-user tools requiring persistence without network overhead.

In-Memory (Testing Fallback)

When backend="inmemory" is specified or when adapter initialization fails, Semantica falls back to pure-Python dictionaries combined with VectorIndexer and VectorRetriever classes defined directly in vector_store.py. This mode requires no external dependencies and resets on process termination, serving as the default for unit testing and rapid prototyping.

Architecture and Implementation Details

The _init_backend_store() Method

The backend initialization logic resides in _init_backend_store(), which uses conditional imports to avoid loading unused dependencies. Based on the backend parameter value, it instantiates the corresponding adapter class and assigns it to self._backend_store:

# Simplified view of the initialization pattern

if self.backend == "faiss":
    from .faiss_store import FAISSStore
    self._backend_store = FAISSStore(self.dimension, self.config)
elif self.backend == "weaviate":
    from .weaviate_store import WeaviateStore
    self._backend_store = WeaviateStore(self.config)

# ... additional backends follow same pattern

Adapter Interface and Delegation

All public methods in VectorStore follow a delegation pattern. When _backend_store is present, operations forward to the adapter; otherwise, they execute against the in-memory implementation. For example, store_vectors() checks for the adapter and delegates accordingly:

def store_vectors(self, vectors, metadata=None, **kwargs):
    if self._backend_store:
        return self._backend_store.store_vectors(vectors, metadata, **kwargs)
    # Fallback to in-memory implementation

    return self._inmemory_store_vectors(vectors, metadata)

This ensures API consistency across backends while allowing each adapter to optimize storage formats, indexing algorithms, and connection pooling.

Extending with the Method Registry

Beyond static adapters, Semantica provides a method_registry system in semantica/vector_store/registry.py for runtime extensibility. Developers can register custom storage or search implementations without modifying core library code:

from semantica.vector_store.registry import method_registry

def custom_vector_store(vectors, metadata=None, **opts):
    # Implementation for proprietary storage

    return ["custom_id_1", "custom_id_2"]

method_registry.register(
    task="store",
    method_name="proprietary_backend",
    method_func=custom_vector_store,
    description="Custom KV store integration"
)

# Later retrieval

store_func = method_registry.get("store", "proprietary_backend")

This registry supports tasks including "store", "search", and "index", enabling hybrid strategies that combine multiple backends or implement domain-specific optimization logic.

Practical Examples

Local Development with FAISS

For rapid prototyping without infrastructure setup:

from semantica.vector_store import VectorStore

faiss_store = VectorStore(backend="faiss", dimension=384)

# Store embeddings with metadata

ids = faiss_store.store_vectors(
    vectors=[[0.1, 0.2, 0.3, 0.4]],
    metadata=[{"source": "documentation", "chunk_id": 1}]
)

# Search top-k similar vectors

results = faiss_store.search_vectors(
    query_vector=[0.1, 0.2, 0.3, 0.4], 
    k=5
)

Production Deployment with PgVector

When deploying with existing PostgreSQL infrastructure:

pg_store = VectorStore(
    backend="pgvector",
    config={
        "connection_string": "postgresql://user:pass@localhost/semantica",
        "table_name": "knowledge_embeddings",
        "dimension": 768,
        "distance_metric": "cosine"
    }
)

# Bulk insert with transaction safety

vectors = [[0.0]*768 for _ in range(1000)]
ids = pg_store.store_vectors(
    vectors=vectors, 
    metadata=[{"batch": "v1.0"}] * 1000
)

# Query with PostgreSQL-backed persistence

similar = pg_store.search_vectors(query_vector=vectors[0], k=10)

Custom Backend Integration

Extending Semantica for proprietary infrastructure:

from semantica.vector_store.registry import method_registry

def enterprise_redis_store(vectors, metadata=None, **opts):
    # Connect to internal Redis Vector Library

    redis_client = opts.get("client")
    pipe = redis_client.pipeline()
    for vec, meta in zip(vectors, metadata or []):
        pipe.hset(f"vec:{meta['id']}", mapping={"vector": vec, "meta": meta})
    pipe.execute()
    return [meta['id'] for meta in metadata]

method_registry.register(
    task="store",
    method_name="enterprise_redis",
    method_func=enterprise_redis_store
)

Summary

• Unified API: The VectorStore class in semantica/vector_store/vector_store.py provides a single interface for all vector operations, abstracting backend differences.
• Eight Native Backends: Semantica supports FAISS, Weaviate, Qdrant, Pinecone, Milvus, PgVector, SQLite, and in-memory storage through dedicated adapter classes.
• Dynamic Initialization: The _init_backend_store() method handles backend-specific imports and instantiation based on the backend parameter and SUPPORTED_BACKENDS validation.
• Graceful Fallback: When no external backend is configured, the system degrades to an in-memory implementation using internal dictionaries.
• Runtime Extensibility: The method_registry in registry.py enables custom backend integration without forking the core library.

Frequently Asked Questions

What vector store backends does Semantica support out of the box?

Semantica supports eight backends natively: FAISS for local file-based indexing, Weaviate for cloud-native graph-vector hybrid storage, Qdrant for filtered vector search, Pinecone for managed serverless deployments, Milvus for distributed vector processing, PgVector for PostgreSQL-integrated storage, SQLite for lightweight file-based persistence, and an in-memory dictionary backend for testing.

How do I configure a specific vector store backend in Semantica?

Pass the backend parameter when initializing VectorStore, along with a config dictionary containing connection details. For example, set backend="pgvector" with a connection_string in the config for PostgreSQL, or backend="faiss" with dimension settings for local development. The constructor validates against VectorStore.SUPPORTED_BACKENDS before initializing the appropriate adapter.

Can I use multiple vector store backends simultaneously in the same application?

Yes. You can instantiate multiple VectorStore objects with different backend parameters in the same Python process. Each instance maintains its own _backend_store adapter, allowing you to route different document collections to optimized storage engines (e.g., hot data in FAISS, archival data in PgVector) within a single application.

How do I add a custom vector store backend to Semantica without modifying core code?

Use the method_registry from semantica/vector_store/registry.py to register custom implementations for "store", "search", or "index" tasks. Define a function matching the expected signature (accepting vectors, metadata, and options), then register it with method_registry.register(task="store", method_name="custom", method_func=your_func). This approach avoids modifying vector_store.py while integrating proprietary or specialized storage systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}