How to Configure Vector Store Backends (FAISS, Qdrant, Pinecone, Milvus, and PgVector) in Semantica

Semantica provides a unified VectorStore factory that supports FAISS, Qdrant, Pinecone, Milvus, and PgVector through optional dependencies, allowing you to swap backends by changing initialization parameters without modifying application code.

Semantica's vector storage architecture centers on a pluggable backend system located in semantica/vector_store/. Whether you need local in-memory indexing or cloud-hosted vector databases, you can configure your preferred backend using a consistent API interface while the library handles backend-specific implementation details.

Understanding the Unified VectorStore Architecture

The central entry point is semantica.vector_store.VectorStore defined in semantica/vector_store/vector_store.py. This factory class reads the backend argument and lazily instantiates the appropriate concrete store:

from semantica.vector_store import VectorStore

# Runtime backend selection

store = VectorStore(backend="qdrant", url="http://localhost:6333")

Each backend module is imported only when selected, keeping dependencies optional. If the required client library is missing, the factory raises a clear ProcessingError with an installation hint (for example, "pip install qdrant-client").

Configuring FAISS for Local In-Memory Storage

For local development or offline scenarios, FAISS provides high-performance in-memory vector indexing without external services.

• Source file: semantica/vector_store/faiss_store.py
• Primary class: FAISSStore
• Dependencies: faiss (Python bindings for the C++ library)

The FAISSStore class requires only the vector dimension for basic usage. You can optionally specify index types (flat, ivf, hnsw, pq) through the FAISSIndexBuilder:

from semantica.vector_store import FAISSStore, FAISSIndexBuilder

# Simple flat index

store = FAISSStore(dimension=768)

# Or custom HNSW index for approximate search

builder = FAISSIndexBuilder(dimension=768, index_type="hnsw")
store = builder.build()

Connecting to Qdrant (Self-Hosted or Cloud)

Qdrant supports both local Docker deployments and managed cloud instances through REST API connections.

• Source file: semantica/vector_store/qdrant_store.py
• Primary class: QdrantStore
• Dependencies: qdrant-client

Configuration parameters include:

• url – Base URL of the Qdrant service (e.g., "http://localhost:6333")
• api_key – Optional authentication token for secured deployments
• collection_name – Target collection name (defaults to "default")

from semantica.vector_store import QdrantStore

store = QdrantStore(
    url="http://localhost:6333",
    api_key="my-secure-token",
    collection_name="semantic_search"
)

Setting Up Pinecone (Managed Service)

Pinecone requires API credentials and environment specification to connect to their hosted vector database service.

• Source file: semantica/vector_store/pinecone_store.py
• Primary class: PineconeStore
• Dependencies: pinecone-client

Required configuration parameters:

• api_key – Your Pinecone API key
• environment – Pinecone environment region (e.g., "us-west1-gcp")
• index_name – Name of the index to create or use
• dimension – Vector dimensionality (required for index creation)

from semantica.vector_store import PineconeStore

store = PineconeStore(
    api_key="your-pinecone-api-key",
    environment="us-west1-gcp",
    index_name="my-vectors",
    dimension=768
)

Integrating Milvus Clusters

Milvus supports distributed vector storage with optional authentication and TLS encryption.

• Source file: semantica/vector_store/milvus_store.py
• Primary class: MilvusStore
• Dependencies: pymilvus

Configuration options:

• host – Milvus server hostname (default: "localhost")
• port – Milvus server port (default: 19530)
• username / password – Optional credentials for RBAC-enabled clusters
• secure – Boolean flag to enable TLS encryption

from semantica.vector_store import MilvusStore

store = MilvusStore(
    host="milvus.example.com",
    port=19530,
    username="vector_user",
    password="secure_pass",
    secure=True
)

Using PgVector with PostgreSQL

PgVector extends PostgreSQL with vector similarity search capabilities, ideal for applications already using Postgres.

• Source file: semantica/vector_store/pgvector_store.py
• Primary class: PgVectorStore
• Dependencies: psycopg

Key parameters:

• dsn – PostgreSQL connection string (e.g., "postgresql://user:pass@host/db")
• table_name – Database table for vector storage (default: "vectors")
• dimension – Vector dimensionality
• pool_kwargs – Optional connection pool settings

from semantica.vector_store import PgVectorStore

store = PgVectorStore(
    dsn="postgresql://app_user:password@localhost:5432/semantica_db",
    table_name="document_embeddings",
    dimension=768
)

Common Operations Across All Backends

All backend implementations expose an identical high-level API in semantica/vector_store/vector_store.py. This unified interface handles add_vectors, search_similar, delete, and namespace management regardless of the underlying storage engine.

# Add vectors with metadata

store.add_vectors(
    vectors=[[0.1, 0.2, ...], [0.3, 0.4, ...]],
    metadatas=[{"category": "tech"}, {"category": "science"}]
)

# Semantic search with metadata filtering

results = store.search_similar(
    query_vector=[0.1, 0.2, ...],
    limit=5,
    metadata_filter={"category": "tech"}
)

# Delete by IDs

store.delete(ids=["vec_001", "vec_002"])

# Multi-tenant namespace management

store.create_namespace("customer_a")
store.switch_namespace("customer_a")

The VectorStore wrapper forwards these calls to backend-specific implementations, handling quirks such as FAISS's direct ID mapping versus Qdrant's payload-based filtering transparently.

Summary

• Unified interface: The VectorStore factory in semantica/vector_store/vector_store.py provides consistent initialization across FAISS, Qdrant, Pinecone, Milvus, and PgVector.
• Optional dependencies: Each backend imports its client library lazily; missing dependencies trigger ProcessingError with specific installation commands.
• Local vs. cloud: Choose FAISS for in-memory local development, or Qdrant, Pinecone, Milvus, and PgVector for production distributed systems.
• Consistent API: All backends support add_vectors, search_similar, delete, and namespace operations through the same method signatures.

Frequently Asked Questions

How do I switch from FAISS to Qdrant without rewriting my application code?

Change the backend parameter in your VectorStore initialization and provide Qdrant-specific connection details. The unified API ensures that add_vectors and search_similar calls remain identical across backends. Update your constructor from VectorStore(backend="faiss", dimension=768) to VectorStore(backend="qdrant", url="http://localhost:6333") while keeping all other vector operations unchanged.

What happens if I don't have the required client library installed for my chosen backend?

Semantica implements lazy loading in semantica/vector_store/vector_store.py. If you initialize QdrantStore without qdrant-client installed, the code raises a ProcessingError with a clear message directing you to run pip install qdrant-client. This pattern applies to all backends including Pinecone (pinecone-client), Milvus (pymilvus), and PgVector (psycopg).

Can I use metadata filtering with all vector store backends?

Yes. All backends including FAISSStore, QdrantStore, PineconeStore, MilvusStore, and PgVectorStore implement the search_similar method with metadata_filter support. The filtering implementation varies by backend—FAISS applies post-filtering while Qdrant and PgVector use native database filtering—but the API contract remains consistent across all implementations.

Is FAISS suitable for production workloads with millions of vectors?

FAISS through semantica/vector_store/faiss_store.py works best for development, prototyping, and small-to-medium datasets that fit in memory. For millions of vectors or distributed architectures, migrate to QdrantStore, PineconeStore, or MilvusStore which support horizontal scaling, persistent storage, and managed infrastructure. The factory architecture allows this migration by changing only the initialization parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}