How to Integrate Semantica with Vector Stores: FAISS, Qdrant, Pinecone, and pgVector

Semantica provides a backend-agnostic abstraction layer that lets you switch between FAISS, Qdrant, Pinecone, and pgVector by changing a single configuration parameter.

All four vector databases integrate through the VectorStore class in semantica-agi/semantica, which dynamically loads the appropriate backend implementation based on your backend setting. This design keeps dependencies optional while exposing a unified API across all supported stores.

Understanding the Vector Store Architecture

Backend-Agnostic Design

The core abstraction lives in semantica/vector_store/vector_store.py. When you instantiate VectorStore, it checks the backend parameter against SUPPORTED_BACKENDS and lazily imports the corresponding module:

Backend	Module	Key Classes	Trigger
FAISS	faiss_store.py	FAISSStore, FAISSIndex, FAISSSearch, FAISSIndexBuilder	backend="faiss"
Qdrant	qdrant_store.py	QdrantStore, QdrantClient, QdrantCollection, QdrantSearch	backend="qdrant"
Pinecone	pinecone_store.py	PineconeStore, PineconeClient, PineconeIndex, PineconeSearch	backend="pinecone"
pgVector	pgvector_store.py	PgVectorStore	backend="pgvector"

Each backend wrapper implements the same contract, enabling seamless swapping without rewriting higher-level code.

Unified API Methods

All backends expose these identical methods:

• add_vectors(vectors, ids, metadata) — bulk vector insertion
• search_similar(query_vector, k, **options) — similarity search returning {id, score, metadata, vector, distance}
• save(), load(), delete(), count() — persistence operations (where supported)

Dependency Guarding

The implementation uses availability flags (FAISS_AVAILABLE, QDRANT_AVAILABLE, etc.) to keep third-party dependencies truly optional. If you request a backend without its SDK installed, VectorStore.__init__ raises a clear error indicating the missing package.

Integrating Semantica with FAISS

FAISS runs entirely in-process with no external service required, making it ideal for local development and embedded deployments.

from semantica.vector_store import FAISSStore
import numpy as np

# Create an in-memory FAISS index with 768-dimensional vectors

store = FAISSStore(dimension=768)

# Add vectors with string IDs

store.add_vectors(
    vectors=np.random.rand(5, 768),
    ids=["a", "b", "c", "d", "e"]
)

# Search for 3 nearest neighbors

results = store.search_similar(
    query_vector=np.random.rand(1, 768),
    k=3
)

print("FAISS results:", results)

The FAISSStore class in semantica/vector_store/faiss_store.py handles index building, optional persistence to disk, and GPU acceleration when available.

Integrating Semantica with Qdrant

Qdrant supports both self-hosted deployments and managed cloud instances, with native payload filtering for hybrid search.

from semantica.vector_store import QdrantStore
import numpy as np

# Connect to Qdrant server

store = QdrantStore(url="http://localhost:6333")
store.connect()

# Create collection with cosine similarity

store.create_collection(
    name="my_collection",
    vector_size=768,
    distance="Cosine"
)

# Add vectors with metadata for filtering

store.add_vectors(
    vectors=np.random.rand(10, 768),
    ids=range(10),
    metadata={"category": "demo"}
)

# Search with metadata filter

results = store.search_similar(
    query_vector=np.random.rand(1, 768),
    k=5,
    filter={"must": [{"key": "category", "match": {"value": "demo"}}]}
)

print("Qdrant results:", results)

The QdrantStore implementation translates Semantica's generic filter format into Qdrant's native Filter objects, enabling complex boolean queries alongside vector similarity.

Integrating Semantica with Pinecone

Pinecone is a fully managed vector database with automatic scaling and zero maintenance overhead.

from semantica.vector_store import PineconeStore
import numpy as np

# Initialize with your Pinecone credentials

store = PineconeStore(
    api_key="YOUR_PINECONE_API_KEY",
    environment="us-west1-gcp"
)
store.connect()

# Create index with dot product metric

store.create_index(
    name="my-index",
    dimension=768,
    metric="dotproduct"
)

# Insert vectors with per-item metadata

store.add_vectors(
    vectors=np.random.rand(8, 768),
    ids=[str(i) for i in range(8)],
    metadata=[{"topic": "test"}] * 8
)

results = store.search_similar(
    query_vector=np.random.rand(1, 768),
    k=4
)

print("Pinecone results:", results)

PineconeStore in semantica/vector_store/pinecone_store.py wraps the official Pinecone SDK and handles index lifecycle management.

Integrating Semantica with pgVector

pgVector brings vector search to PostgreSQL, combining relational data operations with similarity search in a single database.

from semantica.vector_store import PgVectorStore
import numpy as np

# Connect via standard PostgreSQL connection string

store = PgVectorStore(
    connection_string="postgresql://user:pass@localhost:5432/dbname"
)
store.connect()

# Ensure table exists with proper vector column

store.ensure_table(table_name="vectors", dimension=768)

# Insert with metadata

store.add_vectors(
    vectors=np.random.rand(6, 768),
    ids=[i for i in range(6)],
    metadata=[{"tag": "pg"}] * 6
)

results = store.search_similar(
    query_vector=np.random.rand(1, 768),
    k=2
)

print("pgVector results:", results)

The PgVectorStore class verifies the pgVector extension is installed and translates similarity searches into optimized SQL queries. For setup details, see docs/vector_stores/pgvector.md in the repository.

Metadata Filtering Across All Backends

A key strength of Semantica's vector store integration is consistent hybrid search support. Each backend translates Semantica's generic metadata filters into native equivalents:

• FAISS: In-memory filtering with post-search refinement
• Qdrant: Native payload filters using Qdrant's Filter DSL
• Pinecone: Metadata dictionaries passed to the query API
• pgVector: SQL WHERE clauses appended to vector similarity queries

This enables complex queries like "find vectors similar to X where category = 'electronics' and price < 100" across any backend.

Progress Tracking and Observability

All vector store implementations integrate with Semantica's utils.progress_tracker and utils.logging utilities. Long-running batch operations report consistent progress regardless of which backend you choose.

CLI Integration

The command-line interface in semantica/cli.py exposes a --backend flag that forwards directly to VectorStore. This lets you switch backends from the command line:

python -m semantica ingest --backend=qdrant --url=http://localhost:6333
python -m semantica ingest --backend=faiss --dimension=768

Summary

• Single abstraction: VectorStore class in vector_store.py routes to the correct implementation based on backend parameter
• Four supported stores: FAISS (local), Qdrant (self-hosted/cloud), Pinecone (managed), pgVector (PostgreSQL)
• Identical API: add_vectors, search_similar, and persistence methods work consistently across all backends
• Optional dependencies: SDKs are lazily imported with clear error messages if missing
• Hybrid search: Metadata filtering works natively on every backend through automatic translation

Frequently Asked Questions

How do I switch between vector stores in Semantica?

Change the backend parameter when constructing VectorStore or via CLI flags. The implementation in vector_store.py handles all routing automatically—no code changes needed in your application logic.

Can I use multiple vector stores simultaneously in one Semantica project?

Yes. Instantiate separate VectorStore objects with different backend values. Each maintains its own connection state and can operate in parallel within the same Python process.

What dependencies are required for each vector store backend?

• FAISS: faiss-cpu or faiss-gpu
• Qdrant: qdrant-client
• Pinecone: pinecone-client
• pgVector: pgvector Python package plus a PostgreSQL server with the pgVector extension

Semantica raises an ImportError with installation instructions if you attempt to use a backend without its required package.

Does Semantica support vector store persistence for FAISS?

Yes. FAISSStore in faiss_store.py implements save() and load() methods that serialize the index to disk using FAISS's native I/O functions, enabling index reuse across process restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}