How to Configure Weaviate Vector Database for RAG Workflows in LMForge

To configure Weaviate for RAG in LMForge, deploy the container with WEAVIATE_HTTP_HOST, WEAVIATE_GRPC_HOST, and WEAVIATE_API_KEY environment variables, then initialize the FlaskWeaviate extension and VectorDatabaseService to enable semantic search.

The LMForge end-to-end LLMOps platform leverages Weaviate as its dedicated vector store to power Retrieval-Augmented Generation (RAG) pipelines. Properly configuring the Weaviate vector database for RAG workflows requires setting up three distinct layers: Docker runtime environment variables, Flask application configuration, and Python service integration. This guide walks through each layer using the actual source implementation from the haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents repository.

Configuration Architecture Overview

The platform separates concerns across three layers to ensure flexibility between local development and production deployments.

Layer	What it does	Where it lives
Docker / Runtime	Launches a Weaviate container and injects connection settings via environment variables.	docker/docker-compose.yaml → WEAVIATE_HTTP_HOST/PORT, WEAVIATE_GRPC_HOST/PORT, WEAVIATE_API_KEY
Application defaults	Provides fallback values that are overridden by the env vars at startup.	api/config/default_config.py
Python integration	Creates a FlaskWeaviate instance, wraps it with LangChain’s WeaviateVectorStore, and exposes a collection called Dataset.	api/internal/extension/weaviate_extension.py, api/internal/service/vector_database_service.py, api/internal/core/retrievers/semantic.py

Docker and Runtime Configuration

Environment Variables

Before starting the stack, define the following variables in your .env file or host environment. These values must match between the Weaviate container and the application services (llmops-api and llmops-celery).

Variable	Meaning	Typical value
WEAVIATE_HTTP_HOST	Host name reachable from the API container.	llmops-weaviate (service name)
WEAVIATE_HTTP_PORT	HTTP port (default 8080).	8080
WEAVIATE_GRPC_HOST	Same host for the gRPC endpoint.	llmops-weaviate
WEAVIATE_GRPC_PORT	gRPC port (default 50051).	50051
WEAVIATE_API_KEY	Secret that authorises every request.	(generate a random UUID string)

Docker Compose Setup

The docker/docker-compose.yaml file defines the llmops-weaviate service with authentication enabled and persistence configured.

# docker/docker-compose.yaml (excerpt)

  llmops-weaviate:
    image: semitechnologies/weaviate:1.28.4
    container_name: llmops-weaviate
    environment:
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'false'       # require API key

      AUTHENTICATION_APIKEY_ENABLED: 'true'
      AUTHENTICATION_APIKEY_ALLOWED_KEYS: '${WEAVIATE_API_KEY}'
      AUTHORIZATION_ADMINLIST_ENABLED: 'true'
      # ... other defaults (persistence, ports, etc.)

    volumes:
      - ./volumes/weaviate:/var/lib/weaviate
    ports:
      - "8080:8080"    # HTTP (REST/GraphQL)

      - "50051:50051"  # gRPC

Both the llmops-api and llmops-celery services in the same compose file receive the WEAVIATE_* environment variables, ensuring the API server and background workers can reach Weaviate.

Application Configuration Layer

Default Configuration

The api/config/default_config.py file provides fallback values for the Weaviate connection parameters. These are overridden by environment variables at runtime.

# api/config/default_config.py (conceptual excerpt)

WEAVIATE_HTTP_HOST = "localhost"
WEAVIATE_HTTP_PORT = 8080
WEAVIATE_GRPC_HOST = "localhost"
WEAVIATE_GRPC_PORT = 50051
WEAVIATE_API_KEY = None

Runtime Configuration

The api/config/config.py file reads the environment variables at startup and makes them available to the Flask application context.

# api/config/config.py (excerpt)

self.WEAVIATE_HTTP_HOST = _get_env("WEAVIATE_HTTP_HOST")
self.WEAVIATE_HTTP_PORT = _get_env("WEAVIATE_HTTP_PORT")
self.WEAVIATE_GRPC_HOST = _get_env("WEAVIATE_GRPC_HOST")
self.WEAVIATE_GRPC_PORT = _get_env("WEAVIATE_GRPC_PORT")
self.WEAVIATE_API_KEY   = _get_env("WEAVIATE_API_KEY")

Keep these values synchronized with your .env file to avoid connection errors during deployment.

Python Integration and Service Layer

FlaskWeaviate Extension

The api/internal/extension/weaviate_extension.py file instantiates a Flask-aware Weaviate client that other services can inject.

# api/internal/extension/weaviate_extension.py

from flask_weaviate import FlaskWeaviate
weaviate = FlaskWeaviate()          # ← creates a Flask‑aware client

This extension reads the configuration values from the Flask app (populated from api/config/config.py) and builds a Weaviate client (self.client).

VectorDatabaseService Wrapper

The api/internal/service/vector_database_service.py file wraps the client in a LangChain-compatible vector store.

# api/internal/service/vector_database_service.py

@property
def vector_store(self) -> WeaviateVectorStore:
    return WeaviateVectorStore(
        client=self.weaviate.client,
        index_name=COLLECTION_NAME,   # "Dataset"

        text_key="text",
        embedding=self.embeddings_service.cache_backed_embeddings,
    )

Key implementation details:

• COLLECTION_NAME = "Dataset" defines the Weaviate class that stores document fragments.
• The embedding parameter receives pre-computed embeddings from EmbeddingsService.

Semantic Retriever for RAG

The api/internal/core/retrievers/semantic.py file implements the similarity search retriever used by RAG pipelines.

# api/internal/core/retrievers/semantic.py (excerpt)

search_result = self.vector_store.similarity_search_with_relevance_scores(
    query=query,
    k=k,
    filters=Filter.all_of([
        Filter.by_property("dataset_id").contains_any([...]),
        Filter.by_property("document_enabled").equal(True),
        Filter.by_property("segment_enabled").equal(True),
    ]),
    **self.search_kwargs,
)

This retriever enforces that returned documents belong to specified dataset_ids and have both document_enabled and segment_enabled set to True.

CRUD Operations and Indexing

Document Indexing Process

The api/internal/service/indexing_service.py file handles the ingestion pipeline. After splitting documents into segments, the _indexing method stores each segment with metadata:

# Conceptual excerpt from indexing_service.py

self.vector_database_service.collection.data.insert(
    uuid=str(uuid4()),
    properties={
        "text": segment.page_content,
        "dataset_id": str(dataset_id),
        "document_id": str(document_id),
        "segment_enabled": True,
        "document_enabled": True,
        "source": metadata.get("source", "unknown"),
    }
)

The collection property in VectorDatabaseService returns self.weaviate.client.collections.get(COLLECTION_NAME), providing direct access to the Weaviate CRUD API.

Updates and Deletions

To update document status, the update_document_enabled method modifies the document_enabled field for all segments matching a specific Weaviate uuid.

For deletions, the delete_document and delete_dataset methods use the Weaviate filter API:

# Conceptual usage

from weaviate.classes.query import Filter

filter_criteria = Filter.by_property("document_id").equal(document_uuid)
self.vector_database_service.collection.data.delete_many(filter_criteria)

This removes all matching objects in a single batch operation.

Practical Implementation Examples

Starting the Stack with Docker

# 1️⃣ Copy the example env file and set a secret API key

cp .env.example .env

# Edit .env – replace WEAVIATE_API_KEY with a UUID, e.g.:

# WEAVIATE_API_KEY=3f9c1d2e-8a4b-4e12-9c3f-5d2b3e6a7c9d

# 2️⃣ Bring up the whole platform

docker compose -f docker/docker-compose.yaml up -d

The API (llmops-api) automatically connects to llmops-weaviate using the injected environment variables.

Adding Documents to the Vector Store

from api.internal.service.vector_database_service import VectorDatabaseService
from api.internal.service.embeddings_service import EmbeddingsService
from api.internal.extension.weaviate_extension import weaviate
from uuid import uuid4

# Initialize services

vector_svc = VectorDatabaseService(
    weaviate=weaviate,
    embeddings_service=EmbeddingsService(...)
)

dataset_id = uuid4()
document_id = uuid4()

# Insert segments (low-level example)

for segment in document_segments:
    vector_svc.collection.data.insert(
        uuid=str(uuid4()),
        properties={
            "text": segment.page_content,
            "dataset_id": str(dataset_id),
            "document_id": str(document_id),
            "segment_enabled": True,
            "document_enabled": True,
            "source": segment.metadata.get("source", "unknown"),
        }
    )

In production, the IndexingService._indexing method orchestrates this process automatically.

Performing Semantic Search for RAG

from api.internal.core.retrievers.semantic import SemanticRetriever
from api.internal.service.vector_database_service import VectorDatabaseService
from api.internal.extension.weaviate_extension import weaviate

# Initialize vector store

vector_store = VectorDatabaseService(
    weaviate=weaviate,
    embeddings_service=EmbeddingsService(...)
).vector_store

# Create retriever for specific datasets

retriever = SemanticRetriever(
    dataset_ids=[dataset_id],
    vector_store=vector_store,
    search_kwargs={"k": 5}
)

# Execute search

query = "How does the platform handle document versioning?"
results = retriever.get_relevant_documents(query)

for doc in results:
    print(f"Relevance: {doc.metadata['score']:.3f} | Content: {doc.page_content[:200]}")

The SemanticRetriever automatically applies filters for dataset_id, document_enabled, and segment_enabled to ensure only valid documents are returned.

Updating Document Status

from api.internal.service.indexing_service import IndexingService
from uuid import UUID

index_svc = IndexingService(...)
document_uuid = UUID("c6d9c7e5-2b1f-4a5c-9e4d-2c3f8b5a6d7e")

# Toggle enabled status

index_svc.update_document_enabled(document_uuid)

This method updates the document_enabled field for every segment belonging to that document in Weaviate, maintaining synchronization with the keyword search index.

Summary

• Deploy Weaviate via Docker Compose using the llmops-weaviate service defined in docker/docker-compose.yaml, ensuring you set WEAVIATE_HTTP_HOST, WEAVIATE_GRPC_HOST, WEAVIATE_API_KEY, and related port variables.
• Configure application defaults in api/config/default_config.py and runtime values in api/config/config.py so the Flask application can resolve the Weaviate endpoint and authentication credentials.
• Initialize the FlaskWeaviate extension from api/internal/extension/weaviate_extension.py to create a singleton client that supports dependency injection across services.
• Use VectorDatabaseService in api/internal/service/vector_database_service.py to wrap the raw client with LangChain's WeaviateVectorStore, targeting the Dataset collection for document storage.
• Implement retrieval via SemanticRetriever in api/internal/core/retrievers/semantic.py, which executes similarity_search_with_relevance_scores with filters for dataset_id, document_enabled, and segment_enabled to ensure accurate RAG context retrieval.

Frequently Asked Questions

What environment variables are required to configure Weaviate for RAG in LMForge?

You must define WEAVIATE_HTTP_HOST, WEAVIATE_HTTP_PORT, WEAVIATE_GRPC_HOST, WEAVIATE_GRPC_PORT, and WEAVIATE_API_KEY. These variables are injected into both the Weaviate container (for authentication) and the LMForge API/Celery services (for client connections), ensuring secure, end-to-end communication.

How does LMForge handle Weaviate authentication and security?

According to the source code in docker/docker-compose.yaml, Weaviate runs with AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'false' and AUTHENTICATION_APIKEY_ENABLED: 'true', requiring the WEAVIATE_API_KEY for every request. The Flask application reads this key from api/config/config.py and passes it to the FlaskWeaviate extension, ensuring that only authenticated services can perform CRUD operations on the vector store.

Which Weaviate collection name does LMForge use for storing RAG documents?

LMForge uses the collection name "Dataset" (defined as COLLECTION_NAME in api/internal/service/vector_database_service.py). This collection stores document segments with properties including text, dataset_id, document_id, segment_enabled, and document_enabled, which the SemanticRetriever filters during similarity searches to ensure only active, relevant chunks are returned for RAG contexts.

How do I perform a semantic search against the Weaviate vector store in LMForge?

Initialize the SemanticRetriever from api/internal/core/retrievers/semantic.py with your target dataset_ids and the VectorDatabaseService.vector_store instance, then call get_relevant_documents(query). This method executes similarity_search_with_relevance_scores with pre-configured filters for document_enabled and segment_enabled, returning ranked LangChain Document objects ready for injection into LLM prompts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}