# How the Knowledge Retrieval (RAG) System Works with the KnowledgeProvider in Heurist Agent Framework

> Learn how the Heurist Agent Framework uses the KnowledgeProvider to enable Retrieval Augmented Generation RAG for efficient knowledge retrieval and LLM prompt injection.

- Repository: [Heurist/heurist-agent-framework](https://github.com/heurist-network/heurist-agent-framework)
- Tags: internals
- Published: 2026-03-03

---

**The Heurist Agent Framework implements Retrieval-Augmented Generation (RAG) by using the KnowledgeProvider to query a vector store of embedded knowledge base entries and inject relevant context into the LLM system prompt before generation.**

The Heurist Agent Framework is a modular system for building AI agents with persistent memory and domain-specific knowledge. Its knowledge retrieval (RAG) system combines dense vector search with prompt augmentation to ground LLM responses in external data sources.

## Architecture Overview

The RAG pipeline follows a clear separation of concerns across four layers: embedding generation, vector storage, knowledge retrieval, and prompt augmentation. The `KnowledgeProvider` class acts as the primary interface between the vector store and the LLM workflow, while `AugmentedLLMCall` orchestrates the end-to-end process.

## Core Components

### Embedding Generation

All text vectorization in the framework flows through `core.embedding.get_embedding`. This utility converts user queries, conversation history, and knowledge base snippets into dense vectors for similarity comparison. Both the conversation manager and the knowledge provider rely on this shared embedding layer to ensure consistent vector spaces across all stored data.

### Vector Storage

The `MessageStore` class provides a unified interface over pluggable storage backends. It delegates persistence to either `PostgresVectorStorage` or `SQLiteVectorStorage`, depending on configuration. All embeddings are stored alongside metadata including `message_type` (e.g., `"knowledge_base"`, `"user_message"`), timestamps, and similarity indices.

### KnowledgeProvider

Located in [`core/components/knowledge_provider.py`](https://github.com/heurist-network/heurist-agent-framework/blob/main/core/components/knowledge_provider.py), the `KnowledgeProvider` handles domain-specific context retrieval. Its primary method, `get_knowledge_context`, receives both the raw user message and its pre-computed embedding. It queries the `MessageStore` for records where `message_type="knowledge_base"` and cosine similarity exceeds **0.6**.

```python
knowledge_base_data = self.message_store.find_similar_messages(
    message_embedding,
    threshold=0.6,
    message_type="knowledge_base",
)

```

Matching entries are concatenated into a markdown-formatted "facts" block that gets appended to the system prompt.

## The RAG Pipeline in Action

### Query Embedding

When `AugmentedLLMCall.process` receives a user message, it first checks configuration flags (`use_knowledge`, `use_similarity_lookup`). If knowledge retrieval is enabled, it requests an embedding for the user query from the conversation manager, which delegates to `core.embedding.get_embedding`.

### Similarity Search

The embedding vector is passed to `KnowledgeProvider.get_knowledge_context`, which executes the similarity search against the vector store. The current implementation uses a **0.6 cosine similarity threshold** to filter out low-relevance knowledge base entries.

### Context Injection

Retrieved knowledge is injected into the prompt within `AugmentedLLMCall.process`:

```python
if options["use_knowledge"] and message_embedding:
    knowledge_context = await self.knowledge_provider.get_knowledge_context(
        message, message_embedding
    )
    system_prompt += f"\n\n{knowledge_context}"

```

The augmented prompt—now containing system instructions, retrieved facts, and conversation history—is sent to the LLM provider via `self.llm_provider.call`.

## Updating the Knowledge Base

The framework supports dynamic knowledge base updates through `KnowledgeProvider.update_knowledge_base`. This method ingests JSON files, converts key-value pairs into readable text blocks, and embeds them for storage.

Duplicate detection is implemented with a **0.99 similarity threshold** to prevent redundant entries:

```python

# Pseudocode representing the duplicate check logic

existing = self.message_store.find_similar_messages(
    new_embedding, threshold=0.99, message_type="knowledge_base"
)
if not existing:
    self.message_store.store_message(new_embedding, metadata)

```

## Implementation Example

The following example demonstrates a complete RAG workflow using the framework's core components:

```python

# 1️⃣ Create a vector store (SQLite in this example)

from core.embedding import SQLiteVectorStorage, SQLiteConfig, MessageStore
storage = SQLiteVectorStorage(SQLiteConfig(db_path="embeddings.db"))
store = MessageStore(storage)

# 2️⃣ Initialise the providers

from core.components.knowledge_provider import KnowledgeProvider
from core.components.conversation_manager import ConversationManager
from core.components.llm_provider import LLMProvider   # (implementation elsewhere)

from core.components.tool_manager import ToolManager   # (implementation elsewhere)

knowledge = KnowledgeProvider(store)
conversation = ConversationManager(store)

# 3️⃣ Load a JSON knowledge file (optional)

await knowledge.update_knowledge_base("data/knowledge.json")

# 4️⃣ Set up the RAG workflow

from core.workflows.augmented_llm import AugmentedLLMCall
augmented = AugmentedLLMCall(
    knowledge_provider=knowledge,
    conversation_provider=conversation,
    tool_manager=ToolManager(),
    llm_provider=LLMProvider(),
)

# 5️⃣ Process a user query

response, image_url, tool_output = await augmented.process(
    message="What are the latest trends in DeFi?",
    system_prompt="You are a helpful finance assistant.",
    chat_id="user123",
)
print(response)

```

## Summary

- The **KnowledgeProvider** in [`core/components/knowledge_provider.py`](https://github.com/heurist-network/heurist-agent-framework/blob/main/core/components/knowledge_provider.py) serves as the primary interface for retrieving domain-specific context from the vector store.
- **Embedding generation** is centralized through `core.embedding.get_embedding`, ensuring consistent vector representations across all components.
- The **vector store** (`MessageStore`) supports both PostgreSQL and SQLite backends via pluggable storage classes.
- **Similarity search** uses a cosine similarity threshold of **0.6** to filter knowledge base entries, with retrieved content injected into the system prompt by `AugmentedLLMCall.process`.
- **Knowledge base updates** support JSON ingestion with duplicate detection at a **0.99** similarity threshold.

## Frequently Asked Questions

### What similarity threshold does the KnowledgeProvider use for retrieval?

The `KnowledgeProvider.get_knowledge_context` method applies a **0.6 cosine similarity threshold** when querying the vector store. Only knowledge base entries exceeding this similarity score relative to the user query embedding are returned and injected into the prompt.

### How does the KnowledgeProvider differ from the ConversationManager?

While both components use the same underlying `MessageStore` and embedding infrastructure, the **KnowledgeProvider** specifically handles domain-specific facts with `message_type="knowledge_base"`, whereas the **ConversationManager** manages chat history and general message similarity lookups. The KnowledgeProvider is invoked conditionally based on the `use_knowledge` flag in `AugmentedLLMCall`.

### Can I use PostgreSQL instead of SQLite for the vector store?

Yes. The `MessageStore` class abstracts storage details and can delegate to either `PostgresVectorStorage` or `SQLiteVectorStorage`. You initialize the appropriate backend (e.g., `PostgresVectorStorage` with connection parameters) and pass it to `MessageStore` to switch between SQL dialects without changing retrieval logic.

### How does the framework prevent duplicate knowledge base entries?

The `KnowledgeProvider.update_knowledge_base` method implements duplicate detection by checking for existing records with a **0.99 cosine similarity threshold** before inserting new embeddings. If a candidate entry matches an existing knowledge base record above this threshold, the insertion is skipped to maintain a clean, non-redundant vector store.