How the Knowledge Retrieval (RAG) System Works with the KnowledgeProvider in Heurist Agent Framework

The Heurist Agent Framework implements Retrieval-Augmented Generation (RAG) by using the KnowledgeProvider to query a vector store of embedded knowledge base entries and inject relevant context into the LLM system prompt before generation.

The Heurist Agent Framework is a modular system for building AI agents with persistent memory and domain-specific knowledge. Its knowledge retrieval (RAG) system combines dense vector search with prompt augmentation to ground LLM responses in external data sources.

Architecture Overview

The RAG pipeline follows a clear separation of concerns across four layers: embedding generation, vector storage, knowledge retrieval, and prompt augmentation. The KnowledgeProvider class acts as the primary interface between the vector store and the LLM workflow, while AugmentedLLMCall orchestrates the end-to-end process.

Core Components

Embedding Generation

All text vectorization in the framework flows through core.embedding.get_embedding. This utility converts user queries, conversation history, and knowledge base snippets into dense vectors for similarity comparison. Both the conversation manager and the knowledge provider rely on this shared embedding layer to ensure consistent vector spaces across all stored data.

Vector Storage

The MessageStore class provides a unified interface over pluggable storage backends. It delegates persistence to either PostgresVectorStorage or SQLiteVectorStorage, depending on configuration. All embeddings are stored alongside metadata including message_type (e.g., "knowledge_base", "user_message"), timestamps, and similarity indices.

KnowledgeProvider

Located in core/components/knowledge_provider.py, the KnowledgeProvider handles domain-specific context retrieval. Its primary method, get_knowledge_context, receives both the raw user message and its pre-computed embedding. It queries the MessageStore for records where message_type="knowledge_base" and cosine similarity exceeds 0.6.

knowledge_base_data = self.message_store.find_similar_messages(
    message_embedding,
    threshold=0.6,
    message_type="knowledge_base",
)

Matching entries are concatenated into a markdown-formatted "facts" block that gets appended to the system prompt.

The RAG Pipeline in Action

Query Embedding

When AugmentedLLMCall.process receives a user message, it first checks configuration flags (use_knowledge, use_similarity_lookup). If knowledge retrieval is enabled, it requests an embedding for the user query from the conversation manager, which delegates to core.embedding.get_embedding.

Similarity Search

The embedding vector is passed to KnowledgeProvider.get_knowledge_context, which executes the similarity search against the vector store. The current implementation uses a 0.6 cosine similarity threshold to filter out low-relevance knowledge base entries.

Context Injection

Retrieved knowledge is injected into the prompt within AugmentedLLMCall.process:

if options["use_knowledge"] and message_embedding:
    knowledge_context = await self.knowledge_provider.get_knowledge_context(
        message, message_embedding
    )
    system_prompt += f"\n\n{knowledge_context}"

The augmented prompt—now containing system instructions, retrieved facts, and conversation history—is sent to the LLM provider via self.llm_provider.call.

Updating the Knowledge Base

The framework supports dynamic knowledge base updates through KnowledgeProvider.update_knowledge_base. This method ingests JSON files, converts key-value pairs into readable text blocks, and embeds them for storage.

Duplicate detection is implemented with a 0.99 similarity threshold to prevent redundant entries:

# Pseudocode representing the duplicate check logic

existing = self.message_store.find_similar_messages(
    new_embedding, threshold=0.99, message_type="knowledge_base"
)
if not existing:
    self.message_store.store_message(new_embedding, metadata)

Implementation Example

The following example demonstrates a complete RAG workflow using the framework's core components:

# 1️⃣ Create a vector store (SQLite in this example)

from core.embedding import SQLiteVectorStorage, SQLiteConfig, MessageStore
storage = SQLiteVectorStorage(SQLiteConfig(db_path="embeddings.db"))
store = MessageStore(storage)

# 2️⃣ Initialise the providers

from core.components.knowledge_provider import KnowledgeProvider
from core.components.conversation_manager import ConversationManager
from core.components.llm_provider import LLMProvider   # (implementation elsewhere)

from core.components.tool_manager import ToolManager   # (implementation elsewhere)

knowledge = KnowledgeProvider(store)
conversation = ConversationManager(store)

# 3️⃣ Load a JSON knowledge file (optional)

await knowledge.update_knowledge_base("data/knowledge.json")

# 4️⃣ Set up the RAG workflow

from core.workflows.augmented_llm import AugmentedLLMCall
augmented = AugmentedLLMCall(
    knowledge_provider=knowledge,
    conversation_provider=conversation,
    tool_manager=ToolManager(),
    llm_provider=LLMProvider(),
)

# 5️⃣ Process a user query

response, image_url, tool_output = await augmented.process(
    message="What are the latest trends in DeFi?",
    system_prompt="You are a helpful finance assistant.",
    chat_id="user123",
)
print(response)

Summary

• The KnowledgeProvider in core/components/knowledge_provider.py serves as the primary interface for retrieving domain-specific context from the vector store.
• Embedding generation is centralized through core.embedding.get_embedding, ensuring consistent vector representations across all components.
• The vector store (MessageStore) supports both PostgreSQL and SQLite backends via pluggable storage classes.
• Similarity search uses a cosine similarity threshold of 0.6 to filter knowledge base entries, with retrieved content injected into the system prompt by AugmentedLLMCall.process.
• Knowledge base updates support JSON ingestion with duplicate detection at a 0.99 similarity threshold.

Frequently Asked Questions

What similarity threshold does the KnowledgeProvider use for retrieval?

The KnowledgeProvider.get_knowledge_context method applies a 0.6 cosine similarity threshold when querying the vector store. Only knowledge base entries exceeding this similarity score relative to the user query embedding are returned and injected into the prompt.

How does the KnowledgeProvider differ from the ConversationManager?

While both components use the same underlying MessageStore and embedding infrastructure, the KnowledgeProvider specifically handles domain-specific facts with message_type="knowledge_base", whereas the ConversationManager manages chat history and general message similarity lookups. The KnowledgeProvider is invoked conditionally based on the use_knowledge flag in AugmentedLLMCall.

Can I use PostgreSQL instead of SQLite for the vector store?

Yes. The MessageStore class abstracts storage details and can delegate to either PostgresVectorStorage or SQLiteVectorStorage. You initialize the appropriate backend (e.g., PostgresVectorStorage with connection parameters) and pass it to MessageStore to switch between SQL dialects without changing retrieval logic.

How does the framework prevent duplicate knowledge base entries?

The KnowledgeProvider.update_knowledge_base method implements duplicate detection by checking for existing records with a 0.99 cosine similarity threshold before inserting new embeddings. If a candidate entry matches an existing knowledge base record above this threshold, the insertion is skipped to maintain a clean, non-redundant vector store.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}