AntSK Knowledge Base Retrieval with BGE Embeddings and Reranking: A Deep Dive into the Pipeline

AntSK retrieves knowledge-base answers by converting documents to dense vectors using BGE embedding models, indexing them in a vector store, performing similarity search, and optionally refining results with a BGE reranker before generating the final LLM response.

The AntSK open-source project implements a sophisticated retrieval-augmented generation (RAG) pipeline that leverages BGE (BAAI General Embedding) models for both document encoding and result reranking. This article examines the complete technical implementation, from memory initialization through final answer generation, based on the actual source code in the aidotnet/antsk repository.

Initializing the Kernel Memory Pipeline

The retrieval process begins with constructing a MemoryServerless instance configured for your specific application. When a user invokes an app, KMService.GetMemoryByApp orchestrates the setup by selecting the vector database and attaching the appropriate embedding generator.

The system supports multiple vector backends—including PostgreSQL, Disk, and Qdrant—configured through global KernelMemoryOption settings. The BGE embedding generator is attached via WithTextEmbeddingGenerationByAIType when the model's AIType is set to BgeEmbedding, as implemented in src/AntSK.Domain/Domain/Service/KMService.cs (lines 72-76).

case Model.Enum.AIType.BgeEmbedding:
    string pyDll = embedModel.EndPoint;
    string bgeEmbeddingModelName = embedModel.ModelName;
    memory.WithBgeTextEmbeddingGeneration(
        new HuggingfaceTextEmbeddingGenerator(pyDll, bgeEmbeddingModelName));
    break;

This configuration creates a HuggingfaceTextEmbeddingGenerator instance that wraps a Python-based BGE model, loading it once via BgeEmbeddingConfig.LoadModel (lines 24-48) and invoking model.embed_query to produce float[] embeddings (lines 37-41).

Document Indexing with BGE Embeddings

When users upload files to the knowledge base, the system checks for existing indexes before processing new documents. The ChatService.SendKmsByAppAsync method handles document import by chunking text and generating embeddings for each segment.

During the import phase, each text chunk passes through the BGE embedder, producing dense vectors that are stored in the configured vector database. The document metadata—including application ID and file ID tags—is preserved alongside the vectors for filtered retrieval.

await memory.ImportDocumentAsync(
    new Document(fileId).AddFile(filePath)
        .AddTag(KmsConstantcs.AppIdTag, app.Id)
        .AddTag(KmsConstantcs.FileIdTag, fileId),
    index: KmsConstantcs.FileIndex);

The HuggingfaceTextEmbeddingGenerator class in src/AntSK.Domain/Common/Embedding/HuggingfaceTextEmbeddingGenerator.cs serves as the C# wrapper that bridges the .NET runtime with the Python BGE model, ensuring efficient batch processing of document chunks.

Vector Similarity Search and Candidate Retrieval

For incoming queries, AntSK performs initial retrieval using Kernel Memory's similarity search capabilities. The KMService.GetRelevantSourceList method (lines 18-24) executes the search against the indexed documents, returning candidate passages ranked by vector similarity.

var searchResult = await memory.SearchAsync(
    msg,
    index: KmsConstantcs.KmsIndex,
    filters: filters);

The searchResult contains MemoryResult objects where each Partition holds the original text, a similarity score (Relevance), and associated metadata. This initial retrieval cast a wide net to ensure high recall, capturing potentially relevant passages that may include noise or false positives.

BGE Reranking for Precision Enhancement

To improve result quality, AntSK implements an optional BGE reranking stage when app.RerankModelID is configured. The system loads a specialized reranker model—such as bge-reranker-v2-miniCPM-layerwise—via BegRerankConfig.LoadModel (lines 24-49) in src/AntSK.Domain/Domain/Other/Bge/BegRerankConfig.cs.

For each candidate passage, the code constructs a query-passage pair and computes a refined relevance score:

List<string> rerank = new List<string> { questions, item.Text };
item.RerankScore = BegRerankConfig.Rerank(rerank);

The BegRerankConfig.Rerank method (lines 65-78) executes the Python model's compute_score function, returning a normalized float that better represents semantic relevance than raw vector similarity alone. Results are then reordered by RerankScore and trimmed to app.MaxMatchesCount, with only passages exceeding the Relevance threshold proceeding to the final generation stage.

Constructing the Final LLM Prompt

After filtering and reranking, the selected passages are concatenated into a context document (dataMsg) and injected into the application's prompt template (app.Prompt). The ChatService orchestrates this final stage, streaming the response back to the user interface.

var prompt = app.Prompt;
var doc = string.Join("\n", relevantSources.Select(s => s.ToString()));
var kernel = kernelService.GetKernelByApp(app);
var func = kernel.CreateFunctionFromPrompt(prompt,
    new OpenAIPromptExecutionSettings { Temperature = app.Temperature / 100 });

var result = kernel.InvokeStreamingAsync(func,
    new KernelArguments { ["doc"] = doc, ["input"] = query });

This implementation ensures that the LLM receives only the most relevant, reranked context, significantly improving answer accuracy while respecting token limits and relevance thresholds configured per application.

Summary

• AntSK uses Kernel Memory to orchestrate the entire retrieval pipeline, from embedding generation to vector storage and search.
• BGE embedding models encode both documents and queries into dense vectors via HuggingfaceTextEmbeddingGenerator, loaded through BgeEmbeddingConfig.
• Vector similarity search retrieves initial candidates using the configured vector database (Postgres, Qdrant, or Disk).
• Optional BGE reranking refines results using BegRerankConfig to compute cross-encoder scores, filtering to MaxMatchesCount based on Relevance thresholds.
• Filtered context is injected into the application prompt for final LLM response generation, ensuring high-precision answers grounded in the knowledge base.

Frequently Asked Questions

How does AntSK handle different vector databases with BGE embeddings?

AntSK abstracts vector database selection through KMService.WithMemoryDbByVectorDB, which configures the MemoryServerless instance based on global KernelMemoryOption settings. Whether using PostgreSQL, Qdrant, or local Disk storage, the BGE embedding generation remains consistent through the HuggingfaceTextEmbeddingGenerator wrapper, ensuring compatible vector dimensions across different backends.

What is the difference between the embedding model and reranker model in AntSK?

The embedding model (configured via AIType.BgeEmbedding) generates dense vector representations of text for initial similarity search in the vector database. The reranker model (configured via RerankModelID) is a cross-encoder that takes query-passage pairs and produces a refined relevance score. While embeddings enable fast approximate nearest neighbor search, the reranker provides slower but more accurate semantic relevance scoring for the top candidates.

Where does the actual Python BGE model execution happen in the codebase?

Python model execution occurs in the Domain/Other/Bge/ directory. BgeEmbeddingConfig.cs handles embedding model loading and embed_query invocation through modelscope, while BegRerankConfig.cs manages reranker loading and compute_score execution. The C# wrappers in HuggingfaceTextEmbeddingGenerator.cs call these Python functions via interop, loading the DLL specified in the model's EndPoint configuration.

Can AntSK work without the reranking stage?

Yes, reranking is optional. If app.RerankModelID is null or empty, the system skips the BegRerankConfig initialization and uses raw vector similarity scores from the initial search. In this mode, results are filtered and ordered by the Relevance field from MemoryResult.Partitions, then directly passed to the prompt construction stage without the additional cross-encoder scoring step.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}