How Kernel Memory Handles Document Ingestion and Search in AntSK

AntSK leverages Microsoft Kernel Memory (KM) to create isolated, per-knowledge-base memory instances that ingest documents through configurable pipelines and perform filtered vector searches using KMS tags, enabling precise retrieval-augmented generation workflows.

AntSK integrates Microsoft Kernel Memory as its core engine for knowledge management, providing a robust architecture for document processing and semantic search. This article examines the technical implementation of Kernel Memory document ingestion and search in AntSK within the aidotnet/antsk repository, demonstrating how the platform orchestrates vector storage, text extraction, and relevance-based retrieval through dedicated service classes.

Building MemoryServerless Instances per Knowledge Base

AntSK creates dedicated MemoryServerless instances for each knowledge base (KMS) through the KMService.GetMemoryByKMS method. This approach ensures complete isolation between different knowledge bases while allowing customized configurations for text generation models, embedding models, vector stores, and optional OCR capabilities.

The builder pattern configures the pipeline through provider-specific extension methods:

var memoryBuilder = new KernelMemoryBuilder()
    .WithSearchClientConfig(searchClientConfig);

WithTextGenerationByAIType(memoryBuilder, chatModel, chatHttpClient);
WithTextEmbeddingGenerationByAIType(memoryBuilder, embedModel, embeddingHttpClient);
WithMemoryDbByVectorDB(memoryBuilder);

if (kms.IsOCR == 1) 
    memoryBuilder.WithCustomImageOcr(new AntSKOcrEngine());

_memory = memoryBuilder
    .AddSingleton<IKernelService>(_kernelService)
    .Build<MemoryServerless>();

Source: KMService.GetMemoryByKMS in src/AntSK.Domain/Domain/Service/KMService.cs (lines 71-89)

The configuration supports multiple AI providers including OpenAI, Azure OpenAI, Bge embeddings, and DashScope, while vector storage options include Postgres, Disk, and Qdrant as defined in appsettings.json.

Document Ingestion Pipeline

Document processing occurs through ImportKMSService.ImportKMSTask, which handles files, URLs, plain text, and Excel sources. Each document receives a unique file identifier and the KmsIdTag to enable later filtering.

Standard Import Process

For standard document imports, the service invokes ImportDocumentAsync, ImportWebPageAsync, or ImportTextAsync on the memory instance:

var importResult = await _memory.ImportDocumentAsync(
    new Document(fileId)
        .AddFile(req.FilePath)
        .AddTag(KmsConstantcs.KmsIdTag, req.KmsId),
    index: KmsConstantcs.KmsIndex);

Source: ImportKMSService.ImportKMSTask in src/AntSK.Domain/Domain/Service/ImportKMSService.cs (lines 44-140)

QA-Enabled Ingestion with Custom Steps

When QA processing is enabled, AntSK injects custom orchestrator steps to control the pipeline execution. This configuration extracts text, generates embeddings using the specific chat model, and saves memory records in sequence:

var steps = new[] { 
    "extract_text", 
    kms.ChatModelID, 
    "generate_embeddings", 
    "save_memory_records" 
};

var importResult = await _memory.ImportDocumentAsync(
    new Document(fileId)
        .AddFile(req.FilePath)
        .AddTag(KmsConstantcs.KmsIdTag, req.KmsId),
    index: KmsConstantcs.KmsIndex,
    steps: steps);

Source: File branch with QA flag in src/AntSK.Domain/Domain/Service/ImportKMSService.cs (lines 48-58)

Search Operations and Retrieval

AntSK performs semantic searches using the SearchAsync method with KMS-specific filters, ensuring results originate only from the designated knowledge base.

Filtered Vector Search by KMS Tags

The KMService.GetRelevantSourceList method constructs memory filters using the KmsIdTag to scope searches to specific knowledge bases. This prevents cross-contamination between different KMS instances:

var filters = kmsIdList
    .Select(id => new MemoryFilter()
        .ByTag(KmsConstantcs.KmsIdTag, id))
    .ToList();

var searchResult = await memory.SearchAsync(
    query, 
    index: KmsConstantcs.KmsIndex, 
    filters: filters);

foreach (var item in searchResult.Results)
{
    foreach (var part in item.Partitions)
    {
        // Access part.Text and part.Relevance
        var source = new RelevantSource {
            SourceName = item.SourceName,
            Text = part.Text,
            Relevance = part.Relevance
        };
    }
}

Source: KMService.GetRelevantSourceList in src/AntSK.Domain/Domain/Service/KMService.cs (lines 308-334)

Retrieving Document Fragments by File ID

To retrieve all stored partitions of a specific document, KMService.GetDocumentByFileID enumerates all indexes and underlying memory databases, querying with MemoryFilter().ByDocument(fileId):

var memories = await memory.ListIndexesAsync();
var dbs = memory.Orchestrator.GetMemoryDbs();

foreach (var idx in memories)
{
    foreach (var db in dbs)
    {
        var items = await db.GetListAsync(
            idx.Name,
            new List<MemoryFilter> { 
                new MemoryFilter().ByDocument(fileId) 
            },
            limit: 1000, 
            withEmbeddings: true).ToListAsync();
            
        // items contain DocumentId, Text, Url, LastUpdate, File
    }
}

Source: KMService.GetDocumentByFileID in src/AntSK.Domain/Domain/Service/KMService.cs (lines 82-102)

Summary

• AntSK creates isolated MemoryServerless instances per knowledge base via KMService.GetMemoryByKMS, supporting multiple AI providers and vector stores.
• Document ingestion uses ImportKMSService with optional QA pipeline steps (extract_text, generate_embeddings, save_memory_records) and mandatory KmsIdTag tagging.
• Search operations filter by KMS tags using MemoryFilter().ByTag() to ensure knowledge base isolation, implemented in KMService.GetRelevantSourceList.
• Fragment retrieval enumerates indexes and memory databases to reconstruct documents by file ID through KMService.GetDocumentByFileID.

Frequently Asked Questions

How does AntSK isolate different knowledge bases in Kernel Memory?

AntSK isolates knowledge bases by tagging every ingested document with a KmsIdTag containing the knowledge base identifier. During search operations, the system constructs MemoryFilter objects using ByTag(KmsIdTag, kmsId) to restrict results to specific KMS instances. Additionally, separate MemoryServerless instances can be built per knowledge base with distinct embedding models and vector stores.

What vector databases does AntSK support for Kernel Memory storage?

According to the source code in KMService.WithMemoryDbByVectorDB, AntSK supports multiple vector storage backends including Postgres, Disk (local file system), and Qdrant. The selection is configured in appsettings.json under the KernelMemory section's VectorDb property.

Can AntSK process images and PDFs with OCR during ingestion?

Yes, when kms.IsOCR == 1, the KMService.GetMemoryByKMS method configures the memory builder with WithCustomImageOcr(new AntSKOcrEngine()). This enables text extraction from images and scanned PDFs during the document ingestion pipeline, integrating OCR capabilities directly into the Kernel Memory workflow.

What is the difference between GetMemoryByKMS and GetMemoryByApp in AntSK?

GetMemoryByKMS builds a knowledge-base-scoped memory instance configured with OCR support and specific KMS settings, used for document storage and retrieval. GetMemoryByApp creates a session-scoped memory using the application's chat and embedding models without OCR, primarily for conversational contexts and plugin integration via KernelService.ImportFunctionsByApp.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}