# How AntSK Handles File Uploads and Document Processing: A Complete Pipeline Guide > **AntSK processes file uploads through a multi-stage pipeline that saves files to disk, validates them via KMService, queues background import tasks through BackgroundTaskBroker, and processes documents using KernelMemory with ... - Repository: [AIDotNet/antsk](https://github.com/aidotnet/antsk) - Tags: how-to-guide - Published: 2026-02-24 --- **AntSK processes file uploads through a multi-stage pipeline that saves files to disk, validates them via KMService, queues background import tasks through BackgroundTaskBroker, and processes documents using KernelMemory with configurable LLM embeddings and vector database storage.** The AntSK file upload and document processing pipeline is implemented in the `aidotnet/antsk` repository as a sophisticated workflow that bridges frontend uploads with backend AI document processing. This system handles everything from initial file validation to vector embedding generation, enabling semantic search capabilities across uploaded documents. ## File Upload and Storage The pipeline begins at the API layer with a dedicated endpoint for receiving multipart form data. In [`src/AntSK/Controllers/FileController.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Controllers/FileController.cs), the `UploadFile` method handles `POST /api/File/UploadFile` requests: ```csharp // FileController.cs implementation public async Task UploadFile(IFormFile file) { // Validation and storage logic } ``` The controller performs the following operations: - Validates the incoming `IFormFile` for null checks and size constraints - Constructs a storage path using `FileDirOption.DirectoryPath/files` as the base directory - Generates a unique filename using GUID-based naming to prevent collisions - Writes the file stream to disk asynchronously - Returns the physical path (relative to `uploads`) to the caller for downstream processing ## Client-Side Validation and Tracking Before files reach the background processing queue, the frontend validates and tracks them using `KMService`. Located in [`src/AntSK/Domain/Domain/Service/KMService.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Service/KMService.cs), this service provides three critical methods for the upload workflow: **Validation via `BeforeUpload`:** ```csharp public bool BeforeUpload(UploadFileItem file) { // Checks MIME type against allowed extensions // Validates file size < 100MB return true; // Only returns true for valid files } ``` **Completion Tracking via `OnSingleCompleted`:** When the AntDesign Upload component signals completion, this callback adds the file's name and URL to an internal `_fileList` collection. **Public Access via `FileList` Property:** ```csharp public List FileList { get; } ``` UI pages read this list to display uploaded files and enable the final import confirmation button. ## Background Task Queueing When the user confirms the upload on the KMS Detail page, the system transitions from synchronous file storage to asynchronous background processing. In [`src/AntSK/Pages/KmsPage/KmsDetail.razor.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Pages/KmsPage/KmsDetail.razor.cs), the `FileHandleOk` method iterates over the validated files: ```csharp foreach (var item in iKMService.FileList) { await _httpService.PostAsync( $"{NavigationManager.BaseUri}api/KMS/ImportKMSTask", new ImportKMSTaskDTO { ImportType = ImportType.File, KmsId = KmsId, FilePath = item.Url, FileName = item.FileName, IsQA = _isQa }); } ``` The `ImportKMSTaskDTO` (defined in [`src/AntSK/Domain/Domain/Model/ImportKMSTaskReq.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Model/ImportKMSTaskReq.cs)) encapsulates: - `ImportType`: File or URL - `KmsId`: Target knowledge base identifier - `FilePath`: Physical path from the upload step - `IsQA`: Boolean flag enabling QA-specific processing ## Document Processing Pipeline The core processing logic resides in `ImportKMSService`, orchestrated through the background task infrastructure. ### Task Routing 1. **KMSController.ImportKMSTask** receives the DTO and converts it to a request object 2. **BackgroundTaskBroker** enqueues the work 3. **BackGroundTaskHandler.ExecuteAsync** routes to `ImportKMSService.ImportKMSTask` ### KernelMemory Configuration Before processing begins, `KMService.GetMemoryByKMS` constructs a `KernelMemoryBuilder` with provider-specific configurations: ```csharp // Pseudocode representing the configuration logic var memory = new KernelMemoryBuilder() .WithTextGenerationByAIType(chatModel) // OpenAI, Azure, etc. .WithTextEmbeddingGenerationByAIType(embed) // BGE, DashScope, etc. .WithMemoryDbByVectorDB(vectorDb) // Postgres, Redis, Qdrant, etc. .Build(); ``` ### Document Import and Embedding Generation For file imports, `ImportKMSService.ImportKMSTask` executes: 1. **Memory Instance Retrieval**: Calls `_kMService.GetMemoryByKMS(km.Id)` to obtain the configured `MemoryServerless` instance 2. **Optional QA Pipeline**: When `req.IsQA` is true, the pipeline injects additional handlers: - `TextExtractionHandler` for content parsing - `QAHandler` for question-answer pair generation - `GenerateEmbeddingsHandler` for vector creation - `SaveRecordsHandler` for persistence 3. **Document Import**: Executes `_memory.ImportDocumentAsync` with a new `Document` instance: ```csharp await _memory.ImportDocumentAsync( new Document(fileid) .AddFile(req.FilePath) // Additional configuration based on import type ); ``` 4. **Statistics Collection**: Queries the memory store using `_kMService.GetDocumentByFileID(km.Id, fileid)` to count indexed partitions 5. **Metadata Persistence**: Updates the KMS detail record via `_kmsDetails_Repositories.Update(req.KmsDetail)` with: - Original filename - Generated file GUID - Data count (partition count) - Success status flag ## Summary - **FileController.UploadFile** in [`src/AntSK/Controllers/FileController.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Controllers/FileController.cs) handles initial file storage with GUID-based naming and returns physical paths - **KMService** provides client-side validation through `BeforeUpload` and tracks completed uploads in `FileList` for UI coordination - **KmsDetail.razor.cs** triggers background processing by posting `ImportKMSTaskDTO` objects to the KMS controller - **BackgroundTaskBroker** and **BackGroundTaskHandler** route tasks asynchronously to prevent UI blocking during document processing - **ImportKMSService.ImportKMSTask** orchestrates the core pipeline: building KernelMemory instances with configurable LLM and embedding providers, executing document import with optional QA handlers, generating embeddings, and persisting metadata to the KMS repository ## Frequently Asked Questions ### How does AntSK validate file uploads before processing? AntSK validates uploads at two levels. First, the **KMService.BeforeUpload** method in [`src/AntSK/Domain/Domain/Service/KMService.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Service/KMService.cs) checks MIME types against allowed extensions and enforces a 100MB size limit on the client side. Second, **FileController.UploadFile** performs server-side validation of the `IFormFile` object before writing the stream to disk with a GUID-based filename. ### What background task system does AntSK use for document processing? AntSK uses a custom **BackgroundTaskBroker** queueing system implemented in [`src/AntSK/Domain/Domain/Other/BackGroundTaskHandler.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Other/BackGroundTaskHandler.cs). When the UI calls `KMSController.ImportKMSTask`, the controller converts the DTO to a request and enqueues it. The **BackGroundTaskHandler.ExecuteAsync** method dequeues tasks and routes them to **ImportKMSService.ImportKMSTask**, allowing document processing to run asynchronously without blocking the web interface. ### How does AntSK configure AI models and vector databases for document embeddings? AntSK dynamically builds KernelMemory configurations through **KMService.GetMemoryByKMS** in [`src/AntSK/Domain/Domain/Service/KMService.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Service/KMService.cs). The method constructs a `KernelMemoryBuilder` using helper methods like **WithTextGenerationByAIType** (for LLMs such as OpenAI or Azure OpenAI), **WithTextEmbeddingGenerationByAIType** (for embeddings like BGE or DashScope), and **WithMemoryDbByVectorDB** (for vector stores including Postgres, Redis, Qdrant, or Azure AI Search). This configuration is then used by **ImportKMSService** to generate embeddings and store document partitions. ### What happens when the QA mode is enabled during file import? When **ImportKMSTaskDTO.IsQA** is set to true, **ImportKMSService.ImportKMSTask** in [`src/AntSK/Domain/Domain/Service/ImportKMSService.cs`](https://github.com/aidotnet/antsk/blob/main/src/AntSK/Domain/Domain/Service/ImportKMSService.cs) injects additional processing handlers into the KernelMemory pipeline. These include **TextExtractionHandler** for parsing document content, **QAHandler** for generating question-answer pairs from the text, **GenerateEmbeddingsHandler** for creating vector embeddings, and **SaveRecordsHandler** for persisting the results. This QA-specific pipeline enables the system to build searchable knowledge bases optimized for question-answering scenarios.