How ChatService Implements Streaming Responses with Function Calling in AntSK

The ChatService leverages IAsyncEnumerable<string> to stream LLM responses in real-time while automatically detecting and executing function calls through Semantic Kernel's tool-calling infrastructure, seamlessly interleaving text chunks with automated function invocation loops.

The aidotnet/antsk repository provides a robust AI chat framework built on .NET and Semantic Kernel. The ChatService (ChatService.cs) implements a sophisticated dual-mode streaming architecture that handles both pure text generation and complex function-calling workflows without blocking the UI thread.

Kernel Setup and Function Detection

Before initiating any streaming, the service configures a per-application Kernel instance and determines whether the current app requires function-calling capabilities.

The process begins in SendChatByAppAsync by retrieving the kernel via _kernelService.GetKernelByApp(app) and resolving the IChatCompletionService through kernel.GetRequiredService<IChatCompletionService>(). The service then inspects the app's configuration to detect available functions:

• API functions (app.ApiFunctionList)
• Native functions (app.NativeFunctionList)

If either list contains entries, the service imports these functions into the kernel using _kernelService.ImportFunctionsByApp(app, kernel). It also configures the execution settings to enable tool calling:

var settings = new OpenAIPromptExecutionSettings
{
    Temperature = app.Temperature / 100,
    ToolCallBehavior = app.HasFunctions 
        ? ToolCallBehavior.EnableKernelFunctions 
        : null
};

This configuration allows the Semantic Kernel to recognize when the LLM emits a tool call request and route it to the appropriate registered function.

Streaming Loop with Function Call Handling

When function calling is enabled, the service enters a while (true) loop that manages the conversation state machine. This loop handles the asynchronous exchange between the LLM and the local function registry until a final text response is produced.

The core logic resides in src/AntSK.Domain/Domain/Service/ChatService.cs (lines 39-78). Each iteration calls chat.GetChatMessageContentAsync(history, settings, kernel) to await the model's next output. The service then evaluates the response type:

Normal text chunks are detected when result.Content != null. The service immediately yields this content to the caller via yield return result.Content.ConvertToString() and breaks the loop, signaling completion.

Function calls are detected using FunctionCallContent.GetFunctionCalls(result). When tool calls are present, the service:

1. Appends the assistant's tool request to the conversation history (history.Add(result))
2. Iterates through each functionCall and executes it asynchronously via await functionCall.InvokeAsync(kernel)
3. Converts the FunctionResultContent back into a chat message using resultContent.ToChatMessage()
4. Appends the function result to the history and continues the loop

This cycle repeats until the model incorporates all function results and generates a final textual response, ensuring the UI receives a coherent, complete answer rather than raw tool outputs.

Pure Streaming Path for Text-Only Responses

For applications without defined functions (app.HasFunctions == false), the service bypasses the state machine loop and uses the native streaming API directly. This path provides lower latency and reduced memory overhead for simple conversational scenarios.

The implementation calls chat.GetStreamingChatMessageContentsAsync(history, settings, kernel) and yields each token as it arrives from the LLM provider:

await foreach (var token in chat.GetStreamingChatMessageContentsAsync(history, settings, kernel))
{
    yield return token.ConvertToString();
}

This approach streams individual tokens or segments immediately to the consumer, minimizing perceived response time in the Blazor frontend.

Interface Contract and UI Integration

The IChatService interface in src/AntSK.Domain/Domain/Interface/IChatService.cs declares the streaming contract:

IAsyncEnumerable<string> SendChatByAppAsync(Apps app, ChatHistory history);

Consumers, such as the ChatView.razor.cs component, iterate over this async enumerable to render responses in real-time. The UI component processes each chunk as it arrives, updating the message list and triggering StateHasChanged() to refresh the Blazor component without waiting for the entire response:

await foreach (var chunk in _chatService.SendChatByAppAsync(app, history))
{
    MessageList.Add(new Chats { IsSend = false, Context = chunk });
    StateHasChanged();
}

This separation of concerns ensures that ChatService handles the complex orchestration of kernel functions and streaming protocols, while the view layer focuses solely on presentation.

Summary

• Dual-mode architecture: The service automatically selects between a function-calling state machine and pure token streaming based on app.HasFunctions.
• Automatic function invocation: Tool calls are executed via FunctionCallContent.GetFunctionCalls() and InvokeAsync(), with results automatically re-injected into the conversation history.
• Non-blocking streaming: Both paths return IAsyncEnumerable<string>, allowing the Blazor UI to render content incrementally without freezing the main thread.
• Kernel integration: Function imports and tool-call behaviors are managed through the Semantic Kernel's standard OpenAIPromptExecutionSettings and ToolCallBehavior.EnableKernelFunctions.

Frequently Asked Questions

How does ChatService handle multiple consecutive function calls?

The while (true) loop in ChatService.cs continues iterating until the model returns a response with non-null content. Each iteration can process multiple function calls detected via FunctionCallContent.GetFunctionCalls(result), executing them in parallel via await functionCall.InvokeAsync(kernel) and appending all results to the history before requesting the next completion. This supports complex multi-step reasoning where the LLM must call several tools to formulate a final answer.

What happens if the application has no functions defined?

When app.HasFunctions is false, the service skips the import and tool-call configuration steps, entering the pure streaming path that uses GetStreamingChatMessageContentsAsync. This yields each token immediately through IAsyncEnumerable<string> without the overhead of the function-calling state machine, providing optimal performance for simple chat scenarios.

Can the streaming implementation handle concurrent chat sessions?

Yes. The service creates a distinct kernel instance per application via _kernelService.GetKernelByApp(app), ensuring that function imports and conversation history remain isolated between different apps or sessions. The IAsyncEnumerable pattern naturally supports concurrent consumers, as each call to SendChatByAppAsync generates an independent async stream that does not interfere with other active streams.

How are function results formatted before being sent back to the model?

After invoking a function via call.InvokeAsync(kernel), the service converts the resulting FunctionResultContent object into a standard chat message using the ToChatMessage() extension method. This message is then appended to the ChatHistory instance, allowing the Semantic Kernel to serialize it into the appropriate tool-result format required by the LLM provider's API specification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}