# How the Function Calling Strategy Handles Streaming vs Blocking LLM Responses

> Understand how Function Calling strategy processes streaming vs blocking LLM responses. It checks STREAM_TOOL_CALL for incremental or complete result handling in tool call extraction.

- Repository: [Junjie.M/dify-plugin-agent-mcp_sse](https://github.com/junjiem/dify-plugin-agent-mcp_sse)
- Tags: how-to-guide
- Published: 2026-03-05

---

**The Function Calling strategy checks for the `STREAM_TOOL_CALL` model feature to determine whether to process LLM outputs as a streaming generator (yielding tokens incrementally) or a blocking response (processing the complete result at once), handling tool call extraction accordingly in [`strategies/function_calling.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.py).**

The `junjiem/dify-plugin-agent-mcp_sse` repository implements a Function Calling agent strategy that dynamically adapts to different LLM response modes. This capability allows the agent to optimize for either low-latency streaming outputs or complete blocking responses when executing tool calls. Understanding how the strategy handles these two modes is essential for developers configuring MCP (Model Context Protocol) agents in Dify workflows.

## Detecting the Response Mode with STREAM_TOOL_CALL

The strategy determines the response mode by inspecting the model's declared capabilities before invoking the LLM. In [`strategies/function_calling.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.py), the code checks for the `ModelFeature.STREAM_TOOL_CALL` feature flag at lines 106-110.

If the model supports streaming tool calls, the `stream` parameter is set to `True`. Otherwise, the strategy defaults to blocking mode with `stream` set to `False`. This boolean flag is then passed to the LLM invocation method at lines 167-175, determining whether the underlying model client returns a generator or a complete result object.

## Processing Streaming LLM Responses

When `stream` is enabled, the LLM returns a generator yielding `LLMResultChunk` objects. The strategy processes these incrementally in the `_invoke` method of `FunctionCallingAgentStrategy`.

For each chunk, the strategy performs the following actions:

- **Tool call detection**: Uses `self.check_tool_calls(chunk)` at lines 71-75 to identify if the current chunk contains function call signatures.
- **Tool call extraction**: Invokes `self.extract_tool_calls(chunk)` at lines 88-94 to parse the tool call parameters from the streaming delta.
- **Incremental text delivery**: Appends the textual delta to the accumulating response and yields intermediate text messages immediately using `yield self.create_text_message()` at lines 98-106, providing real-time feedback to the user.

This approach minimizes latency by emitting tokens as they arrive while simultaneously monitoring for tool invocations that may interrupt the text generation.

## Processing Blocking LLM Responses

When streaming is not supported, the LLM returns a complete `LLMResult` object containing the full response. The strategy handles this in the blocking branch of the `_invoke` method.

The blocking processing flow involves:

- **Complete result analysis**: Receives the full `LLMResult` object instead of a generator.
- **Tool call detection**: Calls `self.check_blocking_tool_calls(result)` at lines 77-81 to scan the entire response for function calls.
- **Tool call extraction**: Uses `self.extract_blocking_tool_calls(result)` at lines 84-90 to retrieve all tool call parameters from the complete message.
- **Single message emission**: Yields the entire textual response in one operation at lines 124-132 using `yield self.create_text_message()`, ensuring atomic delivery of the complete thought.

This mode simplifies tool call handling by processing the entire context at once but sacrifices the progressive rendering benefits of streaming.

## Implementation Details and Code Structure

The dual-mode handling is implemented in [`strategies/function_calling.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.py) within the `FunctionCallingAgentStrategy` class. The core logic resides in the `_invoke` method, which branches based on whether `chunks` is a generator or a complete result.

**Streaming detection logic (lines 106-110):**

```python
if ModelFeature.STREAM_TOOL_CALL in features:
    stream = True
else:
    stream = False

```

**LLM invocation with mode flag (lines 167-175):**

```python
chunks = self.session.model.llm.invoke(
    model_configured,
    prompt_messages,
    stream=stream,
    **invoke_params
)

```

**Streaming processing branch (simplified from lines 88-106):**

```python
if isinstance(chunks, Generator):
    for chunk in chunks:
        if self.check_tool_calls(chunk):
            tool_calls.extend(self.extract_tool_calls(chunk))
        if chunk.delta.message.content:
            yield self.create_text_message(str(chunk.delta.message.content))

```

**Blocking processing branch (simplified from lines 77-90, 124-132):**

```python
else:
    result = cast(LLMResult, chunks)
    if self.check_blocking_tool_calls(result):
        tool_calls.extend(self.extract_blocking_tool_calls(result))
    if result.message.content:
        yield self.create_text_message(str(result.message.content))

```

Supporting files include [`strategies/function_calling.yaml`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.yaml) for declarative configuration, [`utils/mcp_client.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/utils/mcp_client.py) for MCP server interactions, and [`strategies/base.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/base.py) for shared history management utilities.

## Summary

The Function Calling strategy in `junjiem/dify-plugin-agent-mcp_sse` implements a sophisticated dual-mode architecture for handling LLM responses:

- **Capability-based detection**: Automatically selects streaming or blocking mode by checking for the `STREAM_TOOL_CALL` model feature before invocation.
- **Real-time streaming**: Processes `LLMResultChunk` objects incrementally, yielding text tokens immediately while extracting tool calls from partial deltas.
- **Atomic blocking**: Handles complete `LLMResult` objects, scanning the full response for tool calls before emitting the entire message.
- **Unified tool handling**: Both paths converge on the same tool execution logic after response processing, ensuring consistent behavior regardless of transmission mode.

This design allows the agent to optimize for latency with compatible models while maintaining compatibility with blocking-only LLM providers.

## Frequently Asked Questions

### How does the strategy determine whether to use streaming or blocking mode?

The strategy inspects the model's feature list for `ModelFeature.STREAM_TOOL_CALL` at lines 106-110 in [`strategies/function_calling.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.py). If the feature is present, it sets `stream=True` for the LLM invocation; otherwise, it defaults to `stream=False` for blocking mode.

### What happens to tool calls when using streaming responses?

During streaming, the strategy processes each `LLMResultChunk` incrementally. It calls `check_tool_calls()` and `extract_tool_calls()` on every chunk to detect function signatures as they arrive. Tool calls are accumulated across chunks, and text deltas are yielded immediately unless a tool call is pending.

### Can the Function Calling strategy handle models that don't support streaming tool calls?

Yes. The strategy explicitly supports blocking mode for models lacking the `STREAM_TOOL_CALL` feature. In this mode, it receives a complete `LLMResult` object and uses `check_blocking_tool_calls()` and `extract_blocking_tool_calls()` to parse all tool calls from the finished response before yielding the full text.

### Where is the core logic for switching between streaming and blocking implemented?

The dual-mode logic resides in the `_invoke` method of the `FunctionCallingAgentStrategy` class within [`strategies/function_calling.py`](https://github.com/junjiem/dify-plugin-agent-mcp_sse/blob/main/strategies/function_calling.py). Lines 88-132 contain the branching logic that determines whether to iterate over a generator (streaming) or cast to `LLMResult` (blocking), with specific helper methods handling each mode's tool call extraction requirements.