# How to Build LLM Function Calling Agents in Dify: A Complete Technical Guide

> Master LLM function calling agents in Dify with this technical guide. Learn the reusable pattern using FunctionCallAgentRunner to build powerful, iterative agents.

- Repository: [LangGenius/dify](https://github.com/langgenius/dify)
- Tags: how-to-guide
- Published: 2026-02-25

---

**Dify implements LLM function calling agents through a reusable architectural pattern centered on `FunctionCallAgentRunner`, which executes a continuous loop to detect tool calls, invoke corresponding tools, and iterate until the model returns a final answer.**

The `langgenius/dify` repository provides a production-ready framework for building function calling agents that transform Dify-defined tools into OpenAI-compatible schemas. By configuring the `function_call` strategy and leveraging the `ParameterExtractorNode`, developers can create robust agents that seamlessly invoke external APIs, calculators, or knowledge base retrievers.

## Architecture Overview of Function Calling Agents in Dify

Dify's function calling implementation spans four distinct layers, each handled by specific modules in the codebase.

### The Agent Runner Layer

The `FunctionCallAgentRunner` class in [`api/core/agent/fc_agent_runner.py`](https://github.com/langgenius/dify/blob/main/api/core/agent/fc_agent_runner.py) orchestrates the entire execution flow. It maintains a `function_call_state` boolean that controls the iteration loop. When the LLM returns a response containing `tool_calls`, the runner extracts the tool name and arguments, invokes the corresponding tool, and feeds the observation back into the next prompt iteration.

### Tool Conversion and Registration

`BaseAgentRunner._init_prompt_tools` in [`api/core/agent/base_agent_runner.py`](https://github.com/langgenius/dify/blob/main/api/core/agent/base_agent_runner.py) handles the transformation of Dify `Tool` objects into `PromptMessageTool` instances following the OpenAI function schema. This method iterates over `self.app_config.agent.tools`, converting each tool via `_convert_tool_to_prompt_message_tool` and storing the original tool instances in `tool_instances` for later execution. Dataset retrievers receive special handling through `_convert_dataset_retriever_tool_to_prompt_message_tool`.

### Prompt Generation for Function Calling

The `ParameterExtractorNode._generate_function_call_prompt` method in [`api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py`](https://github.com/langgenius/dify/blob/main/api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py) constructs the system prompt that instructs the model to use function calling. It embeds the user query into `FUNCTION_CALLING_EXTRACTOR_USER_TEMPLATE`, injects a JSON schema describing expected parameters via `node_data.get_parameter_json_schema()`, and appends few-shot examples from `FUNCTION_CALLING_EXTRACTOR_EXAMPLE`.

### Multi-Dataset Routing

When agents need to query multiple knowledge bases, `FunctionCallMultiDatasetRouter` in [`api/core/rag/retrieval/router/multi_dataset_function_call_router.py`](https://github.com/langgenius/dify/blob/main/api/core/rag/retrieval/router/multi_dataset_function_call_router.py) determines which dataset ID should handle a specific retrieval request. This enables a single function calling agent to transparently route queries to different knowledge bases based on the function parameters.

## Implementing the Function Calling Agent Loop

The core execution pattern relies on a stateful loop that continues until the model stops requesting tool executions.

```python

# Conceptual flow from api/core/agent/fc_agent_runner.py

function_call_state = True
iteration_count = 0
max_iteration = self.app_config.agent.max_iteration

while function_call_state and iteration_count < max_iteration:
    # 1. Build prompt with current history and tool definitions

    prompt_messages = self._organize_prompt_messages()
    
    # 2. Invoke LLM with tools

    response = model_instance.invoke_llm(
        prompt_messages=prompt_messages,
        tools=prompt_messages_tools,
        stream=True
    )
    
    # 3. Check for tool calls in response chunks

    tool_calls = self.check_tool_calls(response)  # or check_blocking_tool_calls

    
    if tool_calls:
        function_call_state = True
        # 4. Extract tool call details

        for tool_call in tool_calls:
            tool_id = tool_call.id
            tool_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            
            # 5. Execute tool and get observation

            observation = self._handle_tool_invoke(
                tool_name=tool_name,
                tool_inputs=arguments
            )
            
            # 6. Add observation to message history for next iteration

            self._append_tool_output_to_messages(tool_id, observation)
    else:
        function_call_state = False
    
    iteration_count += 1

# Force final answer if max iterations reached

if iteration_count >= max_iteration:
    prompt_messages_tools = []  # Remove tools to force direct answer

```

## Configuring Tools for Function Calling Agents

Tools must be defined, converted to OpenAI-compatible schemas, and registered in the application configuration.

### Defining Custom Tools

Create tool definitions using Dify's `Tool` entity class:

```python

# my_tool.py

from core.tools.entities.tool_entities import Tool, ToolParameter

my_tool = Tool(
    tool_name="calculate_sum",
    description="Returns the sum of two numbers.",
    parameters=[
        ToolParameter(
            name="a", 
            type=ToolParameter.ToolParameterType.NUMBER, 
            required=True
        ),
        ToolParameter(
            name="b", 
            type=ToolParameter.ToolParameterType.NUMBER, 
            required=True
        ),
    ],
)

```

### Registering Tools in App Configuration

Declare the function calling strategy and tool list in your application configuration:

```yaml

# app_config.yaml

agent:
  strategy: function_call        # Instantiates FunctionCallAgentRunner

  max_iteration: 5
  tools:
    - tool_name: calculate_sum
    - tool_name: weather_lookup
    - tool_name: dataset_retriever  # Knowledge base tool

```

### Dataset Retrievers as Tools

Dataset retrievers are automatically converted to function calling tools via `_convert_dataset_retriever_tool_to_prompt_message_tool` in `BaseAgentRunner`. This allows the agent to query knowledge bases using standard function call syntax:

```json
{
  "name": "dataset_retriever",
  "arguments": "{\"query\": \"latest product specifications\", \"dataset_id\": \"prod_docs_2024\"}"
}

```

## Building Workflows with ParameterExtractorNode

For workflow-based applications, use `ParameterExtractorNode` to leverage function calling for structured data extraction:

```python

# workflow.py

from api.core.workflow.nodes.parameter_extractor.parameter_extractor_node import ParameterExtractorNode

# Create extraction node using function calling reasoning

extractor = ParameterExtractorNode(
    node_id="extract_sum",
    name="Sum Extractor",
    type="parameter_extractor",
    reasoning_mode="function_call",  # Triggers FunctionCallAgentRunner

    tools=["calculate_sum"],
    instruction="Extract two numbers from the user query and return their sum.",
    model_config={
        "provider": "openai",
        "model": "gpt-4"
    }
)

# Execute workflow

result = extractor.run(query="What is 12 plus 30?")
print(result.outputs)   # => {"sum": 42}

```

## Key Source Files for Function Calling Agents in Dify

Understanding the implementation requires familiarity with these specific files:

- **[`api/core/agent/fc_agent_runner.py`](https://github.com/langgenius/dify/blob/main/api/core/agent/fc_agent_runner.py)** — Contains `FunctionCallAgentRunner`, the main execution loop that handles tool call detection and iteration logic.

- **[`api/core/agent/base_agent_runner.py`](https://github.com/langgenius/dify/blob/main/api/core/agent/base_agent_runner.py)** — Defines `BaseAgentRunner` with `_init_prompt_tools` for converting Dify tools to OpenAI-compatible schemas and managing tool instances.

- **[`api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py`](https://github.com/langgenius/dify/blob/main/api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py)** — Implements `ParameterExtractorNode` with `_generate_function_call_prompt` for creating system prompts with JSON schemas and few-shot examples.

- **[`api/core/rag/retrieval/router/multi_dataset_function_call_router.py`](https://github.com/langgenius/dify/blob/main/api/core/rag/retrieval/router/multi_dataset_function_call_router.py)** — Houses `FunctionCallMultiDatasetRouter` for routing function calls to specific knowledge base datasets.

- **[`api/commands.py`](https://github.com/langgenius/dify/blob/main/api/commands.py)** — Contains configuration examples showing `"strategy": "function_call"` declarations.

- **[`api/core/tools/tool_engine.py`](https://github.com/langgenius/dify/blob/main/api/core/tools/tool_engine.py)** — Executes concrete tool logic after function calls are extracted from LLM responses.

## Summary

- **Function calling agents in Dify** rely on `FunctionCallAgentRunner` to execute a continuous loop that detects tool calls, invokes tools, and feeds observations back to the LLM until completion.

- **Tool conversion** happens in `BaseAgentRunner._init_prompt_tools`, which transforms Dify `Tool` entities into OpenAI-compatible `PromptMessageTool` schemas, including special handling for dataset retrievers.

- **Configuration** requires setting `agent.strategy` to `"function_call"` in the application configuration, along with a defined list of available tools and maximum iteration limits.

- **Workflow integration** uses `ParameterExtractorNode` with `reasoning_mode="function_call"` to leverage structured extraction capabilities with JSON schemas and few-shot prompting.

- **Multi-dataset retrieval** is supported through `FunctionCallMultiDatasetRouter`, enabling agents to query specific knowledge bases based on function parameters.

## Frequently Asked Questions

### What is the difference between function calling and CoT agents in Dify?

**Function calling agents** use the `FunctionCallAgentRunner` which relies on the LLM's native function calling capability to emit structured JSON tool calls, while **Chain-of-Thought (CoT) agents** typically use reasoning prompts without structured function schemas. Function calling agents are configured via `"strategy": "function_call"` and provide more reliable tool invocation through explicit schema definitions, whereas CoT agents may use `"strategy": "cot"` or similar configurations depending on the specific implementation in `api/core/agent/`.

### How does Dify convert custom tools to OpenAI-compatible function schemas?

Dify converts custom tools through the `BaseAgentRunner._init_prompt_tools` method in [`api/core/agent/base_agent_runner.py`](https://github.com/langgenius/dify/blob/main/api/core/agent/base_agent_runner.py). This method iterates over the configured tools and calls `_convert_tool_to_prompt_message_tool` to transform Dify `Tool` entities (defined in [`core/tools/entities/tool_entities.py`](https://github.com/langgenius/dify/blob/main/core/tools/entities/tool_entities.py)) into `PromptMessageTool` objects that follow the OpenAI function calling specification, including proper mapping of `ToolParameter` definitions to JSON schema properties.

### Can function calling agents query multiple knowledge bases simultaneously?

Yes, function calling agents can query multiple knowledge bases through the `FunctionCallMultiDatasetRouter` located in [`api/core/rag/retrieval/router/multi_dataset_function_call_router.py`](https://github.com/langgenius/dify/blob/main/api/core/rag/retrieval/router/multi_dataset_function_call_router.py). When a dataset retriever tool is invoked, this router determines which specific dataset ID should handle the request based on the function arguments. This allows a single agent configuration to expose multiple knowledge bases as separate tools or route to different datasets based on the query content, enabling transparent multi-dataset retrieval within the function calling loop.

### Where is the function calling strategy configured in Dify applications?

The function calling strategy is configured in the application's agent configuration, typically found in configuration files or database entries managed through [`api/commands.py`](https://github.com/langgenius/dify/blob/main/api/commands.py). Specifically, developers must set `agent.strategy` to `"function_call"` within the app configuration object. This configuration instructs Dify to instantiate `FunctionCallAgentRunner` instead of other agent runners, and must be accompanied by a `tools` list defining the available functions and a `max_iteration` limit to prevent infinite loops.