How to Build LLM Function Calling Agents in Dify: A Complete Technical Guide

Dify implements LLM function calling agents through a reusable architectural pattern centered on FunctionCallAgentRunner, which executes a continuous loop to detect tool calls, invoke corresponding tools, and iterate until the model returns a final answer.

The langgenius/dify repository provides a production-ready framework for building function calling agents that transform Dify-defined tools into OpenAI-compatible schemas. By configuring the function_call strategy and leveraging the ParameterExtractorNode, developers can create robust agents that seamlessly invoke external APIs, calculators, or knowledge base retrievers.

Architecture Overview of Function Calling Agents in Dify

Dify's function calling implementation spans four distinct layers, each handled by specific modules in the codebase.

The Agent Runner Layer

The FunctionCallAgentRunner class in api/core/agent/fc_agent_runner.py orchestrates the entire execution flow. It maintains a function_call_state boolean that controls the iteration loop. When the LLM returns a response containing tool_calls, the runner extracts the tool name and arguments, invokes the corresponding tool, and feeds the observation back into the next prompt iteration.

Tool Conversion and Registration

BaseAgentRunner._init_prompt_tools in api/core/agent/base_agent_runner.py handles the transformation of Dify Tool objects into PromptMessageTool instances following the OpenAI function schema. This method iterates over self.app_config.agent.tools, converting each tool via _convert_tool_to_prompt_message_tool and storing the original tool instances in tool_instances for later execution. Dataset retrievers receive special handling through _convert_dataset_retriever_tool_to_prompt_message_tool.

Prompt Generation for Function Calling

The ParameterExtractorNode._generate_function_call_prompt method in api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py constructs the system prompt that instructs the model to use function calling. It embeds the user query into FUNCTION_CALLING_EXTRACTOR_USER_TEMPLATE, injects a JSON schema describing expected parameters via node_data.get_parameter_json_schema(), and appends few-shot examples from FUNCTION_CALLING_EXTRACTOR_EXAMPLE.

Multi-Dataset Routing

When agents need to query multiple knowledge bases, FunctionCallMultiDatasetRouter in api/core/rag/retrieval/router/multi_dataset_function_call_router.py determines which dataset ID should handle a specific retrieval request. This enables a single function calling agent to transparently route queries to different knowledge bases based on the function parameters.

Implementing the Function Calling Agent Loop

The core execution pattern relies on a stateful loop that continues until the model stops requesting tool executions.

# Conceptual flow from api/core/agent/fc_agent_runner.py

function_call_state = True
iteration_count = 0
max_iteration = self.app_config.agent.max_iteration

while function_call_state and iteration_count < max_iteration:
    # 1. Build prompt with current history and tool definitions

    prompt_messages = self._organize_prompt_messages()
    
    # 2. Invoke LLM with tools

    response = model_instance.invoke_llm(
        prompt_messages=prompt_messages,
        tools=prompt_messages_tools,
        stream=True
    )
    
    # 3. Check for tool calls in response chunks

    tool_calls = self.check_tool_calls(response)  # or check_blocking_tool_calls

    
    if tool_calls:
        function_call_state = True
        # 4. Extract tool call details

        for tool_call in tool_calls:
            tool_id = tool_call.id
            tool_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            
            # 5. Execute tool and get observation

            observation = self._handle_tool_invoke(
                tool_name=tool_name,
                tool_inputs=arguments
            )
            
            # 6. Add observation to message history for next iteration

            self._append_tool_output_to_messages(tool_id, observation)
    else:
        function_call_state = False
    
    iteration_count += 1

# Force final answer if max iterations reached

if iteration_count >= max_iteration:
    prompt_messages_tools = []  # Remove tools to force direct answer

Configuring Tools for Function Calling Agents

Tools must be defined, converted to OpenAI-compatible schemas, and registered in the application configuration.

Defining Custom Tools

Create tool definitions using Dify's Tool entity class:

# my_tool.py

from core.tools.entities.tool_entities import Tool, ToolParameter

my_tool = Tool(
    tool_name="calculate_sum",
    description="Returns the sum of two numbers.",
    parameters=[
        ToolParameter(
            name="a", 
            type=ToolParameter.ToolParameterType.NUMBER, 
            required=True
        ),
        ToolParameter(
            name="b", 
            type=ToolParameter.ToolParameterType.NUMBER, 
            required=True
        ),
    ],
)

Registering Tools in App Configuration

Declare the function calling strategy and tool list in your application configuration:

# app_config.yaml

agent:
  strategy: function_call        # Instantiates FunctionCallAgentRunner

  max_iteration: 5
  tools:
    - tool_name: calculate_sum
    - tool_name: weather_lookup
    - tool_name: dataset_retriever  # Knowledge base tool

Dataset Retrievers as Tools

Dataset retrievers are automatically converted to function calling tools via _convert_dataset_retriever_tool_to_prompt_message_tool in BaseAgentRunner. This allows the agent to query knowledge bases using standard function call syntax:

{
  "name": "dataset_retriever",
  "arguments": "{\"query\": \"latest product specifications\", \"dataset_id\": \"prod_docs_2024\"}"
}

Building Workflows with ParameterExtractorNode

For workflow-based applications, use ParameterExtractorNode to leverage function calling for structured data extraction:

# workflow.py

from api.core.workflow.nodes.parameter_extractor.parameter_extractor_node import ParameterExtractorNode

# Create extraction node using function calling reasoning

extractor = ParameterExtractorNode(
    node_id="extract_sum",
    name="Sum Extractor",
    type="parameter_extractor",
    reasoning_mode="function_call",  # Triggers FunctionCallAgentRunner

    tools=["calculate_sum"],
    instruction="Extract two numbers from the user query and return their sum.",
    model_config={
        "provider": "openai",
        "model": "gpt-4"
    }
)

# Execute workflow

result = extractor.run(query="What is 12 plus 30?")
print(result.outputs)   # => {"sum": 42}

Key Source Files for Function Calling Agents in Dify

Understanding the implementation requires familiarity with these specific files:

• api/core/agent/fc_agent_runner.py — Contains FunctionCallAgentRunner, the main execution loop that handles tool call detection and iteration logic.

• api/core/agent/base_agent_runner.py — Defines BaseAgentRunner with _init_prompt_tools for converting Dify tools to OpenAI-compatible schemas and managing tool instances.

• api/core/workflow/nodes/parameter_extractor/parameter_extractor_node.py — Implements ParameterExtractorNode with _generate_function_call_prompt for creating system prompts with JSON schemas and few-shot examples.

• api/core/rag/retrieval/router/multi_dataset_function_call_router.py — Houses FunctionCallMultiDatasetRouter for routing function calls to specific knowledge base datasets.

• api/commands.py — Contains configuration examples showing "strategy": "function_call" declarations.

• api/core/tools/tool_engine.py — Executes concrete tool logic after function calls are extracted from LLM responses.

Summary

• Function calling agents in Dify rely on FunctionCallAgentRunner to execute a continuous loop that detects tool calls, invokes tools, and feeds observations back to the LLM until completion.

• Tool conversion happens in BaseAgentRunner._init_prompt_tools, which transforms Dify Tool entities into OpenAI-compatible PromptMessageTool schemas, including special handling for dataset retrievers.

• Configuration requires setting agent.strategy to "function_call" in the application configuration, along with a defined list of available tools and maximum iteration limits.

• Workflow integration uses ParameterExtractorNode with reasoning_mode="function_call" to leverage structured extraction capabilities with JSON schemas and few-shot prompting.

• Multi-dataset retrieval is supported through FunctionCallMultiDatasetRouter, enabling agents to query specific knowledge bases based on function parameters.

Frequently Asked Questions

What is the difference between function calling and CoT agents in Dify?

Function calling agents use the FunctionCallAgentRunner which relies on the LLM's native function calling capability to emit structured JSON tool calls, while Chain-of-Thought (CoT) agents typically use reasoning prompts without structured function schemas. Function calling agents are configured via "strategy": "function_call" and provide more reliable tool invocation through explicit schema definitions, whereas CoT agents may use "strategy": "cot" or similar configurations depending on the specific implementation in api/core/agent/.

How does Dify convert custom tools to OpenAI-compatible function schemas?

Dify converts custom tools through the BaseAgentRunner._init_prompt_tools method in api/core/agent/base_agent_runner.py. This method iterates over the configured tools and calls _convert_tool_to_prompt_message_tool to transform Dify Tool entities (defined in core/tools/entities/tool_entities.py) into PromptMessageTool objects that follow the OpenAI function calling specification, including proper mapping of ToolParameter definitions to JSON schema properties.

Can function calling agents query multiple knowledge bases simultaneously?

Yes, function calling agents can query multiple knowledge bases through the FunctionCallMultiDatasetRouter located in api/core/rag/retrieval/router/multi_dataset_function_call_router.py. When a dataset retriever tool is invoked, this router determines which specific dataset ID should handle the request based on the function arguments. This allows a single agent configuration to expose multiple knowledge bases as separate tools or route to different datasets based on the query content, enabling transparent multi-dataset retrieval within the function calling loop.

Where is the function calling strategy configured in Dify applications?

The function calling strategy is configured in the application's agent configuration, typically found in configuration files or database entries managed through api/commands.py. Specifically, developers must set agent.strategy to "function_call" within the app configuration object. This configuration instructs Dify to instantiate FunctionCallAgentRunner instead of other agent runners, and must be accompanied by a tools list defining the available functions and a max_iteration limit to prevent infinite loops.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}