How to Configure and Use the LLM Provider for `call_llm` and `call_llm_with_tools` in Heurist Agent Framework

The LLM Provider in the Heurist Agent Framework is a configuration wrapper that exposes call() for standard text completion and call_with_tools() for function calling, both delegating to low-level functions in core/llm.py and configurable via environment variables or constructor arguments.

The Heurist Agent Framework provides a centralized LLM Provider class to streamline interactions with large language models. Located in core/components/llm_provider.py, this component abstracts API configuration complexity and exposes two primary entry points that wrap the underlying call_llm and call_llm_with_tools functions. Understanding how to configure and utilize this provider is essential for building agents that require both standard text generation and tool-enabled reasoning.

Understanding the LLM Provider Architecture

The LLM Provider serves as a high-level interface between your application logic and the low-level HTTP client implemented in core/llm.py. When you invoke methods on the provider, it handles authentication, model selection (defaulting to large_model_id), and thread pool execution before delegating to the core functions.

The architecture provides two distinct pathways:

• LLMProvider.call(): Routes to call_llm in core/llm.py (line 32) for standard text completion when skip_tools=True (the default behavior).
• LLMProvider.call_with_tools(): Routes to call_llm_with_tools in core/llm.py (line 88) by internally calling call() with skip_tools=False and enabling function calling capabilities.

Configuring the LLM Provider via Environment Variables

The LLMProvider constructor accepts configuration through explicit arguments or environment variables. According to the implementation in core/components/llm_provider.py (line 16), the initialization logic automatically checks for the following environment variables when constructor arguments are omitted:

• HEURIST_BASE_URL: The base URL endpoint for the LLM API service.
• HEURIST_API_KEY: The authentication token for API access.
• LARGE_MODEL_ID: The default model identifier for standard completion tasks.
• SMALL_MODEL_ID: The lightweight model identifier used for cost-sensitive classification operations.

Standard Text Completion with call_llm

Use the call() method to execute standard LLM completion through the call_llm function. The provider implementation (lines 30-69 in core/components/llm_provider.py) selects the appropriate model, builds the request payload, and executes the call within a thread pool to prevent blocking.

The method returns a tuple containing three elements: text (the generated response), image_url (if applicable), and tool_back (typically empty for standard calls).

from core.components.llm_provider import LLMProvider

# Uses environment variables for configuration

provider = LLMProvider()

text, image_url, tool_back = provider.call(
    system_prompt="You are a helpful assistant.",
    user_prompt="Explain the difference between PoW and PoS.",
    temperature=0.5,
)

print(text)

Tool-Enabled Requests with call_llm_with_tools

To enable function calling, invoke call_with_tools() instead of call(). This method sets skip_tools=False internally (as implemented in lines 22-41 of the provider) and forwards the request to call_llm_with_tools. The provider then extracts any tool_calls from the LLM response, executes the matching tool via the injected tool_manager, and merges the tool's result back into the final reply.

Pass your tool definitions using the OpenAI function schema format:

from core.components.llm_provider import LLMProvider

provider = LLMProvider()

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for a query",
            "parameters": {
                "type": "object",
                "properties": {"query": {"type": "string"}},
                "required": ["query"],
            },
        },
    }
]

text, image_url, tool_back = provider.call_with_tools(
    system_prompt="You can call external tools.",
    user_prompt="Find the latest price of ETH.",
    temperature=0.2,
    tools=tools,
    tool_choice="auto",
)

print("LLM reply:", text)
print("Tool payload:", tool_back)

Asynchronous Alternatives

For non-blocking operations, bypass the synchronous LLMProvider class and import call_llm_async and call_llm_with_tools_async directly from core/__init__.py (lines 30-55). These low-level functions allow you to await LLM responses without thread pool overhead.

import asyncio
from core import call_llm_async

async def main():
    response = await call_llm_async(
        base_url="https://api.openai.com/v1",
        api_key="sk-your-api-key",
        model_id="gpt-4o-mini",
        system_prompt="You are a terse bot.",
        user_prompt="Say hello.",
    )
    print(response.content)

asyncio.run(main())

Summary

• The LLM Provider class in core/components/llm_provider.py centralizes configuration for the Heurist Agent Framework and coordinates between your application and the low-level LLM functions.
• Configure the provider using environment variables (HEURIST_BASE_URL, HEURIST_API_KEY, LARGE_MODEL_ID) or explicit constructor arguments.
• Call provider.call() to execute standard text completion via the call_llm function in core/llm.py.
• Call provider.call_with_tools() to enable function calling via call_llm_with_tools, passing tool schemas in OpenAI format and receiving tool execution results in the tool_back return value.
• Import call_llm_async and call_llm_with_tools_async from core for asynchronous operations that bypass the synchronous provider wrapper.

Frequently Asked Questions

What is the difference between call_llm and call_llm_with_tools?

The call_llm function handles standard text completion without tool capabilities, while call_llm_with_tools enables the function calling interface required for agents that interact with external APIs. The LLMProvider class abstracts this distinction through the call() and call_with_tools() methods, where the latter sets skip_tools=False to trigger the tool-capable code path.

How do I configure the LLM Provider without hardcoding API keys?

Instantiate LLMProvider() with no arguments after setting the HEURIST_BASE_URL, HEURIST_API_KEY, and LARGE_MODEL_ID environment variables in your shell or .env file. The constructor automatically reads these values from the environment as implemented in core/components/llm_provider.py (line 16), keeping sensitive credentials out of your source code.

Can I use different models for different types of requests within the same provider instance?

Yes. The provider maintains both a large_model_id for standard operations and a small_model_id for lightweight classification tasks. While provider.call() defaults to large_model_id, you can override the model for specific requests by passing a model_id parameter directly to the method call.

What does the tool_back return value contain in call_with_tools?

The tool_back variable contains the raw payload returned by the tool manager after executing any tool calls requested by the LLM. This includes the actual function results that were merged back into the conversation context to generate the final response, allowing you to inspect exactly what data the LLM received from external tools.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}