# How FreeLLMAPI Handles Tool Calling: Router, Discovery, and Fusion Implementation

> Discover how FreeLLMAPI handles tool calling requests. Learn about its router, discovery, and fusion implementation for efficient tool integration and validation.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-09-02

---

**FreeLLMAPI processes tool-calling requests by validating model capabilities against a `supports_tools` database flag, probing unknown providers with a dummy weather tool, and extracting tool calls from provider responses via the fusion service.**

FreeLLMAPI serves as an open-source universal gateway for large language models, treating tool calling as a first-class capability that requires coordination across multiple architectural layers. When a client submits a request containing a `tools` array or `tool_choice` parameter, the `tashfeenahmed/freellmapi` repository executes a validation pipeline to ensure only compatible providers receive the payload. This pipeline involves capability probes, database-backed filtering, and response parsing to reliably route and process tool-calling requests.

## Detecting Provider Tool Support

Before routing any request, FreeLLMAPI determines whether a provider actually implements the OpenAI-compatible tool-calling protocol. Rather than relying solely on static configuration, the system actively tests providers and persists the results.

### The Tool Probe Mechanism

In [`server/src/services/model-discovery.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-discovery.ts) (lines 617-682), the discovery service sends a synthetic request containing a dummy `get_weather` tool definition with `tool_choice: 'auto'`. It then inspects the response's `finish_reason` to determine if the provider correctly terminated with a tool call:

```typescript
// Tool probe sent to unverified providers
const toolRes = await provider.chatCompletion(apiKey, toolMessages, modelId, {
  tools: [TOOL_PROBE],  // Dummy get_weather function
  tool_choice: 'auto',
});
const finishReason = toolRes.choices?.[0]?.finish_reason;
const supportsTools = finishReason === 'tool_calls';

```

If the provider returns `tool_calls` as the finish reason, the system marks it as capable; otherwise, the probe fails safely and the provider is flagged as incompatible.

### Database Flag Persistence

The discovery layer updates the model database with a binary **supports_tools** flag (`0` for no support, `1` for support). This persistent flag eliminates redundant network probes on subsequent requests, allowing the router to make instant eligibility decisions based on cached capability data.

## Routing Requests with Tool Requirements

The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines 996-1004) acts as the gatekeeper, filtering the candidate model pool using the persisted capability flags before forwarding any request containing tool definitions.

### Capability-Based Filtering

When a request specifies `requireTools`, the router iterates through available models and checks the `entry.supports_tools` property. If a model lacks the flag but the request requires tools, the router immediately discards that candidate:

```typescript
// Router exclusion logic for tool requirements
if (requireTools && !entry.supports_tools) {
  diag.push(`${label}: no tool-calling support`);
  continue;
}

```

This prevents incompatible providers from receiving payloads they cannot process, avoiding malformed responses or silent failures.

### Exhaustion Diagnostics

If the filtering process eliminates all candidate models, the router's `summarizeExhaustion` function generates a specific diagnostic message: **"model lacks tool-calling"**. This provides clear visibility into why the request failed, distinguishing between general unavailability and specific capability gaps.

## Extracting Tool Calls from Responses

Once a qualified provider processes the request, the **fusion** service handles response parsing and tool call extraction in [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts) (lines 244-250).

### Response Parsing Logic

The fusion layer accesses the provider's JSON response through the path `choice?.message?.tool_calls`. It validates that the field exists and contains an array with length greater than zero:

```typescript
// Extraction and validation of tool calls
const toolCalls = choice?.message?.tool_calls;
const hasToolCalls = Array.isArray(toolCalls) && toolCalls.length > 0;
if (hasToolCalls) {
  // Forward to tool execution layer
}

```

### Execution Flow

Upon detecting valid tool calls, the fusion service forwards the parsed function definitions and arguments to the appropriate tool handlers. The execution results are then re-injected into the conversation context, allowing the model to generate a final response incorporating the tool output before returning to the client.

## Summary

- FreeLLMAPI stores provider capability in a `supports_tools` database flag to enable instant eligibility checks without repeated network probes
- The **model discovery service** actively probes unknown providers using a dummy weather tool to verify OpenAI-compatible tool-calling support (lines 617-682 in [`server/src/services/model-discovery.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-discovery.ts))
- The **router** filters models based on the `supports_tools` flag, excluding incompatible providers when tools are required and generating specific "model lacks tool-calling" diagnostics (lines 996-1004 in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts))
- The **fusion service** extracts tool calls by checking `choice.message.tool_calls` array length before forwarding to executors (lines 244-250 in [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts))

## Frequently Asked Questions

### How does FreeLLMAPI determine if a provider supports tool calling?

The system sends a probe request containing a dummy `get_weather` tool and inspects the response's `finish_reason`. If the value equals `tool_calls`, the provider is marked with `supports_tools = 1` in the database; otherwise, it remains flagged as incompatible.

### What happens if no available models support tool calling?

The router generates a "model lacks tool-calling" diagnostic through its exhaustion summary and rejects the request immediately. This prevents tool payloads from reaching providers that would fail to process them correctly.

### Where does FreeLLMAPI extract tool calls from provider responses?

The fusion service extracts them from the `choice?.message?.tool_calls` path within the provider's JSON response, verifying that the field is a non-empty array before passing the calls to the execution layer.

### Can FreeLLMAPI handle mixed deployments with partial tool support?

Yes, the per-model `supports_tools` flag allows the system to route tool requests only to verified capable models while simultaneously sending standard completion requests to any available provider in the same deployment.