# How Needle Handles Off-Topic Queries That Don't Match Any Tools

> Discover how Needle handles off-topic queries. Learn why Needle refuses queries that don't match tools by returning an empty list of function calls, avoiding free-text generation.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-20

---

**Needle refuses off-topic queries by returning an empty list of function calls (`[]`), with no fallback to free-text generation.**

When a user submits a query that cannot be satisfied by any declared tool, Needle enforces a strict refusal contract. This behavior is hardcoded into the execution loop and reinforced through training data patterns. Understanding this mechanism is essential for developers building reliable tool-calling systems with Needle.

## The Core Refusal Mechanism

Needle's handling of off-topic queries follows a simple, unambiguous rule: **no matching tool equals empty function calls**. This design eliminates ambiguity and prevents hallucinated responses.

### The Empty List Contract

At the heart of Needle's refusal system is the `function_calls` field in the model's JSON response. According to [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), an off-topic request "is refused with the empty call `[]`" and critically, there is **"no free-text fallback"** [[source]](https://github.com/cactus-compute/needle/blob/main/doc/apis.md#L103). This means:

- The model never generates conversational responses for unsupported queries
- The application layer receives a predictable, machine-readable signal
- No partial or speculative tool invocations occur

The [`llms.txt`](https://github.com/cactus-compute/needle/blob/main/llms.txt) file reinforces this contract, specifying that while `type` is `"call"` for valid tool requests, an empty `function_calls` array unequivocally represents refusal for off-topic input [[source]](https://github.com/cactus-compute/needle/blob/main/llms.txt#L101).

## Execution Flow in the Core Loop

The refusal logic is implemented directly in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py). The execution loop fetches and checks the `function_calls` field, treating an empty result as a hard stop.

```python

# Inside needle/__init__.py — core execution loop

calls = response.get("function_calls") or []
if not calls:                         # Empty list → refusal

    # Handle refusal: log, ask rephrase, or route elsewhere

    return []

# Otherwise, execute returned tool calls

```

This pattern ensures **consistent behavior across all refusal scenarios**—whether the query is completely unrelated to available tools or simply too ambiguous to map reliably.

## Training Data Patterns for Refusal

Needle's refusal behavior is not emergent; it is explicitly taught through training data structure. The [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) documents that off-topic examples use an empty `answers` field [[source]](https://github.com/cactus-compute/needle/blob/main/README.md#L81):

```json
{
  "tools": [],
  "query": "Tell me a joke about quantum physics",
  "answers": []
}

```

The [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) file elaborates on this pattern, recommending that fine-tuning datasets include off-topic examples with empty answers to teach the model correct refusal behavior [[source]](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md#L16). This data-driven approach ensures the model internalizes the refusal contract rather than learning it through post-processing rules.

## Confidence-Based Rejection

Needle applies additional filtering even for seemingly plausible tool matches. A **confidence gating** system combines post-hoc head scores with decoding probabilities to veto uncertain calls. If the combined confidence falls below threshold, the request is rejected with the same empty call list used for off-topic queries. This prevents low-confidence executions that could produce incorrect results.

## Practical Implementation Example

Here's how Needle's refusal behavior manifests in practice:

```python
from needle import Needle

# Case 1: No tools declared — any query is off-topic

needle = Needle(tools=[])
response = needle.run("What is the weather in Paris?")
print(response)                      # => []  (refusal)

# Case 2: Tools exist but query doesn't match

needle = Needle(tools=[search_tool, calculator_tool])
response = needle.run("Write a poem about recursion")
print(response)                      # => []  (refusal — no creative writing tool)

```

## Avoiding Common Misconceptions

Developers sometimes assume tool-calling frameworks provide graceful degradation to conversational responses. **Needle explicitly rejects this pattern.** The empty list response requires intentional handling:

- **Log refusals** for analytics on coverage gaps
- **Prompt users** to rephrase or clarify their intent
- **Route to alternative handlers** (human agents, web search, or different model instances)

## Summary

- **Strict contract**: Off-topic queries return `[]` via `function_calls` with no free-text fallback
- **Core implementation**: [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) checks for empty calls and halts execution
- **Training reinforcement**: Empty `answers` arrays in data teach the refusal pattern
- **Confidence gating**: Additional threshold prevents uncertain tool invocations
- **No hidden behavior**: Refusals are explicit, observable, and consistent

## Frequently Asked Questions

### What happens if I forget to handle empty function_calls in my application?

Your application will receive an empty list and should treat it as a refusal. Without explicit handling, the user experience may appear unresponsive. Implement a check for `if not calls:` to branch to clarification prompts or alternative workflows.

### Can I override Needle to generate free-text responses for off-topic queries?

No. The refusal contract in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) is fundamental—there is intentionally no free-text fallback mechanism. If you need conversational capabilities alongside tool calling, you must implement a separate model invocation outside Needle's execution loop.

### How should I structure training data to teach refusal behavior?

Include examples where `"answers": []` corresponds to queries no available tool can satisfy. The [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) documentation recommends covering diverse off-topic categories to ensure robust refusal learning without overfitting to specific phrasings.

### Does Needle distinguish between "no match" and "low confidence" refusals?

Both result in identical empty `function_calls` outputs. Your application cannot differentiate the cause without additional instrumentation. If granularity matters, implement pre-call confidence estimation or post-hoc logging of model scores.