How Needle Handles Off-Topic Queries That Don't Match Any Tools

Needle refuses off-topic queries by returning an empty list of function calls ([]), with no fallback to free-text generation.

When a user submits a query that cannot be satisfied by any declared tool, Needle enforces a strict refusal contract. This behavior is hardcoded into the execution loop and reinforced through training data patterns. Understanding this mechanism is essential for developers building reliable tool-calling systems with Needle.

The Core Refusal Mechanism

Needle's handling of off-topic queries follows a simple, unambiguous rule: no matching tool equals empty function calls. This design eliminates ambiguity and prevents hallucinated responses.

The Empty List Contract

At the heart of Needle's refusal system is the function_calls field in the model's JSON response. According to doc/apis.md, an off-topic request "is refused with the empty call []" and critically, there is "no free-text fallback" [source]. This means:

• The model never generates conversational responses for unsupported queries
• The application layer receives a predictable, machine-readable signal
• No partial or speculative tool invocations occur

The llms.txt file reinforces this contract, specifying that while type is "call" for valid tool requests, an empty function_calls array unequivocally represents refusal for off-topic input [source].

Execution Flow in the Core Loop

The refusal logic is implemented directly in needle/__init__.py. The execution loop fetches and checks the function_calls field, treating an empty result as a hard stop.

# Inside needle/__init__.py — core execution loop

calls = response.get("function_calls") or []
if not calls:                         # Empty list → refusal

    # Handle refusal: log, ask rephrase, or route elsewhere

    return []

# Otherwise, execute returned tool calls

This pattern ensures consistent behavior across all refusal scenarios—whether the query is completely unrelated to available tools or simply too ambiguous to map reliably.

Training Data Patterns for Refusal

Needle's refusal behavior is not emergent; it is explicitly taught through training data structure. The README.md documents that off-topic examples use an empty answers field [source]:

{
  "tools": [],
  "query": "Tell me a joke about quantum physics",
  "answers": []
}

The doc/finetuning.md file elaborates on this pattern, recommending that fine-tuning datasets include off-topic examples with empty answers to teach the model correct refusal behavior [source]. This data-driven approach ensures the model internalizes the refusal contract rather than learning it through post-processing rules.

Confidence-Based Rejection

Needle applies additional filtering even for seemingly plausible tool matches. A confidence gating system combines post-hoc head scores with decoding probabilities to veto uncertain calls. If the combined confidence falls below threshold, the request is rejected with the same empty call list used for off-topic queries. This prevents low-confidence executions that could produce incorrect results.

Practical Implementation Example

Here's how Needle's refusal behavior manifests in practice:

from needle import Needle

# Case 1: No tools declared — any query is off-topic

needle = Needle(tools=[])
response = needle.run("What is the weather in Paris?")
print(response)                      # => []  (refusal)

# Case 2: Tools exist but query doesn't match

needle = Needle(tools=[search_tool, calculator_tool])
response = needle.run("Write a poem about recursion")
print(response)                      # => []  (refusal — no creative writing tool)

Avoiding Common Misconceptions

Developers sometimes assume tool-calling frameworks provide graceful degradation to conversational responses. Needle explicitly rejects this pattern. The empty list response requires intentional handling:

• Log refusals for analytics on coverage gaps
• Prompt users to rephrase or clarify their intent
• Route to alternative handlers (human agents, web search, or different model instances)

Summary

• Strict contract: Off-topic queries return [] via function_calls with no free-text fallback
• Core implementation: needle/__init__.py checks for empty calls and halts execution
• Training reinforcement: Empty answers arrays in data teach the refusal pattern
• Confidence gating: Additional threshold prevents uncertain tool invocations
• No hidden behavior: Refusals are explicit, observable, and consistent

Frequently Asked Questions

What happens if I forget to handle empty function_calls in my application?

Your application will receive an empty list and should treat it as a refusal. Without explicit handling, the user experience may appear unresponsive. Implement a check for if not calls: to branch to clarification prompts or alternative workflows.

Can I override Needle to generate free-text responses for off-topic queries?

No. The refusal contract in doc/apis.md is fundamental—there is intentionally no free-text fallback mechanism. If you need conversational capabilities alongside tool calling, you must implement a separate model invocation outside Needle's execution loop.

How should I structure training data to teach refusal behavior?

Include examples where "answers": [] corresponds to queries no available tool can satisfy. The doc/finetuning.md documentation recommends covering diverse off-topic categories to ensure robust refusal learning without overfitting to specific phrasings.

Does Needle distinguish between "no match" and "low confidence" refusals?

Both result in identical empty function_calls outputs. Your application cannot differentiate the cause without additional instrumentation. If granularity matters, implement pre-call confidence estimation or post-hoc logging of model scores.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}