How Needle Handles Off-Topic Requests That Don't Match Available Tools

Needle treats requests that cannot be mapped to declared tools as refusals, using a confidence gate in agent.complete() that clears function calls and returns a polite rejection message when the confidence score falls below the configured min_confidence threshold.

Needle, an open-source agent framework maintained at cactus-compute/needle, implements strict safeguards to prevent hallucinated tool calls. When a user submits an off-topic request that does not correspond to any available tool, the engine automatically triggers a refusal mechanism rather than attempting to force a match.

The Confidence Gate Mechanism

Needle applies a confidence gate to every request. The default min_confidence threshold is set to 0.0, meaning any generated output with a confidence score at or below this value is treated as a refusal.

When the engine processes a prompt, it first attempts to generate a function call. If no tool matches the request, the model falls back to a normal textual response. However, if the resulting confidence score remains at or below the configured minimum, Needle interprets this as an off-topic query and returns a polite refusal message: "I'm sorry, I can't help with that".

According to the source code in needle/environments/_harness.py (lines 17-22), this gate ensures that low-confidence or unmatchable requests are rejected before reaching execution.

Runtime Enforcement in agent.complete()

The refusal logic executes at the conclusion of the completion cycle. After agent.complete() finishes processing a prompt, the runtime performs two critical validation checks:

1. It verifies whether any function_calls were produced
2. It compares the output confidence against the min_confidence threshold

If either check fails—meaning no tools matched or the confidence is too low—the call list is cleared and the result is interpreted as a refusal. This implementation in needle/environments/_harness.py (lines 27-30) ensures that off-topic requests are automatically intercepted and never executed as hallucinated tool invocations.

Tool Schema Validation

Before the confidence gate evaluates a response, Needle determines tool eligibility using strict JSON schemas defined in needle/agent/tools.py. Each available tool is described with precise input requirements, enabling the model to accurately decide whether a request matches a defined capability. When no schema aligns with the user intent, the system naturally produces no function calls, triggering the refusal pathway in the harness.

Configuring the Refusal Threshold

Developers can adjust the sensitivity of off-topic detection by modifying the min_confidence parameter. Raising this threshold creates a stricter environment where only high-confidence tool calls are accepted, further reducing the risk of inappropriate executions.

import needle
from needle.environments.smart_home import SMART_HOME

agent = needle.Needle(tools=SMART_HOME.TOOLS, system=SMART_HOME.SYSTEM)

# Strict mode: only accept high-confidence matches

resp = agent.complete(
    "Turn on the living-room lights.",
    min_confidence=0.4
)
print(resp.get("function_calls"))  # [{'name': 'turn_on_light', ...}]

print(resp.get("confidence"))      # 0.78 → accepted

Practical Example: Handling Philosophical Queries

Off-topic requests that fall outside the defined toolset receive immediate refusal. Consider a smart home agent asked about existential philosophy:

import needle
from needle.environments.smart_home import SMART_HOME

agent = needle.Needle(tools=SMART_HOME.TOOLS, system=SMART_HOME.SYSTEM)

resp = agent.complete("What's the meaning of life?")
print(resp.get("function_calls"))  # [] → no tool match

print(resp.get("confidence", 0))   # 0.0 → below min_confidence

# Returns: "I'm sorry, I can't help with that"

In this scenario, no function calls are generated, the confidence remains at zero, and the harness triggers the refusal response rather than attempting to map the philosophical query to a home automation tool.

Orchestration in the Runtime Engine

The final response assembly occurs in needle/_worker.py, which orchestrates parsing of the model output and applies the confidence check. This module integrates the tool descriptions from needle/agent/tools.py with the validation logic from needle/environments/_harness.py, producing the final refusal message when off-topic detection criteria are met.

Summary

• Confidence gate: Needle uses a configurable min_confidence threshold (default 0.0) to filter low-quality tool matches.
• Automatic refusal: Requests producing no function calls or insufficient confidence are cleared and converted to refusal messages.
• Source locations: The logic resides in needle/environments/_harness.py (lines 17-22 and 27-30) with orchestration in needle/_worker.py.
• Hallucination prevention: By validating against JSON schemas in needle/agent/tools.py and enforcing confidence checks, Needle prevents the model from inventing undefined tools.

Frequently Asked Questions

What happens when Needle receives an off-topic request?

When a request cannot be mapped to any declared tool, agent.complete() detects zero function calls and a confidence score at or below min_confidence. The system clears the call list and returns a standard refusal message: "I'm sorry, I can't help with that".

How does the min_confidence parameter affect tool selection?

The min_confidence parameter sets the minimum acceptable confidence score for executing a tool call. When set to 0.4, for example, only outputs scoring above this threshold are accepted; anything lower triggers a refusal, effectively filtering ambiguous or off-topic requests.

Where is the refusal logic implemented in the Needle codebase?

The primary refusal logic is implemented in needle/environments/_harness.py, specifically in the validation checks at lines 17-22 (confidence gate) and lines 27-30 (call list clearing). The orchestration engine in needle/_worker.py coordinates these checks during response generation.

Can I customize the refusal message for off-topic queries?

While the analysis references a default polite refusal message, the source code structure in needle/environments/_harness.py suggests that the string "I'm sorry, I can't help with that" is the standard implementation. Developers wishing to customize this would need to modify the refusal handling logic in the harness module.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}