How Needle 2 Handles Off-Topic Input: The Empty Array Contract

Needle 2 refuses off-topic requests by returning an empty answers (or function_calls) array rather than generating free-text responses, enforcing a strict "refuse-by-default" tool boundary.

Needle 2 is an open-source inference framework designed for structured tool-use scenarios. When user queries fall outside the scope of declared tool schemas, the system implements a deterministic refusal mechanism that prioritizes safety over helpfulness, returning explicit empty collections instead of hallucinated responses.

The Core Refusal Mechanism

Needle 2 implements a strict architectural contract where the inference loop terminates immediately when no tool matches the user query. According to the LLM contract defined in llms.txt (line 156), an empty call represented as [] explicitly signals a refusal for off-topic input. This design eliminates the risk of unsanctioned content generation by ensuring free-text fallbacks are structurally impossible.

Fine-Tuning Data Templates

The training pipeline encodes this refusal behavior directly into the data generation templates. In needle/model/finetune.py (lines 51-53), off-topic examples are explicitly constructed with empty answers arrays, teaching the model to associate out-of-scope queries with refusal signals. The project README (line 101) reiterates this data format specification.

# Template structure from needle/model/finetune.py (L51-53)

example = {
    "query": "Tell me a joke about bananas.",
    "reasoning": "off-topic",  # No declared tool handles this request

    "answers": []              # Empty array enforces refusal

}

API and Runtime Behavior

At runtime, the public API documented in doc/apis.md (line 103) clarifies that requests no tool can serve are refused with the empty call []. Client applications must explicitly check for this condition, as the system provides no explanatory text or fallback content.

# Client implementation pattern for detecting refusals

result = needle.run(query)
if not result.get("answers"):  # Empty list indicates off-topic refusal

    print("Request refused: no matching tool available")
else:
    for call in result["answers"]:
        execute_tool(call)

Playground UI Implementation

The behavior is consistently reflected in the web interface. In needle/playground/app.js, the UI logic detects empty responses and displays a specific message indicating the refusal state rather than rendering blank output.

// UI feedback logic from needle/playground/app.js
pre.textContent = "no tool call (off-topic / refused)";

Summary

• Empty array protocol: Needle 2 signals off-topic refusals via empty answers or function_calls arrays, never through free-text explanations.
• Contract enforcement: The behavior is formally specified in llms.txt (line 156) and doc/apis.md (line 103).
• Training alignment: Data templates in needle/model/finetune.py (lines 51-53) explicitly mark off-topic examples with answers: [].
• No fallback mechanism: Unlike conversational LLMs, Needle 2 cannot be configured to provide explanatory text for refused queries.
• Client responsibility: Applications must detect empty arrays to handle off-topic scenarios gracefully.

Frequently Asked Questions

What constitutes off-topic input in Needle 2?

Off-topic input refers to any user request that cannot be satisfied by the declared tool schemas available to the model. When the inference loop determines that no tool matches the query intent—such as creative writing tasks when only data retrieval tools are declared—the system classifies the request as off-topic and triggers the refusal protocol.

How does the API indicate an off-topic refusal?

The API returns an empty array—either answers: [] or function_calls: []—to indicate the input is off-topic. As documented in doc/apis.md (line 103) and llms.txt (line 156), this empty array is the sole signal of refusal; the system does not return HTTP error codes or natural language explanations.

Where is the off-topic handling behavior defined in the source?

The behavior is defined across multiple authoritative files: the training data template in needle/model/finetune.py (lines 51-53), the LLM contract in llms.txt (line 156), the API specification in doc/apis.md (line 103), and the data format documentation in the README (line 101). The inference engine respects these specifications by terminating generation when the tool matcher returns no candidates.

Can Needle 2 generate explanations for refused queries?

No. The system architecture explicitly prevents free-text fallbacks for off-topic input. This constraint is enforced at the training data level (through fine-tuning templates that force answers: []) and at the inference level (through loop termination logic), ensuring consistent boundary enforcement without configuration options to bypass the refusal mechanism.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}