What Happens When Off-Topic Input Is Provided to the Needle Agent

When you submit off-topic input to the Needle agent, it returns a structured refusal response with "type": "refuse" and "reason": "off-topic" instead of attempting to fulfill the request.

The Needle agent from cactus-compute/needle is designed as a tool-calling agent that categorizes every input into one of three possible outcomes: invoke a tool ("call"), return a direct text answer ("text"), or decline the request ("refuse"). Off-topic input triggers the third path, ensuring the agent stays within its operational boundaries.

How Needle Detects Off-Topic Input

The agent's inference pipeline evaluates whether a user query matches any available tool schemas. When no schema can satisfy the request, the pipeline classifies the input as off-topic and generates a refusal.

This behavior is explicitly trained into the model through the finetuning template located at needle/model/finetune.py. Lines 51-53 of this file instruct the model to recognize inputs that fall outside the scope of defined schemas and emit a refusal response accordingly.

# From needle/model/finetune.py (conceptual excerpt)

# The generation template includes instructions like:

# "If the user query does not match any tool schema, 

#  respond with type='refuse' and reason='off-topic'"

The fetch() function in needle/agent/fetch.py implements the runtime logic that packages this decision into a standardized response object.

The Refusal Response Structure

When Needle refuses an off-topic input, the response follows a consistent JSON structure:

{
  "type": "refuse",
  "reason": "off-topic",
  "message": "I'm sorry, but I cannot help with that."
}

• type: Always "refuse" for declined requests
• reason: Set to "off-topic" when no tool schema applies
• message: Optional human-readable explanation

Testing Off-Topic Refusal Behavior

The Needle test suite validates this behavior across multiple test files, ensuring consistent handling of off-topic inputs.

Test Coverage Locations

Test File	Purpose	Key Lines
tests/test_weights.py	Verifies tuned weights produce correct refusals	L60-L68
tests/test_inference.py	Confirms inference results include "refuse" type	L16
tests/test_environments.py	Checks environment-driven responses honor refusal	L72

These tests assert that response["type"] equals one of "call", "text", or "refuse", with the off-topic case specifically expecting "refuse".

Practical Examples

CLI Usage

Run an off-topic query directly through the Needle CLI:

# Request general knowledge outside any tool schema

needle run "What is the capital of France?"

# Expected response

{
  "type": "refuse",
  "reason": "off-topic",
  "message": "I'm sorry, but I cannot help with that."
}

Programmatic Usage

Use the Python API to handle refusals in application code:

from needle.agent import fetch

# Submit a query with no matching tool

response = fetch("Tell me a joke about bananas.")

# Inspect the refusal

print(response["type"])    # → "refuse"

print(response["reason"])  # → "off-topic"

Unit Test Pattern

Reproduce the test suite's validation approach:

def test_off_topic_refusal():
    """Verify off-topic inputs return structured refusals."""
    resp = fetch("How many moons does Mars have?")
    
    assert resp["type"] == "refuse"
    assert resp["reason"] == "off-topic"

Key Implementation Files

• needle/model/finetune.py — Training template that teaches off-topic detection (L51-L53)
• needle/agent/fetch.py — Request handler returning type values of "call", "text", or "refuse"
• tests/test_weights.py — Weight-loading tests for refusal behavior
• tests/test_inference.py — Core inference validation
• tests/test_environments.py — Environment-specific refusal checks

Summary

• Needle categorizes all inputs as tool calls, text answers, or refusals
• Off-topic input triggers a refusal when no tool schema matches
• Response format is strictly structured JSON with "type": "refuse" and "reason": "off-topic"
• Training and testing infrastructure ensures consistent behavior across deployments

Frequently Asked Questions

What counts as off-topic for the Needle agent?

Any user query that does not align with at least one available tool schema is considered off-topic. This includes general knowledge questions, creative writing requests, or tasks outside the agent's configured capabilities. The agent does not attempt to answer these; it refuses immediately.

Can I customize the refusal message for off-topic inputs?

The refusal response structure is fixed in the core agent logic within needle/agent/fetch.py. While the base responsealways includes "type": "refuse" and "reason": "off-topic", you can wrap the fetch() call in application-level code to intercept and modify the human-readable message field before presenting it to users.

How does Needle's off-topic detection differ from general LLM guardrails?

Unlike broad safety filters that reject harmful content, Needle's off-topic detection is schema-driven and specific to tool availability. A query may be perfectly safe and appropriate but still receive a refusal if no registered tool can handle it. This is an intentional design choice to maintain predictable agent behavior.

Is the refusal behavior tested in CI/CD?

Yes. The tests/test_weights.py, tests/test_inference.py, and tests/test_environments.py files all contain assertions verifying that off-topic inputs return "refuse" responses. These tests run against trained model weights to ensure the finetuning template's instructions are correctly learned.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}