# What Happens When Off-Topic Input Is Provided to the Needle Agent

> Discover how the Needle agent handles off-topic input. Learn about its structured refusal response, including type and reason for rejection.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-31

---

**When you submit off-topic input to the Needle agent, it returns a structured refusal response with `"type": "refuse"` and `"reason": "off-topic"` instead of attempting to fulfill the request.**

The Needle agent from [cactus-compute/needle](https://github.com/cactus-compute/needle) is designed as a **tool-calling agent** that categorizes every input into one of three possible outcomes: invoke a tool (`"call"`), return a direct text answer (`"text"`), or decline the request (`"refuse"`). Off-topic input triggers the third path, ensuring the agent stays within its operational boundaries.

## How Needle Detects Off-Topic Input

The agent's inference pipeline evaluates whether a user query matches any **available tool schemas**. When no schema can satisfy the request, the pipeline classifies the input as off-topic and generates a refusal.

This behavior is explicitly trained into the model through the **finetuning template** located at [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py). Lines 51-53 of this file instruct the model to recognize inputs that fall outside the scope of defined schemas and emit a refusal response accordingly.

```python

# From needle/model/finetune.py (conceptual excerpt)

# The generation template includes instructions like:

# "If the user query does not match any tool schema, 

#  respond with type='refuse' and reason='off-topic'"

```

The `fetch()` function in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) implements the runtime logic that packages this decision into a standardized response object.

## The Refusal Response Structure

When Needle refuses an off-topic input, the response follows a consistent JSON structure:

```json
{
  "type": "refuse",
  "reason": "off-topic",
  "message": "I'm sorry, but I cannot help with that."
}

```

- **`type`**: Always `"refuse"` for declined requests
- **`reason`**: Set to `"off-topic"` when no tool schema applies
- **`message`**: Optional human-readable explanation

## Testing Off-Topic Refusal Behavior

The Needle test suite validates this behavior across multiple test files, ensuring consistent handling of off-topic inputs.

### Test Coverage Locations

| Test File | Purpose | Key Lines |
|-----------|---------|-----------|
| [`tests/test_weights.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py) | Verifies tuned weights produce correct refusals | [L60-L68](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py#L60-L68) |
| [`tests/test_inference.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_inference.py) | Confirms inference results include `"refuse"` type | [L16](https://github.com/cactus-compute/needle/blob/main/tests/test_inference.py#L16) |
| [`tests/test_environments.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_environments.py) | Checks environment-driven responses honor refusal | [L72](https://github.com/cactus-compute/needle/blob/main/tests/test_environments.py#L72) |

These tests assert that `response["type"]` equals one of `"call"`, `"text"`, or `"refuse"`, with the off-topic case specifically expecting `"refuse"`.

## Practical Examples

### CLI Usage

Run an off-topic query directly through the Needle CLI:

```bash

# Request general knowledge outside any tool schema

needle run "What is the capital of France?"

# Expected response

{
  "type": "refuse",
  "reason": "off-topic",
  "message": "I'm sorry, but I cannot help with that."
}

```

### Programmatic Usage

Use the Python API to handle refusals in application code:

```python
from needle.agent import fetch

# Submit a query with no matching tool

response = fetch("Tell me a joke about bananas.")

# Inspect the refusal

print(response["type"])    # → "refuse"

print(response["reason"])  # → "off-topic"

```

### Unit Test Pattern

Reproduce the test suite's validation approach:

```python
def test_off_topic_refusal():
    """Verify off-topic inputs return structured refusals."""
    resp = fetch("How many moons does Mars have?")
    
    assert resp["type"] == "refuse"
    assert resp["reason"] == "off-topic"

```

## Key Implementation Files

- **[`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)** — Training template that teaches off-topic detection ([L51-L53](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py#L51-L53))
- **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)** — Request handler returning `type` values of `"call"`, `"text"`, or `"refuse"`
- **[`tests/test_weights.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py)** — Weight-loading tests for refusal behavior
- **[`tests/test_inference.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_inference.py)** — Core inference validation
- **[`tests/test_environments.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_environments.py)** — Environment-specific refusal checks

## Summary

- **Needle** categorizes all inputs as tool calls, text answers, or refusals
- **Off-topic input** triggers a refusal when no tool schema matches
- **Response format** is strictly structured JSON with `"type": "refuse"` and `"reason": "off-topic"`
- **Training and testing** infrastructure ensures consistent behavior across deployments

## Frequently Asked Questions

### What counts as off-topic for the Needle agent?

Any user query that does not align with at least one **available tool schema** is considered off-topic. This includes general knowledge questions, creative writing requests, or tasks outside the agent's configured capabilities. The agent does not attempt to answer these; it refuses immediately.

### Can I customize the refusal message for off-topic inputs?

The refusal response structure is fixed in the core agent logic within [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py). While the base responsealways includes `"type": "refuse"` and `"reason": "off-topic"`, you can wrap the `fetch()` call in application-level code to intercept and modify the human-readable message field before presenting it to users.

### How does Needle's off-topic detection differ from general LLM guardrails?

Unlike broad safety filters that reject harmful content, Needle's off-topic detection is **schema-driven** and specific to tool availability. A query may be perfectly safe and appropriate but still receive a refusal if no registered tool can handle it. This is an intentional design choice to maintain predictable agent behavior.

### Is the refusal behavior tested in CI/CD?

Yes. The [`tests/test_weights.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py), [`tests/test_inference.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_inference.py), and [`tests/test_environments.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_environments.py) files all contain assertions verifying that off-topic inputs return `"refuse"` responses. These tests run against trained model weights to ensure the finetuning template's instructions are correctly learned.