# How Needle Handles Off-Topic Requests: Empty Answers and Schema-Based Refusals

> Discover how Needle handles off-topic requests with empty answers and schema-based refusals. Learn about its fine-tuning and refusal mechanisms for accurate responses.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-09-04

---

**Needle handles off-topic requests by returning an empty `answers` array when no tool schema matches the user query, a behavior enforced through specialized fine-tuning data generation and the `render_example` function in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py).**

Needle, the open-source tool orchestration framework from cactus-compute/needle, implements strict schema adherence by training its models to recognize and refuse requests that fall outside defined tool capabilities. This article examines the source code implementation of off-topic detection, from the data generation templates that create refusal examples to the runtime rendering logic that signals when a request cannot be fulfilled.

## Training Data Generation with Explicit Refusal Patterns

The foundation of Needle's off-topic handling lies in its **fine-tuning data generation strategy**. Rather than encountering refusals accidentally during training, the model is explicitly exposed to off-topic examples through the `_GEN_TEMPLATE` string defined in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 45-53).

### The _GEN_TEMPLATE Configuration

The template instructs the data generation engine to produce varied training examples, including a specific quota of off-topic inputs. As implemented in the source:

```python

# needle/model/finetune.py (lines 45-53)

_GEN_TEMPLATE = """Schemas available (JSON):
{tools}

Produce {n} varied examples as a JSON array. Each element is an object:
  {"query": "<a natural user request ...>",
   "reasoning": "<one short line deriving each argument ...>",
   "answers": [{"name": "<schema name>", "arguments": {...}}]}

Rules:
- Cover single-call, multi-call, and about {refusals} off-topic inputs that no
  schema can serve (for those, "answers" is []).
"""

```

The critical clause—*"about {refusals} off-topic inputs that no schema can serve"*—ensures the training dataset contains explicit examples where the `answers` field is empty. The `{refusals}` placeholder allows configuration of refusal frequency during data generation.

## Runtime Rendering of Off-Topic Requests

During inference, Needle determines whether to refuse a request based on the contents of the `answers` array. The **`render_example`** function processes these arrays to generate the target output strings that the model learns to reproduce.

### The render_example Function

Located in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 94-106), `render_example` transforms structured example data into prompt-target pairs. When processing an off-topic request, the function receives an empty `answers` list:

```python

# needle/model/finetune.py - handling off-topic at runtime

example = {
    "query": "Tell me a joke about quantum physics.",
    "reasoning": "off-topic",
    "answers": []          # No tool schema matches this request

}

prompt, target = render_example(example)

```

In this case, `render_example` produces a target string containing only the conversation markers without any tool call blocks. The resulting target teaches the model that when no schema applies, it must output `[]` between the tool call markers, signaling a refusal to act.

## Validating Refusal Behavior in the Test Suite

Needle's test suite enforces this behavior through concrete examples. In **[`tests/test_finetune.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_finetune.py)** (lines 20-21), the validation includes a fixture explicitly testing the off-topic scenario:

```python

# tests/test_finetune.py - off-topic validation

{
    "reasoning": "off-topic",
    "answers": []
}

```

This test ensures that the data generation pipeline correctly labels refusals with the "off-topic" reasoning string and maintains the empty answers array contract. By validating these structures during testing, Needle guarantees that the model consistently recognizes boundary conditions where user requests exceed available tool capabilities.

## Summary

- **Data Generation**: The `_GEN_TEMPLATE` in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) explicitly creates off-topic training examples with empty `answers` arrays, using the `{refusals}` parameter to control frequency.
- **Runtime Rendering**: The `render_example` function converts these empty arrays into target strings containing only markers, training the model to output `[]` when refusing requests.
- **Schema Enforcement**: At inference time, an empty `answers` array signals that no tool schema matches the query, causing Needle to refuse the request rather than hallucinate a tool call.
- **Test Coverage**: [`tests/test_finetune.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_finetune.py) validates the off-topic labeling and empty array structure to ensure reliable refusal behavior.

## Frequently Asked Questions

### What happens when Needle receives an off-topic request?

When Needle processes a request that no defined tool schema can satisfy, it returns an empty `answers` array. According to the `render_example` implementation in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), this generates a target output containing only conversation markers with no actual tool calls, effectively refusing to execute any action while maintaining the expected response format.

### How does Needle generate training data for refusals?

Needle generates refusal training data through the `_GEN_TEMPLATE` string in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 45-53). This template includes a specific instruction to create approximately `{refusals}` examples where the input is off-topic and the `answers` field is an empty list `[]`, ensuring the model learns to recognize out-of-scope requests during fine-tuning.

### Where is the off-topic handling logic implemented in the codebase?

The core logic resides in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), specifically within the `_GEN_TEMPLATE` definition (lines 45-53) that structures training data, and the `render_example` function (lines 94-106) that renders these examples into model-ready formats. The behavior is validated in [`tests/test_finetune.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_finetune.py), which includes explicit test cases for off-topic scenarios.

### Can Needle's refusal behavior be customized?

Yes, the refusal behavior can be adjusted through the `{refusals}` parameter in the data generation template, which controls the ratio of off-topic examples in the training set. However, the fundamental mechanism—returning an empty `answers` array when no schema matches—is hardcoded in the `render_example` function and represents the core architectural approach to schema boundary enforcement in the cactus-compute/needle framework.