# How to Set Confidence Thresholds for Needle 2 Responses

> Learn how to set confidence thresholds for Needle 2 responses. Control actions based on the calibrated confidence score in your product.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-16

---

**Needle 2 returns a calibrated confidence score (0–1) for every response; you check the `"confidence"` field and gate actions behind a threshold you choose per product.**

The **confidence score** in Needle 2 is produced by a dedicated *confidence head* defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)【/cactus-compute/needle/main/needle/model/architecture.py#L63-L76】. This score represents the minimum of two signals—a calibrated post-hoc head and the decode probability—which must agree for a response to be trusted.

## Understanding Confidence Score Availability

Needle 2 handles confidence differently depending on which weights you load.

### Base Weights: Confidence Enabled

When you instantiate `Needle()` with **base weights**, the confidence head remains active. Every response includes a `"confidence"` field populated by the confidence head:

```python
from needle import Needle

agent = Needle()
response = agent.complete("Summarize the latest AI research trends.")

# response contains: {"type": "answer", "content": "...", "confidence": 0.94}

```

### Fine-Tuned Weights: Confidence Disabled

When you load **fine-tuned weights** via `Needle(weights=...)`, the confidence head is *not* updated by the fine-tuning process. According to [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)【/cactus-compute/needle/main/needle/__init__.py#L58-L62】, Needle deliberately disables the score and sets `"confidence": null` to prevent misleading reliability signals. A warning is emitted the first time such an agent is constructed.

The documentation in [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md)【/cactus-compute/needle/main/doc/finetuning.md#L76-L84】 clarifies that responses from fine-tuned agents must be treated as *untrusted* since the confidence head no longer reflects the model's actual calibration.

## Implementing a Confidence Threshold Check

The standard workflow for setting confidence thresholds follows four steps as documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md)【/cactus-compute/needle/main/doc/apis.md#L130-L134】:

1. Generate a response with `agent.complete()` or `agent.run()`.
2. Inspect the `"confidence"` field.
3. Compare against your chosen threshold.
4. Act only if confidence meets or exceeds the threshold; otherwise re-ask, route to a higher-capacity model, or escalate.

### Basic Threshold Example

```python
from needle import Needle

agent = Needle()  # base weights, confidence enabled

response = agent.complete("Summarize the latest AI research trends.")
if response.get("confidence", 0) >= 0.8:
    print("✅ High-confidence answer:", response["content"])
else:
    print("⚠️ Low confidence – ask again or route elsewhere.")

```

### Retry Loop with Threshold

For interactive applications, implement a retry loop that re-queries until confidence exceeds your threshold or reaches a maximum step limit:

```python
from needle import Needle

agent = Needle()
max_steps = 5
threshold = 0.85

for _ in range(max_steps):
    resp = agent.run("Help the user fix a broken npm install.")
    conf = resp.get("confidence")
    
    if conf is None:
        # Fine-tuned weights detected

        print("⚠️ Confidence unavailable – treat as fallback.")
        break
        
    if conf >= threshold:
        print("✅ Accepted response:", resp["content"])
        break
    else:
        print(f"🔎 Confidence {conf:.2f} < {threshold} – retrying.")

```

### Handling Fine-Tuned Models Gracefully

Always check for `null` confidence when your codebase supports both base and fine-tuned models:

```python
from needle import Needle

agent = Needle(weights="my_finetuned.cact")

resp = agent.complete("Explain quantum entanglement.")

if resp.get("confidence") is None:
    print("ℹ️ Confidence disabled – using fallback verification.")
    # Implement alternative validation: human review, secondary model, etc.

else:
    if resp["confidence"] >= 0.8:
        deliver_response(resp["content"])
    else:
        escalate_to_human(resp)

```

## Choosing Your Confidence Threshold

The documentation in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md)【/cactus-compute/needle/main/doc/apis.md#L151-L156】 explains that the confidence score is the **minimum** of the calibrated head output and the decode probability. Both signals must agree—this conservatism means:

- A threshold of **0.8** typically captures high-precision responses suitable for direct user-facing answers.
- A threshold of **0.9** or higher adds safety for sensitive applications (medical, financial, legal).
- Lower thresholds (0.6–0.7) may be acceptable with additional post-hoc verification.

Set your threshold **per product** based on error tolerance and available fallback options【/cactus-compute/needle/main/doc/apis.md#L130-L134】.

## Key Source Files

| File | Purpose |
|------|---------|
| [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) – **ConfidenceHead** class【/cactus-compute/needle/main/needle/model/architecture.py#L63-L76】 | Implements the calibrated confidence scoring mechanism. |
| [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) – response construction【/cactus-compute/needle/main/needle/__init__.py#L58-L62】 | Injects or nulls the `"confidence"` field based on weight type. |
| [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) – confidence gating contract【/cactus-compute/needle/main/doc/apis.md#L151-L156】 | Documents the minimum-of-two-signals logic and threshold recommendations. |
| [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) – fine-tuning effects【/cactus-compute/needle/main/doc/finetuning.md#L76-L84】 | Explains why confidence is disabled for LoRA-fine-tuned weights. |

## Summary

- Needle 2 confidence scores are **always available with base weights**, **always null with fine-tuned weights**.
- The score in `response["confidence"]` is a float ∈ [0, 1] or `null`—check explicitly before comparison.
- Apply a product-specific **confidence threshold** (commonly 0.8–0.9) to gate high-stakes actions.
- For fine-tuned agents, implement **fallback verification** since the confidence head cannot attest to response reliability.

## Frequently Asked Questions

### What happens if I use a fine-tuned model without checking for null confidence?

Your code will likely throw a `TypeError` when comparing `None` to a numeric threshold, or worse, silently accept untrusted responses. Always check `resp.get("confidence") is None` and branch to fallback logic.

### Why does fine-tuning disable the confidence head?

The confidence head is not updated during LoRA fine-tuning ([`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md)【/cactus-compute/needle/main/doc/finetuning.md#L76-L84】). Keeping it active would produce miscalibrated scores that falsely suggest reliability, so Needle explicitly sets `"confidence": null` to signal uncertainty.

### Can I recalibrate the confidence head after fine-tuning?

Not through the current Needle 2 API. Recalibration would require retraining the confidence head on fine-tuned model outputs—a feature not exposed in the open-source release. For now, treat fine-tuned responses as requiring external validation.

### What's the recommended threshold for production use?

Start with **0.8** for general applications, **0.9+** for high-stakes domains. The documentation emphasizes choosing per-product: err toward higher thresholds when errors are costly, lower ones when you have robust retry or human-in-the-loop fallbacks.