Needle 2 Model Confidence None After Fine-Tuning: Why It Happens and How to Fix It

Fine-tuning a Needle 2 model deliberately skips the confidence head, causing the confidence field to return None because uncalibrated scores would be misleading.

The Needle library implements a specific architectural decision during fine-tuning: only the main language model weights are updated, while the confidence head—the sub-network that produces calibrated confidence scores—remains frozen at its pre-training state. This design choice prevents unreliable confidence values from being reported after weight adjustments, but it can surprise developers expecting continuous confidence outputs. Understanding this behavior requires examining the source code in needle/__init__.py and needle/model/finetune.py from the cactus-compute/needle repository.

Why the Confidence Head Is Not Updated During Fine-Tuning

The Needle 2 architecture separates the core language model from its confidence estimation component. When you invoke finetune(), the library applies updates exclusively to the main weights, as shown in the implementation at lines 16-22 of needle/__init__.py:

def finetune(model, config: FinetuneConfig, **kwargs):
    """Fine‑tune a pre‑trained model.
    """
    # Apply fine‑tuning steps; the confidence head is *not* updated.

    # This mirrors the behaviour of the original Needle implementation.

    model.apply_finetune(config, **kwargs)
    warnings.warn(
        "finetuning does not update the confidence head, so scores are "
        "uncalibrated for tuned weights; this agent reports confidence as None",
        UserWarning,
    )
    return model

The apply_finetune() function in needle/model/finetune.py reinforces this decision with explicit logging:

def apply_finetune(model: NeedleArchitecture, config: FinetuneConfig, **kwargs):
    # Perform fine‑tuning on the model weights, but deliberately skip the confidence head.

    # The confidence head remains at its pre‑training state and is not calibrated to the new weights.

    # This mirrors the behaviour of the original Needle implementation.

    # ... (fine‑tuning logic) ...

    print(f"  {'note':<9} confidence reports None with tuned weights; the head is not tuned")
    # No further updates to the confidence head.

This approach preserves training stability and prevents the confidence head from producing arbitrarily miscalibrated values that no longer correlate with actual model accuracy on the fine-tuned task.

How the Inference Layer Handles Missing Confidence

The run() function in needle/__init__.py ensures API consistency by always including a confidence field, defaulting to None when unavailable:

def run(model, inputs, **kwargs):
    """Run inference on the model.
    """
    response = model.forward(inputs, **kwargs)
    # Ensure a confidence field is present; after fine‑tuning it will be None.

    response.setdefault("confidence", None)
    return response

This design guarantees that downstream code can always access response["confidence"] without KeyError exceptions, even though the value will be None for fine-tuned models.

Reproducing the Confidence None Behavior

The following complete example demonstrates loading, fine-tuning, and running inference with the resulting None confidence value:

import warnings
from needle import load_model, finetune, run
from needle.model.finetune import FinetuneConfig

# Load a base Needle 2 model

model = load_model("cactus-compute/needle-2-base")

# Configure fine-tuning (hyperparameters omitted for brevity)

ft_config = FinetuneConfig(
    learning_rate=2e-5,
    num_epochs=3,
    batch_size=16
)

# Suppress warning for clean output, or capture it as needed

with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    
    # Fine-tune: this triggers the confidence-head warning

    tuned_model = finetune(model, config=ft_config)
    
    # Verify warning was issued

    assert len(w) == 1
    assert "confidence as None" in str(w[0].message)

# Run inference—the confidence field is explicitly None

result = run(tuned_model, inputs="Explain quantum computing in one sentence.")

print(f"Response: {result['text']}")
print(f"Confidence: {result['confidence']}")  # Output: None

Expected output:

  note      confidence reports None with tuned weights; the head is not tuned
Response: Quantum computing leverages quantum bits to perform calculations fundamentally differently from classical computers.
Confidence: None

Options for Restoring Confidence Scores After Fine-Tuning

Developers requiring calibrated confidence values have three primary paths forward:

Re-train the confidence head on the fine-tuned model using labeled data with correctness indicators. The Needle library provides ConfidenceHeadTrainer utilities in needle/training/confidence.py for this purpose.

Use the pre-trained base model's confidence for applications where confidence matters more than task-specific accuracy, accepting that the main predictions come from fine-tuned weights while confidence scores derive from the frozen base.

Implement task-specific calibration using temperature scaling or Platt scaling on a held-out validation set, treating the fine-tuned model's raw outputs as uncalibrated logits.

Summary

• Fine-tuning updates only main weights, not the confidence head, by design in Needle 2.
• None confidence prevents misleading uncalibrated scores after weight changes.
• The finetune() warning at needle/__init__.py:20-21 explicitly documents this behavior.
• response.setdefault("confidence", None) ensures consistent API responses.
• Confidence restoration requires additional training of the confidence head or external calibration techniques.

Frequently Asked Questions

Why doesn't Needle 2 automatically re-train the confidence head during fine-tuning?

The confidence head requires labeled correctness data that may not be available in standard fine-tuning datasets. Automatically re-training it without proper labels would produce equally unreliable scores. The Needle maintainers prioritized explicit None values over hidden miscalibration, following the principle that misleading confidence is worse than absent confidence.

Can I access raw confidence logits even when the calibrated score is None?

Raw logits from the frozen confidence head are accessible via model.confidence_head.forward() directly, though these values lack calibration and should not be interpreted as probabilities. The None return value in run() specifically indicates that no calibration transformation has been applied to these raw outputs.

Does quantization also affect confidence scores in Needle 2?

The quantize() function in needle/__init__.py applies to both main weights and confidence head weights simultaneously when using standard quantization configs. However, confidence scores post-quantization may exhibit slight miscalibration; the library does not automatically set them to None unless you explicitly fine-tune after quantization.

How can I verify whether a loaded model has valid confidence scores?

Check the model.has_calibrated_confidence attribute (boolean) or inspect needle/__init__.py:47 where the run() function determines confidence presence. A model loaded from a checkpoint with a separately trained confidence head will return True and produce non-None confidence values during inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}