# What Happens to the Confidence Score During LoRA Fine-Tuning in Needle

> Discover why Needle resets the confidence score to None during LoRA fine-tuning. Learn how adapter training impacts original confidence parameters and maintains model integrity.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-26

---

**Needle resets the confidence score to `None` during LoRA fine-tuning because the confidence head is frozen while only adapter weights are trained, leaving the original confidence parameters uncalibrated for the adapted model.**

The Needle library implements a dedicated **confidence head** that predicts per-token confidence scores for base model generations. However, when you apply **LoRA (Low-Rank Adaptation)** fine-tuning, this confidence mechanism is intentionally disabled to prevent misleading outputs.

## How LoRA Fine-Tuning Affects Model Components

LoRA fine-tuning injects trainable low-rank matrices into specific layers of a language model while keeping the majority of original weights frozen. In Needle's implementation, this selective update strategy creates a critical mismatch with the confidence system.

### The Confidence Head Remains Frozen

The confidence head is instantiated separately from the main transformer layers. In [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), the architecture defines:

```python
self.confidence_head = ConfidenceHead(cfg.jax_dtype)

```

This head appears at line 488 and receives hidden states through `forward_confidence` (lines 572-576). Because LoRA training only modifies adapter matrices—not the base model structure—the `ConfidenceHead` parameters stay completely untouched during fine-tuning.

## Why Confidence Scores Become `None`

After LoRA adaptation, the language model weights shift to a new distribution, but the confidence head retains its original (pre-tuning) parameters. This creates a **calibration mismatch**: the confidence head was trained to interpret hidden states from the base model, not the adapted model.

Needle handles this explicitly. As documented in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 60-61), the library warns users that "finetuning does not update the confidence head, so scores are uncalibrated for tuned weights; this agent reports confidence as **None**."

## Verifying Confidence Behavior in Code

You can observe this behavior when running inference with a merged LoRA adapter:

```python
import needle

# Load base model and merge LoRA weights

model = needle.load("meta-llama/Meta-Llama-3-8B-Instruct")
model = model.merge_lora_adapter("my_lora_adapter")

# Generate with adapted weights

response = model.generate("Explain quantum computing in simple terms.")

print(response["text"])           # Normal generation output

print(response.get("confidence")) # → None (explicitly disabled)

```

The `confidence` field returns `None` rather than a numeric score, protecting users from relying on uncalibrated confidence estimates.

## Architecture Implications

| Component | LoRA Training? | Post-Tuning State |
|-----------|--------------|-------------------|
| Attention/FFN adapter matrices | **Yes** | Updated |
| Base transformer weights | No | Frozen original |
| `ConfidenceHead` parameters | **No** | Frozen original (stale) |
| Confidence output | N/A | `None` |

This design prioritizes **safety over convenience**. Rather than exposing potentially misleading confidence values, Needle opts for transparent omission.

## Recalibrating Confidence After LoRA

To restore confidence scores, you would need to:

1. Collect hidden states from the LoRA-adapted model on a representative dataset
2. Retrain or fine-tune the `ConfidenceHead` on these new hidden distributions
3. Merge the recalibrated head with your adapted model

No automatic recalibration is currently implemented in the Needle codebase.

## Summary

- **Confidence head freeze**: The `ConfidenceHead` in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) receives no gradient updates during LoRA training
- **Intentional `None` return**: [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) explicitly documents and enforces this behavior to prevent uncalibrated score usage
- **Calibration gap**: Original confidence parameters cannot interpret adapted model hidden states reliably
- **Safety-first design**: Needle opts for transparent omission rather than misleading confidence values

## Frequently Asked Questions

### Why can't Needle just keep using the original confidence scores after LoRA?

The confidence head was trained to interpret hidden state patterns from the base model. After LoRA adaptation, those hidden states shift to a new distribution. The frozen confidence head would systematically miscalibrate—typically overestimating or underestimating confidence in ways that are hard to predict. Needle returns `None` to signal this uncertainty explicitly.

### Can I recalibrate the confidence head myself after LoRA fine-tuning?

Yes, but no built-in utilities exist in Needle currently. You would extract hidden states from your LoRA-adapted model on a validation set, then train the `ConfidenceHead` (defined at `needle/model/architecture.py:488`) on those adapted representations before reattaching it to your model.

### Does this limitation apply to full fine-tuning or other adapter methods?

The analysis focuses on LoRA specifically. Full fine-tuning would update all parameters including the confidence head if included in the training loop. Other adapter methods (IA³, adapters) would require case-by-case verification of whether `ConfidenceHead` parameters are touched—check [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) and your specific training configuration.

### Where is the confidence head actually used during generation?

The `forward_confidence` method at `needle/model/architecture.py:572-576` routes hidden cells to `self.confidence_head`. This path is conditionally skipped when LoRA weights are active, as enforced by the initialization logic in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).