# What Does a High KL Divergence Score Mean in Heretic? Interpretation Guide

> Understand what a high KL divergence score means in Heretic. Learn how this score indicates model drift and potential degradation of language capabilities and knowledge.

- Repository: [Philipp Emanuel Weidmann/heretic](https://github.com/p-e-w/heretic)
- Tags: deep-dive
- Published: 2026-02-19

---

**A high KL divergence score in Heretic indicates that the ablated model has drifted significantly from the original model's probability distribution, signaling potential degradation of core language capabilities and factual knowledge.**

In the Heretic framework for model unlearning, **KL divergence** serves as a critical guardrail during optimization. This metric quantifies how much the probability distribution over the first generated token of the ablated model differs from that of the base model when both process the same set of "good" (non-refusal-inducing) prompts.

## How Heretic Calculates KL Divergence

The calculation occurs in [`src/heretic/evaluator.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/evaluator.py) within the `get_score` method. The evaluator computes per-prompt log-probabilities using `model.get_logprobs_batched` for both the current trial and the untouched base model. These tensors feed into `torch.nn.functional.kl_div` with specific parameters:

- `reduction="batchmean"` – averages across the batch dimension
- `log_target=True` – indicates the target is already log-probabilities

The resulting scalar is stored as `trial.user_attrs["kl_divergence"]` in [`src/heretic/main.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/main.py) (lines 542-556) and subsequently displayed in the CLI UI and markdown reports.

## Interpreting KL Divergence Values in Heretic

The magnitude of the KL divergence directly correlates with model fidelity. Heretic implements specific thresholds to guide interpretation:

| KL Divergence | Interpretation |
|---------------|----------------|
| **≈ 0** | The ablated model behaves almost identically to the base model on good prompts, indicating minimal alteration of knowledge or language ability. |
| **0 – 0.01** | The default target range (`settings.kl_divergence_target`). The model successfully avoids refusing harmful prompts while preserving original capabilities. |
| **> 0.01** | The optimizer begins penalizing the trial. The model distribution is drifting away from the original, potentially compromising performance. |
| **> 1** | **Critical threshold** flagged in the UI ([`src/heretic/main.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/main.py), lines 662-668). Significant damage to core capabilities—fluency, factual recall, and coherence. The model may no longer resemble the original. |

## KL Divergence in Multi-Objective Optimization

Heretic employs KL divergence as a constraint within a multi-objective optimization framework managed by Optuna. Two configuration parameters in [`src/heretic/config.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/config.py) (lines 163-176) govern its behavior:

- **`kl_divergence_target`** (default: 0.01) – The desired maximum divergence
- **`kl_divergence_scale`** (default: 1.0) – Scaling factor for the penalty term

When the measured KL falls below the target, the optimizer prioritizes reducing refusals. When KL exceeds the target, the divergence penalty dominates the objective function, preventing runaway model damage.

## Accessing and Configuring KL Divergence

### Retrieving KL Scores from Completed Trials

```python
import optuna

# Assume `study` is an optuna Study that has completed trials

for trial in study.trials:
    if trial.state == optuna.trial.TrialState.COMPLETE:
        kl = trial.user_attrs["kl_divergence"]
        refusals = trial.user_attrs["refusals"]
        print(f"Trial {trial.user_attrs['index']}: KL={kl:.4f}, refusals={refusals}")

```

The `user_attrs` dictionary is populated in [`src/heretic/main.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/main.py) after each evaluation.

### Adjusting KL Targets in Configuration

```toml

# config.toml – user-provided overrides

kl_divergence_target = 0.02   # relax the target slightly

kl_divergence_scale = 2.0     # make KL penalties less aggressive

```

These fields are defined in [`src/heretic/config.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/config.py) and directly influence the optimization objective.

## Summary

- **KL divergence** in Heretic measures distribution shift between the ablated and base models on good prompts, calculated in [`src/heretic/evaluator.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/evaluator.py) using `torch.nn.functional.kl_div`.
- **High scores (>0.01)** trigger optimization penalties; **scores >1** indicate severe model degradation and trigger UI warnings in [`src/heretic/main.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/main.py).
- The default **target of 0.01** balances refusal reduction with capability preservation, configurable via `kl_divergence_target` and `kl_divergence_scale` in [`src/heretic/config.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/config.py).
- Access runtime KL values through `trial.user_attrs["kl_divergence"]` after evaluation.

## Frequently Asked Questions

### What is the default KL divergence target in Heretic?

The default target is **0.01**, defined in [`src/heretic/config.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/config.py) as `kl_divergence_target`. This value represents the desired maximum divergence where the model should maintain original capabilities while avoiding refusals. The optimizer treats values below this threshold as acceptable and begins penalizing trials that exceed it.

### How does Heretic calculate KL divergence between models?

Heretic calculates KL divergence in [`src/heretic/evaluator.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/evaluator.py) by first obtaining log-probabilities for the next token from both the ablated trial model and the base model using `model.get_logprobs_batched`. These tensors are passed to `torch.nn.functional.kl_div` with `reduction="batchmean"` and `log_target=True`, producing a single scalar value that quantifies the distribution difference.

### What should I do if my KL divergence exceeds 1.0?

If KL divergence exceeds **1.0**, Heretic displays a warning in the UI (implemented in [`src/heretic/main.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/main.py), lines 662-668) indicating significant model damage. You should consider stopping the optimization and either adjusting your ablation parameters or increasing the `kl_divergence_scale` in your configuration to apply stronger penalties earlier, preventing further degradation of core language capabilities.

### Can I adjust how strongly KL divergence affects the optimization score?

Yes, you can adjust the impact of KL divergence through the `kl_divergence_scale` parameter in [`src/heretic/config.py`](https://github.com/p-e-w/heretic/blob/main/src/heretic/config.py) (default 1.0). Increasing this value makes the penalty term more aggressive when divergence exceeds the target, while decreasing it allows more flexibility at the risk of greater model drift. You can also modify `kl_divergence_target` (default 0.01) to set a different acceptable threshold for your specific use case.