Understanding direction_index and Layer Selection in Heretic's Ablation Scope

The direction_index parameter selects either a globally interpolated refusal direction applied to all layers or defers to per-layer directions, controlled by the direction_scope setting in Heretic's abliteration process.

Heretic is an open-source tool for "abliterating" language models—systematically nudging layers toward refusing harmful outputs while preserving helpful capabilities. The ablation process relies on pre-computed refusal directions that can be applied either uniformly across all layers or tailored individually per layer. Understanding how direction_index and direction_scope interact is essential for configuring effective ablation trials in the p-e-w/heretic repository.

How direction_index and direction_scope Define the Refusal Strategy

Heretic's abliteration requires a refusal direction for every modified layer. Two trial parameters control which direction vector is used and how it maps to the model's architecture.

Global vs. Per-Layer Scope

The direction_scope parameter determines whether the trial uses a single direction for all layers or individual directions per layer:

• "global": One interpolated direction is computed and applied uniformly to every layer
• "per layer": Each layer uses its own pre-computed direction from the refusal direction sequence

When direction_scope is set to "per layer", the direction_index parameter is forced to None, disabling interpolation and enabling layer-specific selection.

Trial Parameter Sampling in main.py

In src/heretic/main.py, the Optuna trial definition samples these values at lines 64-70 and 81-89:

direction_scope = trial.suggest_categorical(
    "direction_scope",
    [
        "global",
        "per layer",
    ],
)

# ... later in the trial ...

direction_index = trial.suggest_float(
    "direction_index",
    0.4 * last_layer_index,
    0.9 * last_layer_index,
)
if direction_scope == "per layer":
    direction_index = None

The float sampling range [0.4·L‑1, 0.9·L‑1] (where L is the number of layers) ensures the index stays within valid bounds of the pre-computed direction sequence. These parameters are stored on the trial object and passed to model.abliterate() at line 540.

Core Implementation in model.py

The abliterate method in src/heretic/model.py implements the logic that consumes these parameters and applies directions to model components.

Interpolating the Global Direction

When direction_index is not None, the code linearly interpolates between the two nearest refusal directions (lines 88-98):

if direction_index is None:
    refusal_direction = None
else:
    weight, index = math.modf(direction_index + 1)
    refusal_direction = F.normalize(
        refusal_directions[int(index)].lerp(
            refusal_directions[int(index) + 1],
            weight,
        ),
        p=2,
        dim=0,
    )

The math.modf() call splits the float into fractional and integer components, enabling smooth interpolation between discrete direction vectors using PyTorch's lerp() function.

Layer Iteration and Direction Assignment

The method iterates over every layer and applies the selected direction (lines 100-108):

for layer_index in range(len(self.get_layers())):
    for component, modules in self.get_layer_modules(layer_index).items():
        # ... component handling ...

        if refusal_direction is None:
            layer_refusal_direction = refusal_directions[layer_index + 1]
        else:
            layer_refusal_direction = refusal_direction

In per-layer mode (refusal_direction is None), the code pulls the specific vector for that layer from refusal_directions[layer_index + 1]. In global mode, it reuses the interpolated refusal_direction for all layers. The + 1 offset accounts for the sequence indexing in the refusal directions tensor.

Running Heretic with Different Direction Scopes

Using a Global Direction

To apply the same interpolated refusal direction across all layers:

heretic run \
    --trials 30 \
    --direction-scope global \
    --direction-index 5.2

All layers will be nudged along the identical interpolated vector derived from position 5.2 in the refusal direction sequence.

Using Per-Layer Directions

To allow each layer to use its own specific refusal direction:

heretic run \
    --trials 30 \
    --direction-scope per-layer

The driver automatically forces direction_index to None, and the model selects individual directions from refusal_directions[layer_index + 1] for each layer.

Debugging Direction Selection

To inspect the chosen parameters during optimization:

def objective(trial: Trial) -> float:
    # ... sampling logic ...

    print(f"Scope: {direction_scope}")
    print(f"Index: {direction_index}")  # None when per-layer

    # ... rest of objective ...

These values correspond exactly to the parameters consumed by model.abliterate().

Summary

• direction_scope chooses between "global" (uniform direction) and "per layer" (individual directions)
• direction_index selects a specific point in the refusal direction sequence via linear interpolation when using global scope
• When scope is "per layer", direction_index is forced to None to disable interpolation
• In src/heretic/main.py (lines 64-89), Optuna samples these parameters before passing them to the model
• In src/heretic/model.py (lines 88-108), the code either interpolates a global vector or selects layer-specific vectors based on the direction_index value
• The layer loop in model.py uses layer_index + 1 to index into the pre-computed refusal directions array

Frequently Asked Questions

What happens when direction_index is set to None?

When direction_index is None, Heretic operates in per-layer mode. Each transformer layer receives its own refusal direction from the pre-computed sequence at index layer_index + 1. This allows the abliteration to tailor the refusal vector to each layer's specific representation space rather than applying a one-size-fits-all direction.

How does the interpolation calculation work for global directions?

The code uses math.modf(direction_index + 1) to separate the float into integer and fractional parts. The integer component selects the base direction vector, while the fractional component determines the interpolation weight between that vector and the next one using PyTorch's lerp() function. The result is L2-normalized before application to ensure consistent magnitude across different index values.

Can I manually override direction_index from the command line?

Yes, you can pass --direction-index as a command-line argument when running heretic run. However, this only takes effect when --direction-scope global is also specified. If you specify --direction-scope per-layer, the CLI ignores any manually provided index and forces the value to None to ensure each layer uses its specific direction vector.

Which layers are affected when using per-layer mode?

In per-layer mode, every layer returned by self.get_layers() receives a refusal direction, specifically from refusal_directions[layer_index + 1]. The + 1 offset means layer 0 uses refusal direction index 1, layer 1 uses index 2, and so on. This indexing convention aligns with how the refusal directions tensor is structured in the pre-computed cache, ensuring each layer modification corresponds to the correct representation steering vector.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}