How the Reasonix Ablation System Works for Testing Different Configurations

The Reasonix ablation system enables systematic A/B testing by allowing developers to selectively disable subsystems—such as the planner, evidence generator, or retrieval engine—through a declarative Set structure that propagates from CLI flags to runtime conditional logic.

The esengine/DeepSeek-Reasonix repository implements a sophisticated ablation framework to isolate the impact of individual components on benchmark performance. This system treats the agent architecture as a collection of switchable modules, enabling researchers to attribute solve-rate and latency differences to specific subsystems. By leveraging the ablation system, teams can conduct rigorous experiments that determine which components contribute most to reasoning accuracy.

Core Architecture of the Ablation System

The ablation system centers on the Module enum and Set struct defined in internal/ablation/ablation.go. These primitives provide the type safety and state management necessary for selective subsystem disabling.

Module Enum and Set Structure

The Module type defines five switchable subsystems: Evidence, Planner, Subagent, Retrieval, and Compaction. The Set struct maintains an internal off map[Module]bool that tracks which modules are disabled for the current run. A zero-value Set represents the control arm where all modules remain enabled.

// Conceptual representation from internal/ablation/ablation.go
type Module string
const (
    Evidence   Module = "evidence"
    Planner    Module = "planner"
    Subagent   Module = "subagent"
    Retrieval  Module = "retrieval"
    Compaction Module = "compaction"
)

type Set struct {
    off map[Module]bool
}

Parsing and Construction

The Parse function converts user-provided strings into a Set. Valid inputs include comma-separated module names ("evidence,planner"), the string "all" (disables every module), or empty/"none"/"null" (control arm). The New constructor offers a programmatic alternative, accepting variadic Module arguments to create a disabled-module set directly.

// From internal/ablation/ablation.go
arm, err := ablation.Parse("evidence,planner")  // Disables two modules
control, _ := ablation.Parse("")               // Zero value, all enabled
allOff := ablation.New(ablation.Evidence, ablation.Retrieval)

Runtime Guards and Identification

The Off method provides the primary runtime guard, returning true if a specific module is disabled. This check appears throughout the codebase to conditionally skip logic. The Arm method generates stable, human-readable identifiers (e.g., "no-evidence+no-planner" or "full") for result grouping and historical comparison.

Integration with the Reasonix Runtime

The ablation system integrates at multiple layers, from command-line parsing to tool registration and result reporting.

CLI Flag Processing

The e2ebench command-line tool exposes an -ablate flag that accepts the same syntax as ablation.Parse. Located in cmd/e2ebench/main.go, this entry point validates the input and stores the resulting Set in the run configuration.

// Excerpt from cmd/e2ebench/main.go
ablateFlag := flag.String("ablate", "", "ablation arm: subsystems to switch off …")
arm, err := ablation.Parse(*ablateFlag)
if err != nil {
    // handle error
}

Agent Configuration Propagation

The parsed Set propagates through the initialization stack via Options.Ablation. In internal/boot/boot.go, the system invokes .WithAblation(opts.Ablation) to ensure the configuration reaches the agent and its subsystems.

// From internal/boot/boot.go
.WithAblation(opts.Ablation)   // propagates the set to the agent

Conditional Tool Registration

During bootstrapping, the system checks opts.Ablation.Off(module) before registering tools or enabling features. This conditional logic appears in several key paths:

Sub-agent Disabling: When ablation.Subagent is off, the system omits the task and read_only_task tools, preventing any delegation to sub-agents.

if opts.Ablation.Off(ablation.Subagent) {
    // Skip task tool registration
}

Retrieval Configuration: Disabling ablation.Retrieval removes BM25-backed history surfaces, leaving only direct-access session tools. The boot logic switches between retrieval-enabled and retrieval-disabled toolsets based on this check.

if opts.Ablation.Off(ablation.Retrieval) {
    reg.Add(sessiontool.NewListSessionsTool(sessionDir))
    reg.Add(sessiontool.NewReadSessionTool(sessionDir))
} else {
    reg.Add(history.NewTool(...))
    // BM25-backed retrieval tools...
}

Compaction Logic: The agent's compaction system checks a.ablation.Off(ablation.Compaction) to determine whether to collapse snip/fold triggers, effectively disabling context compression when the module is off.

Benchmark Result Tagging

After completion, the Arm method provides a deterministic label for the current configuration. This identifier is stored in JSON and Markdown reports, enabling automated aggregation across experimental runs.

// From cmd/e2ebench/main.go
result.Arm = arm.Arm()  // e.g., "no-retrieval", "full", "no-evidence+no-planner"

Practical Usage Examples

Developers interact with the ablation system through CLI flags, programmatic APIs, and runtime checks.

Running Ablated Benchmarks

To test the system without the planner module, pass the module name to the -ablate flag:

go run ./cmd/e2ebench -ablate planner -out report.md

This command disables planner-specific tool registration and tags results with the "no-planner" arm identifier.

Creating Ablation Sets Programmatically

For unit tests or custom orchestration, construct sets directly using the New constructor:

import "reasonix/internal/ablation"

// Disable evidence and retrieval
set := ablation.New(ablation.Evidence, ablation.Retrieval)

fmt.Println(set.Arm())                  // => "no-evidence+no-retrieval"
fmt.Println(set.Off(ablation.Evidence)) // => true
fmt.Println(set.Off(ablation.Subagent)) // => false

Implementing Runtime Checks in Tools

Tool implementations can accept an ablation set and guard functionality accordingly:

func (t *TaskTool) WithAblation(set ablation.Set) *TaskTool {
    t.ablation = set
    return t
}

// Later, before invoking the sub-agent:
if t.ablation.Off(ablation.Subagent) {
    return errors.New("sub-agent disabled for this run")
}

Summary

• The ablation system in esengine/DeepSeek-Reasonix provides a Set struct and Module enum in internal/ablation/ablation.go to track disabled subsystems.
• CLI integration via the -ablate flag in cmd/e2ebench/main.go allows comma-separated module lists or "all" for complete disablement.
• Runtime guards using the Off method conditionally skip tool registration in internal/boot/boot.go and logic execution in internal/agent/agent.go.
• Stable identifiers generated by Arm() ensure reproducible benchmark labeling across code versions.
• Supported modules include Evidence, Planner, Subagent, Retrieval, and Compaction, each isolating a distinct reasoning capability.

Frequently Asked Questions

How do I disable multiple modules simultaneously in Reasonix?

Pass a comma-separated list to the -ablate flag, such as -ablate evidence,planner. The Parse function in internal/ablation/ablation.go splits this string and creates a Set with both modules disabled. Alternatively, use ablation.New(ablation.Evidence, ablation.Planner) programmatically.

What is the difference between "all" and empty string in the -ablate flag?

The string "all" disables every module in the enum, creating a minimal agent configuration for baseline testing. An empty string, or the literals "none" and "null", result in a zero-value Set where the off map is empty, representing the control arm with all subsystems enabled.

How does the ablation system affect benchmark reporting?

Each Set generates a stable identifier via the Arm method (e.g., "no-retrieval" or "no-evidence+no-planner"). In cmd/e2ebench/main.go, this identifier is assigned to result.Arm and included in output reports, allowing automated grouping and comparison of results across different experimental configurations.

Can I check ablation status inside individual tool implementations?

Yes. Tools can accept an ablation.Set via a setter method like WithAblation and store it as a field. Before executing subsystem-specific logic, call set.Off(ablation.ModuleName) to determine if the current run has disabled that particular capability, returning errors or skipping logic as appropriate.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}