# How to Troubleshoot Experiment Self-Healing and Repair Cycles in AutoResearchClaw

> Troubleshoot experiment self-healing and repair in AutoResearchClaw. Learn how AutoResearchClaw detects failures, generates repairs with LLM or OpenCode, and scores results for quality.

- Repository: [AIMING Lab/AutoResearchClaw](https://github.com/aiming-lab/AutoResearchClaw)
- Tags: how-to-guide
- Published: 2026-05-28

---

**AutoResearchClaw automatically detects experimental failures through structured diagnosis, generates targeted repairs via LLM or OpenCode within a Docker sandbox, and promotes only higher-quality results back to the main pipeline after scoring them against the original.**

AutoResearchClaw's self-healing mechanism transforms failed experiments into publication-ready research through automated diagnosis and repair cycles. Understanding how to troubleshoot these repair cycles is essential when experiments stall at the `technical_report` quality level or fail to produce valid [`experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_summary.json) files. This guide breaks down the internal architecture, file paths, and debugging strategies for the repair pipeline implemented in the `aiming-lab/AutoResearchClaw` repository.

## Understanding the Self-Healing Architecture

The repair system consists of specialized components that work sequentially to diagnose, patch, and validate experiments. Each component is implemented in specific source files within the repository.

### Experiment Diagnosis Engine

The **`ExperimentDiagnosis`** class in [`researchclaw/pipeline/experiment_diagnosis.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/experiment_diagnosis.py) (lines 44-77) serves as the entry point for repair cycles. This component analyzes stdout, stderr, runtime metrics, and the experiment plan to produce a structured list of **deficiencies**. These deficiency objects categorize failures into types such as `MISSING_DEPENDENCY`, `GPU_OOM`, and `TIME_GUARD_DOMINANT`.

### Repair Prompt Construction and Execution

The **`build_repair_prompt`** function in [`researchclaw/pipeline/experiment_repair.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/experiment_repair.py) (lines 88-94) transforms a diagnosis into a detailed LLM prompt. This prompt includes:
- The diagnosed deficiencies
- A scope-reduction guide for time-constrained repairs
- Current source files requiring modification

The repair executor then calls **`_get_repaired_code`** or **`_repair_via_llm`** (lines 158-227) to invoke OpenCode (if enabled) or fall back to a plain LLM chat. These methods extract Python code blocks from the response and merge them with the original experiment code.

### Sandbox Execution and Quality Reassessment

After patching, **`_run_experiment_in_sandbox`** (lines 771-802) executes the modified experiment inside a Docker-based sandbox and returns new stdout, stderr, and metrics. The system then re-runs **`assess_experiment_quality`** on the new [`experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_summary.json) to determine if the repair succeeded.

The **`select_best_results`** function (lines 96-128) scores every version using **`_summary_quality_score`**, comparing metrics richness, completed conditions, and primary metric presence to select the optimal result.

## The Experiment Repair Cycle Flow

According to the implementation in [`researchclaw/pipeline/runner.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/runner.py) and [`experiment_repair.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_repair.py), the repair cycle follows this exact sequence:

1. **Stage 14** produces the initial [`experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_summary.json)
2. **Stage 15** runs `assess_experiment_quality`
3. If quality is insufficient, **`diagnose_experiment`** identifies specific deficiencies
4. **`build_repair_prompt`** constructs the repair instructions
5. The repair executor generates patched files in `stage-14_repair_vN/experiment/` directories
6. **`_run_experiment_in_sandbox`** executes the patched version
7. Quality is reassessed on the new summary
8. The loop repeats up to **`MAX_REPAIR_CYCLES`** (default 3) or until reaching full-paper mode
9. **`select_best_results`** chooses the highest-scoring summary
10. The promotion guard validates the repair against the original before overwriting [`stage-14/experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/stage-14/experiment_summary.json)

The pipeline orchestration logic in **`_run_experiment_repair`** ([`runner.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/runner.py) lines 46-78) manages the handoff between diagnosis and repair execution.

## Common Failure Points and Debugging Strategies

| Symptom | Root Cause | Debug Strategy |
|---------|------------|----------------|
| `ModuleNotFoundError: No module named 'xyz'` | **Missing dependency** (`DeficiencyType.MISSING_DEPENDENCY`) | Check the generated repair prompt for [`requirements.txt`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/requirements.txt) modifications; manually verify dependencies in the sandbox environment |
| `CUDA out of memory` | **GPU OOM** (`DeficiencyType.GPU_OOM`) | Inspect the prompt's suggested fixes for batch size reduction or gradient checkpointing; verify GPU memory limits in the Docker sandbox |
| >50% of conditions skipped | **Time-guard dominance** (`DeficiencyType.TIME_GUARD_DOMINANT`) | Review the automatically injected *Scope Reduction* section in the repair prompt, which suggests keeping only baseline + proposed + one ablation and reducing epochs |
| Missing [`experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_summary.json) after repair | Sandbox crash or timeout | Examine `stage-14_repair_vN/sandbox/*.json` for stdout/stderr; increase `repair_cfg.timeout_sec_per_cycle` in configuration |
| Mode remains `technical_report` after cycles | All repairs scored lower than original | Analyze [`experiment_repair_result.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_repair_result.json) → `cycle_history[i].diagnosis_summary` for deficiency patterns; consider enabling OpenCode via `experiment.repair.use_opencode: true` or adjusting hyper-parameters |

## The Promotion Guard and BUG-198 Protection

The promotion guard in [`pipeline/runner.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/pipeline/runner.py) (lines 70-100) contains a critical safety mechanism documented as **BUG-198**:

> *Only promote if the repair summary is RICHER than the existing stage-14 summary. The repair loop can produce empty summaries (metrics: {}, 0 conditions) which would overwrite enriched data from the analysis stage.*

This guard compares scores computed by `_summary_quality_score`, where higher scores indicate more completed conditions, present primary metrics, and richer metric keys. If the repair score does not exceed the original, the canonical [`stage-14/experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/stage-14/experiment_summary.json) remains unchanged. This prevents downstream stages (sanitizer, paper verifier) from receiving degraded or empty data sets.

## Practical Code Examples for Manual Intervention

### Inspecting Repair Artifacts

After a pipeline run completes, examine the self-healing process through the generated artifacts:

```bash

# View the complete repair cycle history

cat artifacts/rc-20240528-123456/experiment_repair_result.json | jq

# Locate the best repaired code version

ls artifacts/rc-20240528-123456/stage-14_repair_v*/experiment/

```

### Manual Repair Prompt Generation

For debugging specific failures without running the full pipeline:

```python
from pathlib import Path
import json
from researchclaw.pipeline.experiment_diagnosis import diagnose_experiment
from researchclaw.pipeline.experiment_repair import build_repair_prompt

run_dir = Path("artifacts/rc-20240528-123456")
summary = json.loads((run_dir / "stage-14/experiment_summary.json").read_text())
plan = json.loads((run_dir / "stage-09/experiment_design.json").read_text())

# Run diagnosis with captured output

diagnosis = diagnose_experiment(
    experiment_summary=summary,
    experiment_plan=plan,
    stdout="",
    stderr=""
)

# Gather current source files

code = {
    p.name: p.read_text()
    for p in (run_dir / "stage-10/experiment").glob("*.py")
}

# Generate the repair prompt

prompt = build_repair_prompt(
    diagnosis, 
    code, 
    experiment_plan=plan, 
    time_budget_sec=2400
)

print(prompt)  # Feed this directly to an LLM for testing

```

### Extracting Code from LLM Responses

When working with manual LLM interventions, use the internal extraction utility:

```python
from researchclaw.pipeline.experiment_repair import _extract_code_blocks

llm_response = """```python main.py
import torch

# fixed imports

```

```python requirements.txt
torch==2.2.0
numpy>=1.24

```"""

fixed_files = _extract_code_blocks(llm_response)

# Returns: {"main.py": "import torch\n# fixed imports", 

#          "requirements.txt": "torch==2.2.0\nnumpy>=1.24"}

```

### Configuring Repair Cycle Limits

Override the default `MAX_REPAIR_CYCLES = 3` in your configuration file:

```yaml
experiment:
  repair:
    enabled: true
    max_cycles: 5
    timeout_sec_per_cycle: 900
    use_opencode: true

```

## Summary

- **Structured diagnosis** in [`experiment_diagnosis.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_diagnosis.py) categorizes failures into specific deficiency types (dependencies, OOM, time-guards) that drive targeted repairs
- **The repair loop** runs up to 3 cycles by default, executing patched code in Docker sandboxes and re-assessing quality through `_summary_quality_score`
- **BUG-198 protection** prevents the promotion of empty or degraded summaries, ensuring only richer results overwrite the canonical stage-14 data
- **Sandbox outputs** in `stage-14_repair_vN/sandbox/` contain the debugging data needed when experiments crash or time out during repair attempts
- **Manual debugging** is supported through `build_repair_prompt` and `_extract_code_blocks` for isolated testing of repair strategies

## Frequently Asked Questions

### Why does my experiment remain in `technical_report` mode after repair cycles complete?

When all repair cycles finish but the quality mode remains `technical_report`, the promotion guard (BUG-198) has rejected all repairs because they scored lower than the original summary. Check [`experiment_repair_result.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/experiment_repair_result.json) → `cycle_history` to view each cycle's diagnosis summary. The repair likely failed to address the root deficiencies, or the original experiment had incomplete but higher-scoring metrics than the attempted fixes.

### How do I manually trigger or debug a specific repair cycle?

Import the diagnosis and repair functions directly in a Python REPL as shown in the manual prompt generation example above. This allows you to inspect the exact prompt sent to the LLM, modify the source code inputs, and test different repair strategies without re-running the entire pipeline from stage 1.

### What prevents repair cycles from overwriting good data with empty results?

The promotion guard in [`pipeline/runner.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/pipeline/runner.py) implements BUG-198 protection by comparing scores via `_summary_quality_score`. This function evaluates metric richness, condition completion, and primary metric presence. Only summaries with strictly higher scores than the original can overwrite [`stage-14/experiment_summary.json`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/stage-14/experiment_summary.json), protecting downstream stages from empty dictionaries or zero-condition outputs.

### How do I resolve persistent missing dependency errors in the repair loop?

When `DeficiencyType.MISSING_DEPENDENCY` appears repeatedly, verify that the repair prompt correctly identifies the missing package in [`requirements.txt`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/requirements.txt). If using OpenCode (`use_opencode: true`), ensure the sandbox environment has network access for pip installations. For manual fixes, update [`stage-10/experiment/requirements.txt`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/stage-10/experiment/requirements.txt) before the repair cycle begins, or increase the `time_budget_sec` parameter to allow dependency installation during the sandbox execution.