# Configuring CodeAgent v2 with Hard Validation in AutoResearchClaw: A Complete Guide

> Learn to configure CodeAgent v2 with hard validation in AutoResearchClaw. Enable AST syntax checks and LLM repair for robust code analysis before sandbox execution. Get the complete guide.

- Repository: [AIMING Lab/AutoResearchClaw](https://github.com/aiming-lab/AutoResearchClaw)
- Tags: how-to-guide
- Published: 2026-05-28

---

**Enable hard validation in AutoResearchClaw by setting `hard_validation=True` in `CodeAgentConfig`, which triggers AST-based syntax checks and a targeted LLM repair loop before sandbox execution.**

AutoResearchClaw’s **CodeAgent v2** is a multi-phase LLM-driven code-generation engine that plans, generates, and repairs research code. **Hard validation** serves as a critical quality gate between generation and execution, using static analysis to catch syntax errors, import issues, and structural problems before they hit the runtime environment.

## What Is Hard Validation?

Hard validation is Phase 2.5 of the CodeAgent v2 pipeline. According to the implementation in [`researchclaw/pipeline/code_agent.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/code_agent.py), this step runs immediately after sequential file generation to validate the entire codebase using Python’s `ast` module.

The system categorizes findings into two severity levels:

- **Critical issues**: Syntax errors, missing imports, undefined variables, missing `if __name__ == "__main__":` guards, and class quality violations. These trigger an automatic repair cycle.
- **Warnings**: Style concerns or trivial computations that are logged but do not block execution.

When critical issues are detected, the agent invokes `_hard_validate_and_repair()` (lines 567-605 in [`code_agent.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/code_agent.py)), which enters a targeted repair loop limited by `hard_validation_max_repairs`.

## Configuration Methods

You can enable hard validation through three interfaces: direct Python instantiation, YAML configuration, or runtime mutation.

### Python API Configuration

The most explicit method creates a `CodeAgentConfig` instance and embeds it within `ExperimentConfig` (defined in [`researchclaw/config.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/config.py)):

```python
from researchclaw.config import ExperimentConfig, CodeAgentConfig

# Enable CodeAgent v2 with hard validation and 3 repair attempts

config = ExperimentConfig(
    code_agent=CodeAgentConfig(
        enabled=True,
        hard_validation=True,
        hard_validation_max_repairs=3,
        sequential_generation=True,
        architecture_planning=True
    )
)

```

Pass this `config` object to `run_experiment()` or the CLI entry point.

### YAML and CLI Configuration

For batch experiments or reproducible workflows, configure via YAML:

```yaml

# experiment.yaml

code_agent:
  enabled: true
  hard_validation: true          # Activate AST validation gates

  hard_validation_max_repair: 5  # Allow up to 5 repair cycles

  sequential_generation: true
  architecture_planning: true

```

Execute with:

```bash
python -m researchclaw.cli --config experiment.yaml

```

The `ExperimentConfig.code_agent` field (lines 61-62 in [`researchclaw/config.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/config.py)) automatically maps these YAML keys to the `CodeAgentConfig` dataclass.

### Runtime Overrides

For interactive development or Jupyter notebooks, modify an existing configuration using `replace()`:

```python
exp_cfg = ExperimentConfig()
exp_cfg = exp_cfg.replace(
    code_agent=exp_cfg.code_agent.replace(
        hard_validation=False,  # Disable for faster prototyping

        hard_validation_max_repairs=1
    )
)

```

## The Validation and Repair Pipeline

Understanding the internal mechanics helps tune the `hard_validation_max_repairs` parameter effectively.

### AST-Based Checks

The `_hard_validate()` method (lines 707-784 in [`researchclaw/pipeline/code_agent.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/code_agent.py)) parses each generated Python file and runs validators from [`researchclaw/experiment/validator.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/experiment/validator.py) (lines 497-590):

- **Syntax validation**: `ast.parse()` failures are always critical.
- **Class quality**: Detects empty subclasses, duplicate implementations, or improper `nn.Module` usage.
- **Complexity analysis**: Flags hard-coded metrics that bypass actual computation.
- **API correctness**: Validates that all referenced names exist in scope or imports.
- **Import consistency**: Cross-file imports must resolve to existing symbols.
- **Entry point guard**: Missing `if __name__ == "__main__":` in [`main.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/main.py) is treated as critical.

### The Repair Loop

When `_hard_validate()` returns critical issues, `_repair_critical_issues()` (lines 722-734) constructs a detailed prompt containing:

1. The original architecture blueprint.
2. The current file contents.
3. A enumerated list of critical errors.

The LLM responds with corrected code blocks formatted as ` ```filename:xxx.py ``` `. The agent merges these updates and reruns validation until either zero critical issues remain or the repair counter reaches `hard_validation_max_repairs`.

## Complete Usage Example

Below is a minimal script that generates a CNN training script with hard validation enabled:

```python
import pathlib
from researchclaw.cli import run_experiment
from researchclaw.config import ExperimentConfig, CodeAgentConfig

# Configure with hard validation active

cfg = ExperimentConfig(
    code_agent=CodeAgentConfig(
        enabled=True,
        hard_validation=True,
        hard_validation_max_repairs=4,
        sequential_generation=True
    )
)

# Research topic

topic = "Train a CNN on CIFAR-10 with early stopping"

# Execute pipeline

result = run_experiment(
    topic=topic,
    config=cfg,
    work_dir=pathlib.Path("./validation_demo")
)

# Inspect results

print("Generated files:", list(result.files.keys()))
print("Validation log:")
for entry in result.validation_log:
    print(entry)

```

Typical output shows the validation gate in action:

```

[CodeAgent] Phase 2.5: Hard validation gates
  CRITICAL: [main.py] Missing `if __name__ == "__main__":` block
  WARNING: [model.py] Trivial computation detected
  Targeted repair for critical issues
  Repair updated 1 file(s): main.py
  Hard validation passed (1 warning(s), attempt 1)

```

## Summary

- **Hard validation** in AutoResearchClaw’s CodeAgent v2 acts as a static analysis gate between generation and execution, catching syntax and structural errors early.
- Configure it via `CodeAgentConfig(hard_validation=True, hard_validation_max_repairs=N)` in Python or YAML.
- Critical issues found by `_hard_validate()` in [`researchclaw/pipeline/code_agent.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/code_agent.py) trigger a targeted repair loop using `_repair_critical_issues()`.
- Validation helpers in [`researchclaw/experiment/validator.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/experiment/validator.py) check for AST validity, import consistency, class quality, and the presence of [`main.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/main.py) guards.
- Higher `hard_validation_max_repairs` values increase robustness at the cost of additional LLM token usage and latency.

## Frequently Asked Questions

### What types of errors does hard validation catch?

Hard validation catches **syntax errors**, **missing imports**, **undefined variables**, **cross-file import inconsistencies**, and **missing `__main__` guards** in [`main.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/main.py). These are classified as critical issues because they would cause immediate runtime failures. The system also logs **warnings** for style issues and trivial computations, but these do not trigger repair cycles.

### How many repair attempts should I allow?

The optimal `hard_validation_max_repairs` value depends on your LLM model and code complexity. For GPT-4 class models, **3-5 attempts** typically resolve syntax and import issues without excessive cost. If you observe repeated failures beyond 5 iterations, the architectural blueprint may be underspecified, and you should review the validation logs rather than increasing the limit indefinitely.

### Can I disable hard validation for faster iteration?

Yes. Set `hard_validation=False` in `CodeAgentConfig` or override it at runtime. This skips Phase 2.5 entirely, sending generated code directly to the execution-and-repair loop. While this reduces latency, you risk encountering avoidable runtime crashes that hard validation would have caught during the static analysis phase.

### Where are the validation rules defined?

The concrete AST checks are implemented in [`researchclaw/experiment/validator.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/experiment/validator.py) (lines 497-590). These include `check_class_quality`, `check_code_complexity`, `check_api_correctness`, and scoping validators. The orchestration logic that invokes these checks resides in `_hard_validate()` within [`researchclaw/pipeline/code_agent.py`](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/code_agent.py) (lines 707-784).