# How to Export Verification Results to ExperimentRepository in VERONA

> Learn how to export verification results to ExperimentRepository in VERONA. Save outcomes to CSV, generate plots, and store data for reproducible experiments using save_result().

- Repository: [ADA research/verona](https://github.com/ada-research/verona)
- Tags: how-to-guide
- Published: 2026-02-23

---

**Exporting verification results to ExperimentRepository in VERONA involves calling `repo.save_result()` after running `AutoVerifyModule.verify()`, which persists outcomes to CSV files, generates plots via `ReportCreator`, and stores per-ε data for reproducible robustness experiments.**

VERONA is an open-source framework for neural network robustness verification developed by ADA Research. When executing verification experiments, the **ExperimentRepository** class serves as the central database that orchestrates directory creation, persists results, and generates visualizations. This guide explains how to export verification results to ExperimentRepository in VERONA using the `AutoVerifyModule` and the complete data persistence pipeline.

## Understanding the VERONA Verification Pipeline

The verification pipeline in VERONA consists of loosely-coupled components defined in [`ada_verona/verification_module/verification_module.py`](https://github.com/ada-research/verona/blob/main/ada_verona/verification_module/verification_module.py). The abstract `VerificationModule` interface defines the `verify` contract, while concrete implementations like `AutoVerifyModule` handle external verifier integration.

The pipeline flow involves four primary components:

- **VerificationContext** ([`ada_verona/database/verification_context.py`](https://github.com/ada-research/verona/blob/main/ada_verona/database/verification_context.py)) – Bundles the network, data point, temporary folder, and property generator.
- **AutoVerifyModule** ([`ada_verona/verification_module/auto_verify_module.py`](https://github.com/ada-research/verona/blob/main/ada_verona/verification_module/auto_verify_module.py)) – Builds VNNLIB properties, injects VERONA metadata, and calls external verifiers.
- **CompleteVerificationData** ([`ada_verona/database/verification_result.py`](https://github.com/ada-research/verona/blob/main/ada_verona/database/verification_result.py)) – Dataclass returned upon successful verification containing outcomes and counter-examples.
- **ExperimentRepository** ([`ada_verona/database/experiment_repository.py`](https://github.com/ada-research/verona/blob/main/ada_verona/database/experiment_repository.py)) – Persists results to CSV, generates plots, and saves contexts as YAML.

When `AutoVerifyModule.verify()` completes, it returns either a `CompleteVerificationData` object wrapped in a `Result.Ok` or an error string. The repository then persists this data across multiple targets including aggregated CSV files, per-epsilon CSVs, visualization plots, and YAML context files.

## Step-by-Step Guide to Exporting Results

The following workflow demonstrates how to initialize an experiment, run verification for a specific epsilon perturbation, and export the results to the repository:

```python
from pathlib import Path
from ada_verona.database.experiment_repository import ExperimentRepository
from ada_verona.verification_module.auto_verify_module import AutoVerifyModule
from ada_verona.database.dataset.data_point import DataPoint
from ada_verona.verification_module.property_generator.one2any_property_generator import One2AnyPropertyGenerator
from ada_verona.database.machine_learning_model.onnx_network import ONNXNetwork
from autoverify.verifier.verifier import Verifier

# 1. Initialize the experiment repository

repo = ExperimentRepository(
    base_path=Path("experiments"),
    network_folder=Path("networks")
)
repo.initialize_new_experiment("mnist_demo")

# 2. Load network and data point

network = ONNXNetwork(Path("examples/example_experiment/data/networks/mnist-net_256x2.onnx"))
image = Path("examples/example_experiment/data/images/mnist_train_0.pt")
data_point = DataPoint(label=5, data=load(image), id="0")

# 3. Create verification context

prop_gen = One2AnyPropertyGenerator()
ctx = repo.create_verification_context(network, data_point, prop_gen)

# 4. Initialize verifier

verifier = Verifier(name="sdpcrown")
auto_mod = AutoVerifyModule(verifier=verifier, timeout=300)

# 5. Run verification

epsilon = 0.05
result = auto_mod.verify(ctx, epsilon)

# 6. Export to repository

if isinstance(result, dict) or hasattr(result, "result"):
    repo.save_result(result)
else:
    print("Verification failed:", result)

# 7. Generate reports

repo.save_plots()
repo.save_per_epsilon_result_df()

```

## Persistence Targets in ExperimentRepository

When exporting verification results to ExperimentRepository in VERONA, the system creates four distinct persistence targets that together enable comprehensive analysis and reproducibility.

### Results CSV

The **Results CSV** (`results/result_df.csv`) is created by `ExperimentRepository.save_result(s)`. It contains one row per ε-run with columns for the network path, epsilon value, verification outcome, and timestamps.

### Per-ε CSV

The **Per-ε CSV** (`tmp/<network>/image_<id>/epsilons_df.csv`) stores the full list of ε values explored for a single image. These files are aggregated via `ExperimentRepository.save_per_epsilon_result_df()` for fine-grained analysis of robustness boundaries.

### Visualization Plots

Generated by `ExperimentRepository.save_plots()` using `ReportCreator` from [`ada_verona/analysis/report_creator.py`](https://github.com/ada-research/verona/blob/main/ada_verona/analysis/report_creator.py), the following plots are created in the `results/` directory:

- `hist_figure.png` – Histogram of verification outcomes
- `boxplot.png` – Distribution of epsilon values by outcome
- `kde_plot.png` – Kernel density estimation of results
- `ecdf_plot.png` – Empirical cumulative distribution function

### Context YAML

The **Context YAML** (`tmp/<network>/image_<id>/context.yaml`) is created by `ExperimentRepository.save_verification_context_to_yaml()`. This file enables full reproducibility by storing the complete verification context, allowing specific runs to be re-executed or debugged later.

## Metadata Injection for Reproducibility

When using the SDP-CROWN verifier, `AutoVerifyModule.verify()` injects VERONA-specific metadata directly into the VNNLIB property file. This occurs in [`ada_verona/verification_module/auto_verify_module.py`](https://github.com/ada-research/verona/blob/main/ada_verona/verification_module/auto_verify_module.py) (lines 67-80):

```python
if (self.verifier.name == "sdpcrown") and ("; verona_metadata_version:" not in vnnlib_property.content):
    image_flat = verification_context.data_point.data.detach().cpu().numpy().reshape(-1)
    image_csv = ",".join(f"{v:.8f}" for v in image_flat)
    header = (
        "; verona_metadata_version: 1\n"
        f"; verona_epsilon: {float(epsilon):.8f}\n"
        f"; verona_image_class: {int(verification_context.data_point.label)}\n"
        f"; verona_image: {image_csv}\n"
    )
    vnnlib_property.content = header + vnnlib_property.content

```

This header enables downstream tools to parse verification parameters including the exact epsilon value, image class label, and flattened pixel data, facilitating attack reproduction and result validation.

## Summary

- **ExperimentRepository** in VERONA acts as the central database for robustness experiments, managing directory structures and result persistence across multiple formats.
- Call `repo.save_result()` to append verification outcomes to `results/result_df.csv` after successful `AutoVerifyModule.verify()` execution.
- Use `repo.save_plots()` to generate histograms, box plots, KDE, and ECDF visualizations via the `ReportCreator` class.
- Per-ε data is stored in temporary CSV files and aggregated using `repo.save_per_epsilon_result_df()` for detailed boundary analysis.
- Full reproducibility is supported through YAML context serialization via `save_verification_context_to_yaml()`.
- VERONA metadata is embedded directly into VNNLIB files when using SDP-CROWN, storing epsilon values, image classes, and flattened image data for downstream parsing.

## Frequently Asked Questions

### What file format does VERONA use for storing verification results?

VERONA stores aggregated verification results in CSV format at `results/result_df.csv`. Each row represents a single verification run with columns for the network path, epsilon value, verification outcome, and timestamps. Per-epsilon explorations are stored in separate CSV files within the temporary experiment folder structure under `tmp/<network>/image_<id>/epsilons_df.csv`.

### How does ExperimentRepository handle failed verifications?

When `AutoVerifyModule.verify()` returns an error string rather than `CompleteVerificationData`, the repository's `save_result()` method should not be called with the error object. Instead, implement conditional logic to check `isinstance(result, dict)` or `hasattr(result, "result")` before invoking `repo.save_result()` to ensure only valid data is persisted to the CSV files.

### Can I regenerate plots without re-running verifications?

Yes. Since `ExperimentRepository.save_plots()` reads from the accumulated `results/result_df.csv` file and utilizes `ReportCreator` from [`ada_verona/analysis/report_creator.py`](https://github.com/ada-research/verona/blob/main/ada_verona/analysis/report_creator.py), you can regenerate histograms, box plots, KDE plots, and ECDF plots at any time after results have been saved, without re-executing the verification modules or reloading neural networks.

### What is the purpose of the VERONA metadata header in VNNLIB files?

The metadata header injected by `AutoVerifyModule` stores the epsilon value, image class label, and flattened image pixel values as semicolon-prefixed comments in the VNNLIB property file. This allows downstream analysis tools and attack reproduction scripts to parse the exact verification parameters and input data used during the robustness check, ensuring experimental reproducibility.