Implementing Custom Attack Methods in VERONA's AttackEstimationModule

To implement a custom attack in VERONA, subclass the abstract Attack class, implement the execute method, and pass the instance to AttackEstimationModule for verification.

The ada-research/verona repository provides a plug‑in architecture for adversarial robustness evaluation that lets you drop in new attack strategies without modifying the core verification engine. The Attack Estimation Module (AttackEstimationModule) accepts any object implementing the Attack interface and uses it to generate perturbed inputs during verification.

Understanding the Attack Interface in VERONA

All custom attacks must inherit from the abstract base class defined in ada_verona/verification_module/attacks/attack.py. The contract is minimal: you only need to implement the execute method.

The required signature is:

def execute(self, model: Module, data: Tensor, target: Tensor, epsilon: float) -> Tensor:

• model – The torch.nn.Module being verified.
• data – The input Tensor (typically a normalized image in [0, 1]).
• target – The ground‑truth label Tensor.
• epsilon – The maximum perturbation bound (L‑inf radius).

The method must return a perturbed Tensor of the same shape as data, clipped to the valid input range.

Step-by-Step Implementation Guide

Step 1: Create a Custom Attack Class

Create a new file in ada_verona/verification_module/attacks/ and subclass Attack. Below is a complete example implementing a RandomNoiseAttack that adds uniform noise bounded by epsilon:

# File: ada_verona/verification_module/attacks/random_noise_attack.py

import torch
from torch import Tensor
from torch.nn.modules import Module
from ada_verona.verification_module.attacks.attack import Attack

class RandomNoiseAttack(Attack):
    """Add uniform random noise bounded by epsilon."""
    
    def __init__(self, seed: int = 0) -> None:
        super().__init__()
        self.seed = seed
        self.name = f"RandomNoiseAttack (seed={seed})"

    def execute(self, model: Module, data: Tensor, target: Tensor, epsilon: float) -> Tensor:
        torch.manual_seed(self.seed)
        noise = torch.empty_like(data).uniform_(-epsilon, epsilon)
        perturbed = torch.clamp(data + noise, 0.0, 1.0)
        return perturbed

Key implementation details:

• Call super().__init__() to initialize the base Attack class.
• Set self.name for logging and debugging purposes.
• Ensure the returned tensor respects the input domain (clip to [0, 1] for images).

Step 2: Integrate with AttackEstimationModule

The AttackEstimationModule constructor in ada_verona/verification_module/attack_estimation_module.py accepts two arguments:

• attack – An instance of your custom Attack subclass.
• top_k – The number of top predictions to check against the original label (typically 1).

from ada_verona.verification_module.attack_estimation_module import AttackEstimationModule
from ada_verona.verification_module.attacks.random_noise_attack import RandomNoiseAttack

# Instantiate your custom attack

custom_attack = RandomNoiseAttack(seed=42)

# Create the estimation module

estimator = AttackEstimationModule(attack=custom_attack, top_k=1)

Step 3: Execute Verification

The verify method requires a VerificationContext configured with a One2AnyPropertyGenerator. This is currently the only supported property generator for adversarial verification in VERONA.

The verification pipeline:

1. Loads the model from the context.
2. Moves data to the appropriate device (CPU/GPU).
3. Calls self.attack.execute(...) (lines 70‑71 of attack_estimation_module.py).
4. Evaluates whether the original target label appears in the top‑k predictions.

import torch
from ada_verona.database.verification_context import VerificationContext
from ada_verona.verification_module.property_generator.one2any_property_generator import One2AnyPropertyGenerator
from ada_verona.database.dataset.pytorch_experiment_dataset import PyTorchExperimentDataset
from ada_verona.database.machine_learning_model.torch_model_wrapper import TorchModelWrapper

# 1. Load model and dataset

model_wrapper = TorchModelWrapper(model_path="models/mnist_cnn.pt")
dataset = PyTorchExperimentDataset(dataset_path="datasets/mnist_test.pt")
data_point = dataset[0]

# 2. Create verification context with One2AnyPropertyGenerator

prop_gen = One2AnyPropertyGenerator(target_label=data_point.label)
verification_context = VerificationContext(
    network=model_wrapper,
    data_point=data_point,
    property_generator=prop_gen,
)

# 3. Run verification with custom attack

epsilon = 0.2
result = estimator.verify(verification_context, epsilon)

print(f"Verification result: {result.result}")  # SAT or UNSAT

print(f"Duration: {result.took:.3f}s")
print(f"Predicted labels: {result.obtained_labels.squeeze().tolist()}")

Core Architecture Components

Understanding the relationship between these components helps debug integration issues:

Component	Responsibility	Source File
Attack (ABC)	Defines the execute API contract that all attacks must implement.	ada_verona/verification_module/attacks/attack.py
Concrete attacks (e.g., PGDAttack, FGSMAttack, AutoAttackWrapper)	Implement specific perturbation strategies.	ada_verona/verification_module/attacks/pgd_attack.py, ada_verona/verification_module/attacks/fgsm_attack.py, ada_verona/verification_module/attacks/auto_attack_wrapper.py
AttackEstimationModule	Orchestrates model loading, attack execution, and top‑k result checking.	ada_verona/verification_module/attack_estimation_module.py
VerificationContext	Bundles the model (network), data point (data_point), and property generator (property_generator).	ada_verona/database/verification_context.py
One2AnyPropertyGenerator	Currently the only supported property generator for adversarial verification.	ada_verona/verification_module/property_generator/one2any_property_generator.py

Testing Your Custom Attack

Extend the existing test suite to validate your implementation. The example below shows how to swap the default attack in a fixture with your custom implementation:

# tests/test_verification_module/test_custom_random_noise.py

import pytest
import torch
from ada_verona.verification_module.attacks.random_noise_attack import RandomNoiseAttack
from ada_verona.verification_module.attack_estimation_module import AttackEstimationModule

def test_random_noise_attack_estimation():
    # Setup test fixtures

    attack = RandomNoiseAttack(seed=1)
    estimator = AttackEstimationModule(attack=attack, top_k=1)
    
    # Assuming verification_context is provided by a fixture

    epsilon = 0.3
    result = estimator.verify(verification_context, epsilon)
    
    assert result.result in ("SAT", "UNSAT")
    assert isinstance(result.took, float)
    assert result.obtained_labels is not None

Run the test with pytest tests/test_verification_module/test_custom_random_noise.py -v to confirm VERONA correctly integrates your attack.

Summary

• Subclass Attack from ada_verona/verification_module/attacks/attack.py and implement the execute method to create a custom adversarial attack.
• Instantiate AttackEstimationModule with your attack object and a top_k parameter to configure the verification pipeline.
• Use One2AnyPropertyGenerator as the property generator in your VerificationContext, as it is currently the only supported generator for adversarial verification.
• Return perturbed tensors that respect the input domain (typically [0, 1] for images) and match the shape of the original data.
• Test your implementation by running the verify method and checking for SAT (attack successful) or UNSAT (attack failed) results.

Frequently Asked Questions

What is the required signature for the execute method in VERONA?

The execute method must accept four parameters: model (a torch.nn.Module), data (the input Tensor), target (the ground‑truth label Tensor), and epsilon (the perturbation bound as a float). It must return a Tensor of the same shape as data containing the adversarial example. This contract is defined in ada_verona/verification_module/attacks/attack.py.

Can I use external attack libraries like AutoAttack with VERONA?

Yes. VERONA includes AutoAttackWrapper in ada_verona/verification_module/attacks/auto_attack_wrapper.py, which demonstrates how to wrap external libraries. You can create a similar wrapper for other libraries by subclassing Attack and calling the external library's methods inside your execute implementation, ensuring you convert tensors to the expected format and clip outputs to the valid input range.

What property generator should I use with AttackEstimationModule?

You must use One2AnyPropertyGenerator, located in ada_verona/verification_module/property_generator/one2any_property_generator.py. The verify method in AttackEstimationModule explicitly checks that the supplied VerificationContext uses this generator type, as it is currently the only supported property generator for adversarial robustness verification in VERONA.

How does AttackEstimationModule determine if an attack succeeded?

The module calls self.attack.execute(...) to generate a perturbed input, runs the model on that input, and checks whether the original target label appears in the top‑k predictions (where k is set via the top_k constructor parameter). If the original label is not in the top‑k, the result is SAT (satisfiable, meaning the attack succeeded). If the label remains in the top‑k, the result is UNSAT (unsatisfiable, attack failed). This logic is implemented in lines 60‑82 of ada_verona/verification_module/attack_estimation_module.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}