Using AutoAttackWrapper for Adversarial Robustness Evaluation in VERONA

The AutoAttackWrapper class in VERONA integrates the AutoAttack library into the framework's verification pipeline by implementing the abstract Attack interface, enabling standardized adversarial robustness evaluation with configurable norms and attack versions.

VERONA provides a ready-to-use wrapper that bridges the popular AutoAttack library with its modular verification architecture. The AutoAttackWrapper class, located in ada_verona/verification_module/attacks/auto_attack_wrapper.py, allows researchers to evaluate model robustness using state-of-the-art adversarial attacks without modifying the underlying framework code. This integration supports both standalone attack execution and seamless pipeline integration through a unified API.

Architecture and Implementation

The Attack Interface

All adversarial attacks in VERONA implement the abstract Attack class defined in ada_verona/verification_module/attacks/attack.py. This interface standardizes the execution API across different attack methods, requiring concrete implementations to define an execute(model, data, target, epsilon) method that returns perturbed data as a torch.Tensor.

AutoAttackWrapper Execution Flow

The AutoAttackWrapper class extends this interface to wrap the external AutoAttack library. During initialization, it stores the execution device, norm type (Linf or L2), attack version (standard by default), and verbosity settings. When execute() is called, the wrapper:

1. Instantiates an AutoAttack object with the supplied model and parameters
2. Adds a batch dimension to the input data using data.unsqueeze(0) to satisfy AutoAttack's NCHW format expectations
3. Invokes run_standard_evaluation to generate adversarial examples
4. Normalizes the output by extracting the perturbed tensor if AutoAttack returns a tuple (perturbed, predictions)

This normalization ensures backward compatibility while handling variations in the AutoAttack library return types.

How to Use AutoAttackWrapper

Basic Standalone Usage

You can instantiate and run the wrapper independently for single-sample adversarial robustness evaluation:

import torch
from ada_verona.verification_module.attacks.auto_attack_wrapper import AutoAttackWrapper

# Assume model is a torch.nn.Module and data is C×H×W format

model = my_model.eval()
data = test_data          # Shape: (C, H, W)

target = test_target      # Shape: (1,)

# Initialize the wrapper

attack = AutoAttackWrapper(
    device="cpu", 
    norm="Linf", 
    version="standard", 
    verbose=False
)

# Execute attack with epsilon perturbation budget

epsilon = 0.05
perturbed = attack.execute(model, data, target, epsilon)

print("Perturbed shape:", perturbed.shape)   # torch.Size([C, H, W])

Integration with Verification Pipeline

For batch evaluation across datasets, plug the wrapper into VERONA's AttackEstimationModule:

import torch
from ada_verona.verification_module.attacks.auto_attack_wrapper import AutoAttackWrapper
from ada_verona.verification_module.attack_estimation_module import AttackEstimationModule
from ada_verona.database.verification_context import VerificationContext

# Setup model and dataset

model = torch.load("my_model.pt").to("cuda")
dataset = ...   # Dataset yielding (x, y) tuples

# Create verification context for specific sample

vc = VerificationContext(
    model=model,
    dataset=dataset,
    sample_index=0,
)

# Configure estimator with AutoAttackWrapper

estimator = AttackEstimationModule(
    attack=AutoAttackWrapper(
        device="cuda", 
        norm="Linf", 
        version="standard", 
        verbose=False
    )
)

# Estimate robustness

epsilon = 0.1
result = estimator.estimate(vc, epsilon)
print("Robustness result:", result)

The AttackEstimationModule internally invokes attack.execute() for each data point, making the wrapper interchangeable with other attack implementations. See the complete workflow in examples/scripts/create_robustness_dist_autoattack.py.

Testing and Validation

The implementation includes comprehensive unit tests in tests/test_verification_module/attacks/test_auto_attack_wrapper.py. These tests verify correct parameter initialization, device placement, and execution flow using monkey-patched AutoAttack instances to ensure reproducibility without external dependencies.

Summary

• AutoAttackWrapper provides a standardized interface between the AutoAttack library and VERONA's verification framework through the abstract Attack class in ada_verona/verification_module/attacks/attack.py.
• The wrapper handles data format conversion (adding batch dimensions via unsqueeze(0)) and normalizes return values to ensure consistent tensor outputs.
• Configuration options include device, norm (Linf/L2), version, and verbose flags for flexible deployment across different hardware and attack specifications.
• Integration with AttackEstimationModule enables large-scale adversarial robustness evaluation across entire datasets without pipeline modifications.
• Reference implementations are available in examples/scripts/create_robustness_dist_autoattack.py with test coverage in tests/test_verification_module/attacks/test_auto_attack_wrapper.py.

Frequently Asked Questions

What is the AutoAttackWrapper in VERONA?

The AutoAttackWrapper is a concrete implementation of VERONA's abstract Attack interface that integrates the AutoAttack library for adversarial robustness evaluation. According to the source code in ada_verona/verification_module/attacks/auto_attack_wrapper.py, it translates between VERONA's standardized verification pipeline and AutoAttack's specific API requirements, allowing researchers to use state-of-the-art adversarial attacks without modifying framework internals.

How does AutoAttackWrapper handle data formats?

The wrapper automatically adjusts input dimensions to match AutoAttack's expectations. In the execute() method implemented in auto_attack_wrapper.py, it applies data.unsqueeze(0) to add a batch dimension, converting single samples from (C, H, W) to (1, C, H, W) format. It also handles variable return types from AutoAttack, extracting the perturbed tensor when the library returns a tuple (perturbed, predictions) instead of a single tensor.

Can I use different attack norms with AutoAttackWrapper?

Yes, the wrapper supports configurable norm types through the norm parameter in its constructor. You can specify "Linf" for L-infinity norm attacks or "L2" for L2 norm attacks. The default is "Linf", which is the standard setting for most adversarial robustness benchmarks. This parameter is passed directly to the underlying AutoAttack instance during execution.

Where can I find examples of AutoAttackWrapper usage?

Practical examples are available in the repository at examples/scripts/create_robustness_dist_autoattack.py, which demonstrates end-to-end robustness distribution generation using the AttackEstimationModule. Additionally, unit tests in tests/test_verification_module/attacks/test_auto_attack_wrapper.py illustrate proper initialization and execution patterns, including mocking strategies for testing without requiring the full AutoAttack dependency.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}