How the VirtTestLoader Plugin Discovers and Loads VT Tests in Avocado

The VirtTestLoader plugin bridges Avocado’s core test discovery mechanism with the virttest (VT) framework by parsing Cartesian configuration files, filtering test variants, and returning VirtTest instances with bound parameters.

The avocado-framework/avocado-vt repository provides a specialized test loader that enables Avocado to discover and execute virtualization tests written for the VT framework. When you run avocado run -t vt, the VirtTestLoader plugin discovers and loads VT tests by transforming Cartesian configuration matrices into executable Avocado test cases.

VirtTestLoader Architecture and Registration

Plugin Registration via Entry Points

Avocado discovers loader plugins automatically through Python entry points. The VirtTestLoader class in avocado_vt/loader.py defines a static attribute name = "vt" (lines 99–100), which registers the plugin under the "vt" identifier. When users specify -t vt or when Avocado’s resolver encounters VT-specific references, it instantiates this loader.

Class Inheritance and Mixins

The loader inherits from avocado.core.loader.TestLoader and mixes in DiscoveryMixIn from avocado_vt/discovery.py. This architecture separates concerns: the base class handles Avocado integration, while DiscoveryMixIn provides Cartesian parser construction and parameter conversion utilities. The __init__ method (lines 101–127) merges extra VT parameters from the avocado_vt_extra_params argument into the Avocado configuration object, ensuring options like vt.extra_params persist through the discovery pipeline.

VT Test Discovery Workflow

Building the Cartesian Configuration Parser

The discovery process begins when VirtTestLoader.discover() calls _get_parser(), inherited from DiscoveryMixIn. This method instantiates VirtTestOptionsProcess using the current Avocado configuration and returns a virttest.cartesian_config.Parser object. The parser reads the Cartesian configuration files specified by vt.config and builds a matrix of test variants.

If vt.save_config is enabled, the loader persists the parsed configuration to disk via _save_parser_cartesian_config (discovery.py lines 30–39), preserving include statements and filters for debugging or reuse.

URL Filtering and Test Selection

When discover() receives a specific URL argument, it applies filtering logic (loader.py lines 174–186). The loader calls parser.only_filter(url) to retain only configuration entries matching the provided path or variant identifier. If the URL is invalid or references non-existent variants, the parser raises a ParserError, which the loader catches and converts into a "bad discovery" placeholder (NotAvocadoVTTest).

When operating in DiscoverMode.DEFAULT without a URL, the loader returns an empty list unless vt.config is explicitly set (lines 186–191). This guard prevents accidental execution of the entire VT test matrix.

Enumerating Cartesian Test Cases

For valid configurations, the loader iterates through parser.get_dicts() (line 193), which yields a dictionary for every Cartesian combination of parameters—representing variants such as guest OS, machine type, and test-specific options. Each dictionary becomes a distinct test case in the Avocado suite.

Parameter Conversion and Suite Assembly

Raw Cartesian dictionaries require transformation before Avocado can execute them. The loader delegates this to DiscoveryMixIn.convert_parameters(params) (discovery.py lines 41–64). This routine:

• Generates the final test name, respecting vt.short_names_when_config for abbreviated identifiers
• Handles spice-specific naming logic when applicable
• Injects an id key for unique test identification

The converted parameters are packaged as {'name': test_name, 'vt_params': params} and paired with the VirtTest class. The loader returns a list of (VirtTest, params) tuples to Avocado’s test suite builder.

Error Handling and Placeholder Tests

When discovery encounters malformed URLs, missing configuration files, or parser errors, the loader instantiates NotAvocadoVTTest (defined in loader.py lines 86–90). This placeholder class inherits from Avocado’s base Test but carries a special "!VT" label. It allows Avocado to report the discovery failure within the test list rather than failing silently, providing clear feedback about configuration issues.

Code Examples

Running VT Tests from the Command Line

The most common way to trigger the VirtTestLoader is through Avocado’s CLI:

# Run all VT tests defined in a specific cartesian configuration file

avocado run -t vt --vt-config=/path/to/my.cfg

# Run a specific VT test variant by URL

avocado run -t vt "myguest/variants.cfg#variant1"

When executing these commands, Avocado instantiates VirtTestLoader, calls discover() with the appropriate URL or None, parses the Cartesian configuration, and executes each discovered VirtTest instance.

Programmatic Discovery with VirtTestLoader

For custom test runners or debugging purposes, you can invoke the loader directly:

from avocado_vt.loader import VirtTestLoader
from avocado.core import loader as avocado_loader

# Minimal Avocado configuration dictionary (normally built by Avocado)

cfg = {
    'vt.config': '/path/to/my.cfg',
    'avocado.runner.output_dir': '/tmp'
}

# No extra parameters

extra = {}

# Instantiate the loader

vt_loader = VirtTestLoader(cfg, extra)

# Discover all tests (equivalent to avocado run -t vt)

suite = vt_loader.discover(
    url=None,
    which_tests=avocado_loader.DiscoverMode.ALL
)

for test_cls, params in suite:
    print("Discovered:", params['name'])
    # test_cls is VirtTest; you could instantiate it if you wanted to run manually

This script demonstrates the internal discovery flow, printing each VT test name generated by the Cartesian parser and parameter conversion logic.

Filtering Tests by URL

To select specific variants without loading the entire test matrix:

url = "myguest/variants.cfg#variant1"
suite = vt_loader.discover(
    url=url,
    which_tests=avocado_loader.DiscoverMode.DEFAULT
)

# Returns only the matching test or an empty list if the URL does not exist

The only_filter method of the Cartesian parser ensures only configuration entries matching the URL pattern are included in the final test suite.

Summary

• The VirtTestLoader plugin in avocado_vt/loader.py implements the Avocado TestLoader interface to bridge Avocado with the virttest framework.
• It registers under the name "vt" via entry points, allowing activation with -t vt.
• Discovery relies on Cartesian configuration parsing through DiscoveryMixIn._get_parser(), which builds a matrix of test variants from vt.config files.
• The loader filters tests by URL using parser.only_filter(), converts raw parameters via convert_parameters(), and returns tuples of (VirtTest, params).
• Error handling uses NotAvocadoVTTest placeholders to report bad discoveries without breaking the test suite.
• The actual test execution is handled by the VirtTest class in avocado_vt/test.py, which translates VT outcomes into Avocado’s reporting model.

Frequently Asked Questions

How does VirtTestLoader handle invalid or malformed VT test URLs?

When VirtTestLoader.discover() receives a URL that does not match any Cartesian configuration entry, it calls parser.only_filter(url), which raises a ParserError. The loader catches this exception (lines 174–186 in avocado_vt/loader.py) and returns a NotAvocadoVTTest placeholder instance labeled with "!VT". This allows Avocado to display the discovery failure in the test list without crashing the entire discovery process.

What is the difference between DiscoverMode.ALL and DiscoverMode.DEFAULT in VirtTestLoader?

DiscoverMode.ALL instructs the loader to return every test variant found in the Cartesian configuration, effectively enumerating the entire test matrix. DiscoverMode.DEFAULT activates a safety guard: when no URL is specified and the mode is DEFAULT, the loader returns an empty list unless vt.config is explicitly set (lines 186–191). This prevents accidental execution of the full VT suite, which could contain thousands of variants.

Can I use VirtTestLoader without the Avocado command-line interface?

Yes, you can instantiate VirtTestLoader programmatically by passing an Avocado configuration dictionary and optional extra parameters to the constructor. You then call discover(url, which_tests) to retrieve a list of (VirtTest, params) tuples. This approach is useful for custom test runners, debugging tools, or integration with CI systems that need to inspect the VT test matrix before execution.

How does VirtTestLoader convert Cartesian parameters into Avocado-compatible test names?

The loader delegates parameter conversion to DiscoveryMixIn.convert_parameters() in avocado_vt/discovery.py (lines 41–64). This method processes the raw dictionary from the Cartesian parser, applies naming logic based on vt.short_names_when_config, handles spice-specific naming conventions when applicable, and injects a unique id key. The result is packaged as {'name': test_name, 'vt_params': params} and paired with the VirtTest class for execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}