# How the Cartesian Config System Works in Avocado-VT: Variants, Filters, and Combinatorial Test Generation

> **The Avocado-VT Cartesian config system processes test configurations through a three-stage pipeline—lexical parsing into an AST, resolution of combinatorial joins and logical filters, and dictionary generation—to yield flat p...

- Repository: [avocado/avocado-vt](https://github.com/avocado-framework/avocado-vt)
- Tags: 
- Published: 2026-02-25

---

**The Avocado-VT Cartesian config system processes test configurations through a three-stage pipeline—lexical parsing into an AST, resolution of combinatorial joins and logical filters, and dictionary generation—to yield flat parameter sets for each concrete test scenario.**

The Cartesian config system in the `avocado-framework/avocado-vt` repository implements a domain-specific language for describing complex virtual machine test matrices. It allows developers to define combinatorial test scenarios using hierarchical variants, logical operators, and filter constraints, generating thousands of test permutations from concise declarative files.

## Cartesian Config Syntax and Operators

The configuration format uses a mini-language with distinct operators for logical combination and filtering. In [`virttest/cartesian_config.py`](https://github.com/avocado-framework/avocado-vt/blob/main/virttest/cartesian_config.py), the `parse_filter()` function (lines [1390‑1476](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L1390-L1476)) implements the formal grammar for these expressions.

**Core syntax constructs:**

- **`,` (OR)** – Matches any alternative. `qcow2, raw` selects configurations containing either *qcow2* or *raw*.
- **`..` (AND)** – Matches all parts regardless of order. `qcow2..raw` requires both labels to be present.
- **`.` (IMMEDIATELY-FOLLOWED-BY)** – Enforces adjacency. `qcow2.14` requires *14* to follow *qcow2* directly.
- **`- variant:`** – Defines a variant block where indented content belongs to that branch.
- **`only <filter>`** – Retains only configurations satisfying the filter expression.
- **`no <filter>`** – Discards configurations satisfying the filter expression.
- **`join <filter>`** – Performs a Cartesian product between the current node and configurations matching the filter.
- **`(var=value)`** – Specifies a variant constraint for precise matching.

## Core Data Structures

The parser builds an abstract syntax tree using several key classes defined in [`virttest/cartesian_config.py`](https://github.com/avocado-framework/avocado-vt/blob/main/virttest/cartesian_config.py):

- **`Node`** (line [1464](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L1464-L1470)): Represents configuration blocks, holding `name`, `labels`, `content`, and `children`. Content is stored as triples of `(filename, linenum, object)`.
- **`Label`** (lines [1550‑1612](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L1550-L1612)): Wraps variant names or `(var=value)` pairs with custom equality logic for fast matching.
- **`OnlyFilter` / `NoFilter`** (lines [63‑89](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L63-L89)): Implement `match()`, `might_match()`, `is_irrelevant()`, and `requires_action()` to drive the filter engine.
- **`Condition` / `NegativeCondition`** (lines [98‑108](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L98-L108)): Handle conditional blocks that activate only when their predicate matches.
- **`JoinFilter`** (lines [111‑114](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L111-L114)): Marks nodes requiring Cartesian product expansion.

## The Three-Stage Processing Pipeline

### Stage 1: Lexical Analysis and Parsing

The `Lexer` class (lines [81‑124](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L81-L124)) tokenizes input files line-by-line using `StrReader` or `FileReader` backends, yielding tokens like `LIdentifier`, `LOnly`, `LNo`, and `LJoin`. The `Parser._parse()` method (around line [1630](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L1630-L1640)) consumes this stream to construct the `Node` tree:

1. Identifiers followed by operators (`=`, `+=`, `?=`) create assignment nodes.
2. Variant markers (`-`) spawn new child nodes via recursive descent (starting line [1660](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L1660-L1670)).
3. Filter keywords instantiate `OnlyFilter`, `NoFilter`, or `JoinFilter` objects attached to the current node.

### Stage 2: Join Resolution and Cartesian Products

The `join` operator creates combinatorial explosions by multiplying configuration sets. In `Parser.get_dicts()` (starting line [2002](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L2002-L2050)), the system first converts each `JoinFilter` to an equivalent `OnlyFilter` (lines [58‑64](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L58-L64)), then invokes `multiply_join()` (lines [2082‑2110](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L2082-L2110)):

- Temporarily adds the first join filter to the node.
- If no joins remain, yields dictionaries from `get_dicts_plain()`.
- Otherwise recurses, merging dictionaries (`d.update(d2)`) and constructing test names via `mk_name()` (lines [70‑80](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L70-L80)).

This generates the Cartesian product of all joined variant groups.

### Stage 3: Filter Application and Dictionary Yielding

The `process_content()` helper (lines [2120‑2150](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L2120-L2150)) evaluates filters against the current context (`ctx`):

- **Matching**: Calls `OnlyFilter.match()` or `NoFilter.match()` to test context labels.
- **Pruning**: Removes filters via `is_irrelevant()` or enforces them via `requires_action()`.
- **Conditionals**: Unpacks `Condition` or `NegativeCondition` blocks when predicates match, enabling nested filtering.

Failed matches are recorded in `node.failed_cases` (lines [222‑227](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L222-L227)) to short-circuit impossible combinations in subsequent iterations. Valid configurations are flattened into dictionaries and yielded to the test runner.

## Practical Example: Configuring Complex Test Scenarios

Consider a configuration file combining operating systems with disk formats:

```text
- guest_os:
    - Fedora:
        only (arch=x86_64)
    - CentOS:
        no (arch=arm)

- disk_format:
    - raw
    - qcow2

join guest_os
join disk_format

boot = yes

```

**Resolution process:**

1. **Parsing** creates two top-level variant nodes (`guest_os` and `disk_format`) with their respective `OnlyFilter` and `NoFilter` children.
2. **Join handling** converts `join guest_os` and `join disk_format` into filters, then `multiply_join()` produces four combinations: Fedora+raw, Fedora+qcow2, CentOS+raw, and CentOS+qcow2.
3. **Filtering** applies the `(arch=x86_64)` constraint to Fedora variants and excludes `(arch=arm)` from CentOS variants.
4. **Output** yields four flat dictionaries containing `boot=yes`, the selected OS, disk format, and auto-generated names like `Fedora.raw` via `mk_name()`.

## Programmatic Usage

Access the Cartesian config system directly from Python to inspect or filter configurations:

```python
from virttest.cartesian_config import Parser

# Initialize parser with configuration file

parser = Parser('example.cfg')

# Apply additional runtime filters

parser.only_filter('arch=x86_64')
parser.no_filter('arch=arm')

# Iterate over generated test configurations

for cfg in parser.get_dicts():
    print(f"Test: {cfg['name']}")
    print(f"Params: {cfg}")

```

Each iteration provides a complete parameter dictionary for one concrete test scenario, ready for consumption by the Avocado-VT test harness.

## Summary

- The **Cartesian config system** uses a domain-specific language parsed by `Lexer` and `Parser` classes in [`virttest/cartesian_config.py`](https://github.com/avocado-framework/avocado-vt/blob/main/virttest/cartesian_config.py).
- **Variants** (`- variant:`) create hierarchical parameter blocks, while **filters** (`only`, `no`) prune the configuration space using `OnlyFilter` and `NoFilter` logic.
- **Joins** generate Cartesian products through `multiply_join()`, combining independent variant groups into comprehensive test matrices.
- The **dictionary generation** phase (`get_dicts()`) walks the AST, applies filters, and yields flat parameter dictionaries for each valid test combination.

## Frequently Asked Questions

### What is the difference between `only` and `no` filters in the Cartesian config system?

The `only` filter retains only configurations that match its expression, effectively intersecting the current set with the filter criteria, while the `no` filter excludes any configurations that match, performing a set subtraction. Both are implemented via `OnlyFilter` and `NoFilter` classes (lines [63‑89](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L63-L89)) which provide `match()` and `is_irrelevant()` methods to determine applicability.

### How does the `join` operator create combinatorial test scenarios?

The `join` operator instructs the parser to compute a Cartesian product between the current configuration node and all configurations matching the join target. Internally, `Parser.multiply_join()` (lines [2082‑2110](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L2082-L2110)) recursively merges dictionaries from joined variant groups, allowing you to test every combination of, for example, guest operating systems and disk formats without manually enumerating each pair.

### Can I use the Cartesian config parser outside of Avocado-VT test execution?

Yes, the `Parser` class in [`virttest/cartesian_config.py`](https://github.com/avocado-framework/avocado-vt/blob/main/virttest/cartesian_config.py) is designed for independent use. Import it in Python scripts, instantiate it with a configuration file path, and call `get_dicts()` to iterate over test configurations. You can also programmatically add filters using `only_filter()` and `no_filter()` methods before generation to dynamically adjust the test matrix at runtime.

### How does the system handle invalid or impossible filter combinations?

When a filter fails to match, the configuration node records the failure in `node.failed_cases` (lines [222‑227](https://github.com/avocado-framework/avocado-vt/blob/master/virttest/cartesian_config.py#L222-L227)), allowing the engine to short-circuit and skip that entire subtree in subsequent evaluations. This pruning optimization prevents the generation of dictionaries for impossible parameter combinations, improving performance when working with large variant spaces.