# How to Configure BenchmarkDotNet for Side‑by‑Side Performance Comparisons in the dotnet/skills Repository

> Learn to configure BenchmarkDotNet for side-by-side performance comparisons using the dotnet/skills repository. Generate deterministic results with specific flags.

- Repository: [.NET Platform/skills](https://github.com/dotnet/skills)
- Tags: how-to-guide
- Published: 2026-07-11

---

**Use the microbenchmarking skill in dotnet/skills to generate benchmarks that employ `[Benchmark(Baseline = true)]` or `.AsBaseline()` on jobs, then run via `BenchmarkSwitcher` with `--filter "*"` and `--runtimes` flags to produce deterministic side‑by‑side results.**

The dotnet/skills repository provides a specialized microbenchmarking skill that standardizes how agents generate and execute performance comparisons. This skill, located under `plugins/dotnet-diag/skills/microbenchmarking/`, encapsulates best practices for configuring BenchmarkDotNet to run side‑by‑side comparisons across multiple runtimes or implementations. By following the reference documentation found in the skill's markdown files, developers can ensure reproducible benchmarking workflows that avoid interactive prompts and preserve historical results.

## Architecture of the Microbenchmarking Skill

The microbenchmarking skill is structured as a modular set of guidelines rather than a code library. According to [`plugins/dotnet-diag/skills/microbenchmarking/SKILL.md`](https://github.com/dotnet/skills/blob/main/plugins/dotnet-diag/skills/microbenchmarking/SKILL.md), the skill activates whenever a task requires BenchmarkDotNet execution. It delegates implementation details to four key reference documents:

- [`project-setup-and-running.md`](https://github.com/dotnet/skills/blob/main/project-setup-and-running.md) – Scaffold projects and invoke the runner
- [`comparison-strategies.md`](https://github.com/dotnet/skills/blob/main/comparison-strategies.md) – Configure baselines and multiple jobs  
- [`writing-benchmarks.md`](https://github.com/dotnet/skills/blob/main/writing-benchmarks.md) – Syntax and attribute usage
- [`diagnosers-and-exporters.md`](https://github.com/dotnet/skills/blob/main/diagnosers-and-exporters.md) – Additional metrics and output formats

The skill mandates non‑interactive execution by requiring `--filter "*"` and `--noOverwrite` arguments, ensuring that automated agents can run benchmarks without hanging on console prompts.

## Setting Up Side‑by‑Side Comparisons

To configure BenchmarkDotNet for side‑by‑side performance comparisons, you must establish a baseline and define comparative jobs. The workflow follows three distinct configuration layers as documented in [`comparison-strategies.md`](https://github.com/dotnet/skills/blob/main/comparison-strategies.md).

### Marking the Baseline

Every comparison requires a reference point. You can designate a baseline using either declarative attributes or fluent configuration:

- **Method-level baseline**: Apply `[Benchmark(Baseline = true)]` to the reference implementation
- **Job-level baseline**: Call `.AsBaseline()` on a `Job` instance when configuring runtimes

Only one baseline is permitted per comparison group. The baseline method or job serves as the denominator for the **Ratio** column in the final results table.

### Configuring Multiple Jobs

Side‑by‑side comparisons require at least two jobs targeting different variables—whether distinct runtimes (e.g., .NET 6.0 vs .NET 7.0), JIT compilers, or configuration flags. In a custom `ManualConfig` class, instantiate separate `Job` objects and add them to the configuration:

```csharp
var net6 = Job.Default.WithRuntime(".NET 6.0").AsBaseline();
var net7 = Job.Default.WithRuntime(".NET 7.0");
AddJob(net6);
AddJob(net7);

```

### Running with BenchmarkSwitcher

The skill recommends `BenchmarkSwitcher` over `BenchmarkRunner` because it respects CLI arguments required for automation. Essential flags include:

- `--filter "*"` – Executes all benchmarks without interactive selection
- `--noOverwrite` – Preserves previous runs in timestamped subdirectories under `BenchmarkDotNet.Artifacts/`
- `--runtimes net6.0 net7.0` – Targets multiple runtimes in a single execution

As noted in [`project-setup-and-running.md`](https://github.com/dotnet/skills/blob/main/project-setup-and-running.md), redirecting console output to a log file is recommended for CI/CD environments.

## Complete Implementation Example

The following example demonstrates a full side‑by‑side comparison between two sorting algorithms across two .NET runtimes:

```csharp
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;

public class SortBenchmark
{
    private int[] data;

    [GlobalSetup]
    public void Setup()
    {
        var rnd = new Random(42);
        data = Enumerable.Range(0, 10_000).Select(_ => rnd.Next()).ToArray();
    }

    [Benchmark(Baseline = true)]
    public int[] ArraySort() => data.OrderBy(x => x).ToArray();

    [Benchmark]
    public int[] LinqSort() => data.OrderBy(x => x).ToArray();
}

public class ComparisonConfig : ManualConfig
{
    public ComparisonConfig()
    {
        var net6 = Job.Default
            .WithRuntime(".NET 6.0")
            .AsBaseline();

        var net7 = Job.Default
            .WithRuntime(".NET 7.0");

        AddJob(net6);
        AddJob(net7);
    }
}

public class Program
{
    public static int Main(string[] args) =>
        BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly)
            .Run(args, new ComparisonConfig());
}

```

Execute the benchmark with:

```bash
dotnet build -c Release
dotnet run -c Release --filter "*" --noOverwrite --runtimes net6.0 net7.0 > benchmark.log

```

## Interpreting Side‑by‑Side Results

BenchmarkDotNet generates a Markdown summary containing columns for **Mean**, **Error**, **StdDev**, and **Ratio**. The **Ratio** column displays relative performance compared to the baseline (e.g., `1.23×` slower or `0.85×` faster). Results are stored in `BenchmarkDotNet.Artifacts/<timestamp>/`, with each run isolated when using `--noOverwrite`.

## Summary

- The dotnet/skills microbenchmarking skill under `plugins/dotnet-diag/skills/microbenchmarking/` provides the authoritative workflow for BenchmarkDotNet configuration
- Use `[Benchmark(Baseline = true)]` for method baselines or `.AsBaseline()` on `Job` configurations for runtime baselines
- Always invoke benchmarks via `BenchmarkSwitcher` with `--filter "*"` to prevent interactive hangs
- Store historical results by adding `--noOverwrite` to preserve timestamped artifacts in `BenchmarkDotNet.Artifacts/`
- Target multiple runtimes simultaneously using `--runtimes netX.X netY.Y` flags

## Frequently Asked Questions

### What is the difference between BenchmarkRunner and BenchmarkSwitcher?

`BenchmarkRunner` executes benchmarks immediately with hardcoded configuration, while `BenchmarkSwitcher` parses command-line arguments to filter and configure runs dynamically. The dotnet/skills repository mandates `BenchmarkSwitcher` because it supports the `--filter` and `--noOverwrite` flags required for automated, side‑by‑side comparisons without user intervention.

### How do I compare more than two implementations in a single run?

Add multiple `[Benchmark]` methods to your class and designate one as the baseline using `[Benchmark(Baseline = true)]`. BenchmarkDotNet will execute all methods and calculate ratios relative to the baseline. For runtime comparisons, add multiple `Job` instances to your `ManualConfig`, ensuring only one job calls `.AsBaseline()`.

### Why does the skill require the --noOverwrite flag?

The `--noOverwrite` flag prevents BenchmarkDotNet from deleting previous results, instead creating timestamped subdirectories under `BenchmarkDotNet.Artifacts/`. This preserves historical data for trend analysis and ensures that automated agents can safely run benchmarks multiple times without losing prior measurements.

### Where are the benchmark artifacts stored?

By default, BenchmarkDotNet writes results to `BenchmarkDotNet.Artifacts/` in the project root. When using `--noOverwrite`, each execution creates a new subdirectory named with a timestamp. The skill documentation in [`project-setup-and-running.md`](https://github.com/dotnet/skills/blob/main/project-setup-and-running.md) recommends redirecting console output to a log file in addition to these automatic artifacts.