# How to Use PerformanceHelper in Ocaramba: Measure and Report Test Execution Times

> Learn how to use PerformanceHelper in Ocaramba to measure and report test execution times. Time operations with StartMeasure StopMeasure for automated CI reporting.

- Repository: [Accenture/ocaramba](https://github.com/accenture/ocaramba)
- Tags: how-to-guide
- Published: 2026-02-23

---

**PerformanceHelper in the Ocaramba framework lets you time test operations using `StartMeasure()` and `StopMeasure()`, then aggregates results by browser and scenario for automated CI reporting via `PrintPerformanceResultsHelper`.**

The `accenture/ocaramba` repository provides a dedicated utility for capturing and analyzing test performance metrics without cluttering your test code with manual logging. By integrating directly with the `DriverContext`, PerformanceHelper automatically tracks elapsed milliseconds across different browsers and generates statistics suitable for TeamCity and AppVeyor dashboards.

## Core Components of the Performance Measurement System

The implementation relies on three primary types located in the `OcarambaLite` project:

- **`PerformanceHelper`** ([`/OcarambaLite/Helpers/PerformanceHelper.cs`](https://github.com/accenture/ocaramba/blob/main//OcarambaLite/Helpers/PerformanceHelper.cs)): Manages a `Stopwatch` instance, maintains a collection of `SavedTimes` objects, and computes aggregated statistics including average duration and 90th percentile per scenario.
- **`SavedTimes`** ([`/OcarambaLite/Types/SavedTimes.cs`](https://github.com/accenture/ocaramba/blob/main//OcarambaLite/Types/SavedTimes.cs)): A simple DTO storing the scenario name, browser identifier (read from `BaseConfiguration.TestBrowser`), and measured duration in milliseconds.
- **`PrintPerformanceResultsHelper`** ([`/OcarambaLite/Helpers/PrintPerformanceResultsHelper.cs`](https://github.com/accenture/ocaramba/blob/main//OcarambaLite/Helpers/PrintPerformanceResultsHelper.cs)): Formats aggregated data into CI-specific service messages for TeamCity and AppVeyor, typically invoked during test teardown.

Each `DriverContext` instance initializes a single `PerformanceHelper` accessible through the `PerformanceMeasures` property, ensuring consistent measurement scope across test fixtures.

## How to Measure Performance in Your Tests

### Starting and Stopping Measurements

To capture timing data, invoke `StartMeasure()` before the operation and `StopMeasure(string title)` immediately after. The title parameter should uniquely identify the step, often combining the test name with a descriptive suffix.

```csharp
[Test]
public void LoadingMainPage()
{
    // Start the timer
    this.DriverContext.PerformanceMeasures.StartMeasure();
    
    // Execute the operation to benchmark
    this.Driver.Navigate().GoToUrl("https://example.com");
    
    // Stop and record with a descriptive identifier
    this.DriverContext.PerformanceMeasures.StopMeasure(
        TestContext.CurrentContext.Test.Name + "LoadingMainPage");
}

```

This pattern is demonstrated in the NUnit test suite at [`Ocaramba.Tests.NUnit/Tests/PerformanceTestsNUnit.cs`](https://github.com/accenture/ocaramba/blob/main/Ocaramba.Tests.NUnit/Tests/PerformanceTestsNUnit.cs) (lines 39-45), where measurements wrap navigation and page load operations.

### Accessing Aggregated Results Programmatically

For custom reporting or verification, access the `AllGroupedDurationsMilliseconds` property, which returns a collection of `AverageGroupedTimes` objects grouped by scenario and browser:

```csharp
var aggregated = this.DriverContext.PerformanceMeasures.AllGroupedDurationsMilliseconds;

foreach (var item in aggregated)
{
    Console.WriteLine(
        $"{item.StepName} ({item.Browser}) – " +
        $"avg: {item.AverageDuration} ms, " +
        $"p90: {item.Percentile90} ms");
}

```

The aggregation logic, implemented in [`PerformanceHelper.cs`](https://github.com/accenture/ocaramba/blob/main/PerformanceHelper.cs) (lines 68-88), orders measurements by duration, groups them by an anonymous key combining scenario and browser, and calculates both the arithmetic mean and the 90th percentile using `Math.Round()` and index-based percentile extraction.

## Reporting Results to CI Servers

### TeamCity and AppVeyor Integration

The `PrintPerformanceResultsHelper` class provides static methods to output performance data in formats recognized by CI servers. Call these methods in your teardown or fixture cleanup to publish metrics automatically:

```csharp
[TearDown]
public void ReportPerformance()
{
    // TeamCity service messages
    PrintPerformanceResultsHelper.PrintAverageDurationMillisecondsInTeamcity(
        this.DriverContext.PerformanceMeasures);
    PrintPerformanceResultsHelper.PrintPercentiles90DurationMillisecondsinTeamcity(
        this.DriverContext.PerformanceMeasures);
    
    // AppVeyor commands (optional)
    PrintPerformanceResultsHelper.PrintAverageDurationMillisecondsInAppVeyor(
        this.DriverContext.PerformanceMeasures);
    PrintPerformanceResultsHelper.PrintPercentiles90DurationMillisecondsInAppVeyor(
        this.DriverContext.PerformanceMeasures);
}

```

These calls are automatically executed in the common base class [`Ocaramba.Tests.NUnit/ProjectTestBase.cs`](https://github.com/accenture/ocaramba/blob/main/Ocaramba.Tests.NUnit/ProjectTestBase.cs) (lines 91-97), ensuring consistent reporting across all derived test classes without requiring repetitive boilerplate.

## Understanding the Aggregation Logic

When `StopMeasure` completes, the helper creates a `SavedTimes` instance containing the scenario title, current browser from `BaseConfiguration.TestBrowser`, and elapsed milliseconds. These records accumulate in the internal `loadTimeList`.

The `AllGroupedDurationsMilliseconds` property processes this list through the following steps:

1. **Ordering**: Sorts all measurements by duration.
2. **Grouping**: Executes a `GroupBy` operation on an anonymous key combining scenario name and browser.
3. **Calculation**: For each group, computes:
   - `AverageDuration`: `Math.Round(savedTimeses.Average(dur => dur.Duration))`
   - `Percentile90`: The element at index `ceil(count * 0.9) - 1` after sorting
4. **Sorting**: Returns the final collection ordered by `StepName` for consistent reporting.

This approach ensures you receive browser-specific performance baselines, critical for identifying cross-browser performance regressions in Selenium test suites.

## Summary

- **Access PerformanceHelper** through `DriverContext.PerformanceMeasures`, which provides a shared instance for each test context.
- **Capture timings** by wrapping code blocks with `StartMeasure()` and `StopMeasure(title)` to record scenario-specific durations.
- **Retrieve statistics** via `AllGroupedDurationsMilliseconds`, which returns averaged data including 90th percentile calculations per browser.
- **Automate reporting** using `PrintPerformanceResultsHelper` static methods to emit TeamCity service messages or AppVeyor commands during teardown.
- **Locate source code** in [`OcarambaLite/Helpers/PerformanceHelper.cs`](https://github.com/accenture/ocaramba/blob/main/OcarambaLite/Helpers/PerformanceHelper.cs) and [`OcarambaLite/Types/SavedTimes.cs`](https://github.com/accenture/ocaramba/blob/main/OcarambaLite/Types/SavedTimes.cs) for customization or extension.

## Frequently Asked Questions

### How do I access PerformanceHelper in my test class?

Access the helper through the `PerformanceMeasures` property of your `DriverContext` instance. In typical Ocaramba implementations, test classes inherit from a base class that exposes `this.DriverContext`, allowing you to call `this.DriverContext.PerformanceMeasures.StartMeasure()` directly without manual instantiation.

### What metrics does PerformanceHelper calculate?

PerformanceHelper calculates two primary metrics for each scenario-browser combination: **AverageDuration** (arithmetic mean rounded to the nearest millisecond) and **Percentile90** (the value below which 90% of observations fall). These metrics help identify both typical performance and tail latency in your test executions.

### Can I use PerformanceHelper with test frameworks other than NUnit?

Yes. While the examples in `Ocaramba.Tests.NUnit` demonstrate NUnit integration, the core classes in `OcarambaLite` have no NUnit dependencies. You can instantiate and use `PerformanceHelper` directly in MSTest, xUnit, or SpecFlow projects, provided you call the measurement methods and `PrintPerformanceResultsHelper` from the appropriate lifecycle hooks (such as `[TestCleanup]` or `[AfterScenario]`).

### Where are the performance results stored before reporting?

Raw timing data accumulates in an internal `List<SavedTimes>` within the `PerformanceHelper` instance. Each `SavedTimes` object stores the scenario identifier, browser name, and duration. This list persists for the lifetime of the `DriverContext`, allowing you to collect multiple measurements across different test methods before aggregating and reporting them during fixture teardown.