# How to Perform Benchmark Testing of Models on Different Hardware in the Gallery App

> Benchmark test AI models on various hardware using the Gallery app. Measure init latency, prefill throughput, and decode speed. Compare results efficiently.

- Repository: [google-ai-edge/gallery](https://github.com/google-ai-edge/gallery)
- Tags: how-to-guide
- Published: 2026-04-06

---

**The Gallery app performs on-device benchmark testing using a Litert-LM integration that measures init latency, prefill throughput, and decode tokens per second, storing results in a DataStore for cross-hardware comparison.**

The google-ai-edge/gallery repository provides a complete Android implementation for benchmark testing of models on different hardware. This open-source app leverages the Litert-LM library to execute performance benchmarks directly on-device, enabling developers to compare LLM inference speeds across CPU, GPU, and NNAPI configurations without external dependencies.

## Architecture of the On-Device Benchmarking System

The benchmark testing implementation follows a **Model-View-ViewModel (MVVM)** architecture that isolates performance measurement logic from UI presentation.

### UI Layer Components

The user interface resides in three primary Kotlin files under `Android/src/app/src/main/java/com/google/ai/edge/gallery/ui/benchmark/`:

- **[`BenchmarkScreen.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkScreen.kt)**: Hosts the model selector and results list view.
- **[`BenchmarkConfigDialog.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkConfigDialog.kt)**: Presents the configuration dialog for warm-up and benchmark iteration counts (lines 79-87).
- **[`BenchmarkResultsViewer.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkResultsViewer.kt)**: Renders saved benchmark runs and handles baseline comparison logic (lines 156-162, 441-500).

### Data and Execution Layer

Behind the scenes, **[`BenchmarkViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkViewModel.kt)** orchestrates the benchmark execution. This ViewModel creates temporary cache directories under the app's cache folder, invokes the Litert-LM `benchmark()` function, and persists aggregated results to a `DataStore<BenchmarkResults>` defined in [`di/AppModule.kt`](https://github.com/google-ai-edge/gallery/blob/main/di/AppModule.kt).

## Executing a Benchmark Test

The workflow for benchmark testing begins when a user taps **"Run benchmark"** from the chat panel.

First, `BenchmarkConfigDialog` collects configuration parameters:

```kotlin
val warmUpIterations = remember { mutableStateOf(2) }
val benchmarkIterations = remember { mutableStateOf(5) }

Button(
    onClick = {
        onBenchmarkClicked(
            message = selectedMessage,
            warmUpIterations = warmUpIterations.value,
            benchmarkIterations = benchmarkIterations.value
        )
    }
) { Text(stringResource(R.string.run_benchmark)) }

```

The `BenchmarkViewModel.runBenchmark()` function then handles the actual execution (lines 124-251 in [`BenchmarkViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkViewModel.kt)):

```kotlin
fun runBenchmark(
    model: Model,
    warmUpIterations: Int,
    benchmarkIterations: Int,
    onComplete: (BenchmarkResult) -> Unit
) {
    val timestamp = System.currentTimeMillis()
    val cacheDir = File(appContext.cacheDir, "benchmark_$timestamp")
    cacheDir.mkdirs()
    Log.d(TAG, "Using benchmark cache dir: ${cacheDir.path}")

    // Execute Litert-LM benchmark for each iteration
    repeat(benchmarkIterations) {
        val info = benchmark(
            modelPath = model.path,
            cacheDir = cacheDir.path,
            warmUpIterations = warmUpIterations,
        )
        // Collect per-iteration metrics
        prefillSpeeds.add(info.lastPrefillTokensPerSecond)
        decodeSpeeds.add(info.lastDecodeTokensPerSecond)
        timesToFirstToken.add(info.timeToFirstTokenInSecond)
    }

    // Compute averages and persist to DataStore
    val result = BenchmarkResult.newBuilder()
        .setAvgPrefillTokensPerSecond(prefillSpeeds.average())
        .setAvgDecodeTokensPerSecond(decodeSpeeds.average())
        .setAvgTimeToFirstToken(timesToFirstToken.average())
        .build()
    benchmarkResultsStore.updateData { it + result }
    cacheDir.deleteRecursively()
    onComplete(result)
}

```

## Key Performance Metrics for Hardware Evaluation

The Litert-LM library (`com.google.ai.edge.litertlm.benchmark`) extracts four critical metrics that vary significantly across different hardware:

1. **`initTimeInSecond`**: Model initialization latency.
2. **`lastPrefillTokensPerSecond`**: Prompt processing throughput.
3. **`lastDecodeTokensPerSecond`**: Token generation speed during inference.
4. **`timeToFirstTokenInSecond`**: End-to-end latency before the first response token.

Because these benchmarks execute on-device using the hardware's specific CPU, GPU, or NNAPI delegates, results automatically reflect the exact silicon capabilities—whether running on Snapdragon 8 Gen 2, Tensor G3, or other chipsets.

## Comparing Results Across Different Devices

After completing benchmark testing on multiple devices, the **Benchmark Results** screen enables direct comparison through the baseline selection feature.

In [`BenchmarkResultsViewer.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkResultsViewer.kt), users tap a **Baseline** chip on any result (e.g., a Pixel 7 run) to establish a reference point. The UI then calculates percentage differences for all other runs, displaying metrics like "+23% faster decode on Galaxy S23" relative to the selected baseline.

For offline analysis, the **Export CSV** button downloads a spreadsheet containing all numeric metrics from the `benchmark_results.pb` DataStore, allowing detailed comparison of model performance across different hardware configurations in external tools.

## Summary

- The google-ai-edge/gallery app implements benchmark testing through a dedicated MVVM stack in `Android/src/app/src/main/java/com/google/ai/edge/gallery/ui/benchmark/`.
- **[`BenchmarkViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkViewModel.kt)** orchestrates Litert-LM execution, creating temporary cache directories and aggregating metrics into `BenchmarkResult` protobufs.
- Four core metrics—**init latency**, **prefill throughput**, **decode throughput**, and **time-to-first-token**—capture hardware-specific performance characteristics.
- Results persist in a `DataStore` (`benchmark_results.pb`) and support baseline comparison through [`BenchmarkResultsViewer.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkResultsViewer.kt).
- All benchmarks run on-device, ensuring measurements reflect actual hardware capabilities rather than server-side approximations.

## Frequently Asked Questions

### What hardware delegates does the Gallery app benchmark support?

The Litert-LM library automatically utilizes available hardware delegates including CPU, GPU, and NNAPI. The benchmark results reflect whichever delegate the model utilizes on the specific device, allowing you to compare NNAPI performance on Pixel devices against GPU acceleration on Snapdragon platforms.

### How many iterations should I configure for accurate benchmark testing?

The `BenchmarkConfigDialog` defaults to 2 warm-up iterations and 5 benchmark iterations. For stable results across different hardware, use at least 3-5 warm-up passes to stabilize thermals and caches, followed by 5-10 measurement iterations. The ViewModel computes averages across all iterations to minimize variance.

### Where are benchmark results stored on the device?

Results serialize to `benchmark_results.pb` via a DataStore provider defined in [`di/AppModule.kt`](https://github.com/google-ai-edge/gallery/blob/main/di/AppModule.kt). This protobuf file persists in the app's private storage and survives app restarts, enabling longitudinal comparison of model performance as you test different hardware configurations.

### Can I export benchmark data to compare across phones?

Yes. [`BenchmarkResultsViewer.kt`](https://github.com/google-ai-edge/gallery/blob/main/BenchmarkResultsViewer.kt) implements an **Export CSV** feature that writes all metrics—including `avgPrefillTokensPerSecond`, `avgDecodeTokensPerSecond`, and `avgTimeToFirstToken`—to a spreadsheet. This facilitates offline analysis when evaluating which hardware best fits specific LLM performance requirements.