Benchmark Performance Metrics for Keygen and Signing Operations in mpcium

The mpcium CLI benchmark tool captures eleven quantitative metrics—including latency percentiles, throughput rates, and error ratios—by timing each individual operation from request dispatch to result reception.

The mpcium repository provides a comprehensive benchmarking suite within its CLI tool to measure distributed MPC (Multi-Party Computation) performance. The benchmark subsystem, implemented in cmd/mpcium-cli/benchmark.go, automatically calculates benchmark performance metrics for keygen and signing operations by tracking wall-clock timings and success counters across concurrent workloads against a running MPC-IUM service.

Core Benchmark Metrics Structure

The benchmarking system populates a BenchmarkResult struct that aggregates raw timing data into actionable performance indicators. According to the mpcium source code, each benchmark run computes metrics derived from an internal []OperationResult slice, where every entry records StartTime, EndTime, Completed status, and Success boolean flags.

Latency Measurements

Three primary latency metrics are calculated in the calculateBenchmarkResult function (lines 65-79):

• TotalTime: Captured via time.Since(startTime), measuring the wall-clock duration from the first request until the benchmark loop terminates.
• AverageTime: Computed by summing all successful operation durations and dividing by SuccessfulOps.
• MedianTime: Derived by sorting the OperationTimes slice and selecting the middle value.

Throughput and Reliability Metrics

The benchmark tracks operational capacity through:

• OperationsPerMin: Calculated as float64(SuccessfulOps) / totalTime.Minutes() (line 84), expressing throughput as operations completed per minute.
• ErrorRate: Determined by (FailedOps / TotalOperations) * 100 (line 60), providing the percentage of failed or incomplete operations.
• SuccessfulOps and FailedOps: Success counters incremented in the result handlers (lines 44-48) with failures computed as TotalOperations - SuccessfulOps.

Batch Processing Indicators

For signing workloads that utilize the --batch-size flag, the system records:

• BatchSize: The user-supplied batch parameter passed from runSignBenchmark (line 56).
• TotalBatches: The count of actually dispatched batches, calculated as len(batchTimes) (line 98).
• BatchTimes: A slice recording the duration of each individual batch transmission (lines 97-98).

How Metrics Are Calculated

The aggregation logic resides in calculateBenchmarkResult within cmd/mpcium-cli/benchmark.go. This function processes the results slice populated by event listeners (OnWalletCreationResult, OnSignResult, OnResharingResult) that trigger when the MPC service publishes completion events.

Each OperationResult struct (defined at lines 42-50) contains:

type OperationResult struct {
    ID          string
    StartTime   time.Time
    EndTime     time.Time
    Completed   bool
    Success     bool
    ErrorReason string
    ErrorCode   string
}

The function differentiates between TotalOperations (the requested count, line 59) and SuccessfulOps (those receiving a ResultTypeSuccess event, lines 44-48). Partial or missing results count toward FailedOps, ensuring the error rate reflects actual system reliability including timeouts.

Running Benchmark Tests

Execute benchmarks against a running MPC-IUM service using the CLI commands that support both ECDSA and EdDSA cryptographic operations.

Key generation benchmark:

mpcium benchmark keygen 10 \
    --config ./config.yaml \
    --key-path ./event_initiator.key \
    --prompt-password \
    --timeout 5 \
    --output ./benchmark/keygen.txt

ECDSA signing benchmark with batching:

mpcium benchmark sign-ecdsa 200 wallet-123 \
    --batch-size 20 \
    --timeout 30 \
    --output ./benchmark/ecdsa.txt

Both commands generate unique request IDs via generateUniqueID, register asynchronous result listeners, and launch the specified number of operations. The tool waits on a sync.WaitGroup with a global timeout before invoking generateBenchmarkReport to print or save results via the --output flag.

Interpreting Benchmark Results

The generateBenchmarkReport function produces a human-readable summary including percentiles for the first 100 requests. A typical keygen report displays:

===============================
KEYGEN BENCHMARK RESULTS SUMMARY
===============================
Total benchmark time: 2m13.421s
Successful completions: 98
Failed operations: 2
Average operations per minute: 44.30
Average operation time: 1.322s
Median operation time: 1.200s

Signing benchmarks follow an identical format but include the BatchSize column, reflecting the --batch-size parameter's impact on throughput. The fastest and slowest operation times within the first 100 requests provide visibility into latency variance across the distributed MPC nodes.

Summary

• The mpcium CLI implements benchmarking in cmd/mpcium-cli/benchmark.go, providing quantitative analysis of MPC operation performance.
• Eleven distinct metrics are calculated, including AverageTime, MedianTime, OperationsPerMin, and ErrorRate.
• Metrics derive from the OperationResult struct, which tracks precise StartTime and EndTime timestamps for every request.
• The calculateBenchmarkResult function (lines 59-84) aggregates raw timings into statistical summaries while generateBenchmarkReport formats output for analysis.
• Batch processing metrics (TotalBatches, BatchTimes) specifically optimize signing workload measurement when using the --batch-size flag.

Frequently Asked Questions

How does mpcium calculate the median operation time?

The median is computed in calculateBenchmarkResult by first collecting all successful operation durations into the OperationTimes slice, sorting the slice, and then selecting the middle value. This calculation occurs at lines 73-79 of cmd/mpcium-cli/benchmark.go, providing a latency metric less sensitive to outlier values than the average.

What is the difference between TotalOperations and SuccessfulOps?

TotalOperations represents the number of requests dispatched (the length of the results slice, line 59), while SuccessfulOps counts only those operations that received a ResultTypeSuccess event from the MPC service. The difference between these values constitutes FailedOps, which includes both explicitly failed operations and those that never completed due to timeouts.

Can I benchmark EdDSA signing operations separately from ECDSA?

Yes. The mpcium CLI provides distinct commands for each algorithm. Use mpcium benchmark sign-eddsa to run EdDSA benchmarks and mpcium benchmark sign-ecdsa for ECDSA operations. Both commands accept the same batch size, timeout, and output parameters, with results aggregated identically through the OnSignResult listener.

How does batch size affect the benchmark metrics?

The --batch-size flag groups multiple signing requests into single transmissions. The benchmark tracks BatchSize (the requested grouping), TotalBatches (actual transmissions sent), and BatchTimes (duration per batch). This allows calculation of per-operation overhead versus network transmission efficiency, with larger batches typically improving OperationsPerMin but potentially increasing individual latency variance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}