Complete Guide to iroh Endpoint Performance Metrics

The iroh endpoint exposes granular performance metrics through EndpointMetrics, which aggregates SocketMetrics for QUIC traffic counters and NetReportMetrics for network traversal diagnostics, accessible via the endpoint.metrics() method.

Monitoring peer-to-peer network performance requires deep visibility into transport-layer throughput, NAT traversal success, and connection lifecycle events. The n0-computer/iroh repository provides comprehensive instrumentation through the EndpointMetrics struct defined in iroh/src/metrics.rs, enabling real-time observation of bytes sent over IPv4/IPv6/relay transports, path discovery rates, hole-punching attempts, and internal actor loop activity.

EndpointMetrics Structure

The EndpointMetrics struct serves as the top-level aggregation point for all endpoint telemetry. According to the source code in iroh/src/metrics.rs, it contains two primary sub-structures:

• socket: A SocketMetrics instance tracking QUIC socket layer activity
• net_report: A NetReportMetrics instance counting network report executions and port-mapping events

This design separates transport-level counters from network discovery diagnostics, allowing operators to query specific domains independently.

Socket-Level Metrics

Located in iroh/src/socket/metrics.rs, SocketMetrics provides detailed counters for the QUIC socket layer. These metrics track throughput, path types, connection events, and internal actor loop performance.

Transport Throughput Counters

Monitor raw byte flow across different network paths using the send and receive counters:

• send_ipv4, send_ipv6, send_relay: Bytes transmitted over IPv4, IPv6, and relay transports respectively
• recv_data_ipv4, recv_data_ipv6, recv_data_relay, recv_data_custom: Bytes received on the respective transports

These counters help identify bandwidth bottlenecks or over-reliance on relay servers when direct paths are available.

Datagram Processing

Track QUIC datagram volume and kernel optimization efficiency:

• recv_datagrams: Total number of QUIC datagrams received
• recv_gro_datagrams: Number of Generic Receive Offload (GRO) batching events (counts batches, not individual datagrams)

High recv_gro_datagrams values relative to recv_datagrams indicate efficient kernel-level batching, reducing CPU overhead per packet.

Path Discovery and Types

Understand network topology and NAT traversal success through path counters:

• paths_direct: Direct network paths discovered for peers
• paths_relay: Relayed network paths discovered
• paths_custom: User-defined custom paths

Additionally, relay_home_change tracks how frequently the endpoint's home relay assignment changes, including the initial assignment. Frequent changes may indicate unstable NAT conditions or poor relay placement.

Hole-Punching Activity

Monitor NAT traversal attempts:

• holepunch_attempts: Number of times hole-punching was initiated on a connection (client-side only)

Elevated values suggest connectivity difficulties requiring NAT traversal, potentially indicating restrictive firewall configurations.

Connection Lifecycle

Track connection establishment and teardown:

• num_conns_direct: Direct connections opened (counted once per connection)
• num_conns_opened: Total handshaked connections opened
• num_conns_closed: Connections closed

Connection churn metrics help identify instability; spikes in these counters may reveal network partitions or aggressive peer timeouts.

Transport Path Management

Observe dynamic path creation and removal:

• transport_ip_paths_added / transport_ip_paths_removed: IP transport paths opened and closed
• transport_relay_paths_added / transport_relay_paths_removed: Relay transport paths opened and closed
• transport_custom_paths_added / transport_custom_paths_removed: Custom transport paths opened and closed

These counters reveal rapid topology changes and path migration events as the endpoint adapts to network conditions.

Actor Loop Performance

Gauge internal socket thread workload through actor loop counters:

• actor_tick_main: Main socket actor loop iterations
• actor_tick_msg: Messages processed by the socket actor
• actor_tick_re_stun: Re-STUN timer ticks handled
• actor_tick_portmap_changed: Port-mapping change events processed
• actor_link_change: Local network interface (link) change events
• actor_tick_direct_addr_heartbeat: Direct-address heartbeat ticks (currently unused in the codebase)
• actor_tick_other: Other actor-loop events, such as input watcher closures

High values in actor_tick_main relative to message throughput may indicate the socket thread is becoming a CPU hotspot.

Network Report Metrics

The net_report field contains NetReportMetrics from iroh/src/net_report/metrics.rs, tracking the endpoint's self-diagnostic network reporting subsystem:

• reports: Total net-report executions (including both full and partial reports)
• reports_full: Full net-report executions only
• portmap_attempts: Number of port-mapping attempts performed
• portmap_external_address_updated: Times an external address was successfully obtained via port-mapping

These metrics indicate how frequently the endpoint evaluates its own network connectivity and the success rate of UPnP/NAT-PMP port mapping attempts.

Accessing Metrics Programmatically

Retrieve metrics from a bound endpoint using the metrics() method defined in iroh/src/endpoint.rs. The method returns an Arc<EndpointMetrics> for thread-safe access:

use iroh::Endpoint;
use std::sync::Arc;

async fn monitor_endpoint() -> Result<(), Box<dyn std::error::Error>> {
    // Bind an endpoint using the default preset
    let endpoint = Endpoint::bind(iroh::preset::Default).await?;
    
    // Obtain a reference to the aggregated metrics
    let ep_metrics: Arc<iroh::metrics::EndpointMetrics> = endpoint.metrics();
    
    // Access socket-level counters
    println!("IPv4 bytes sent: {}", ep_metrics.socket.send_ipv4.get());
    println!("IPv6 bytes sent: {}", ep_metrics.socket.send_ipv6.get());
    println!("Relay bytes sent: {}", ep_metrics.socket.send_relay.get());
    println!("Datagrams received: {}", ep_metrics.socket.recv_datagrams.get());
    
    // Access path discovery metrics
    println!("Direct paths discovered: {}", ep_metrics.socket.paths_direct.get());
    println!("Relay paths discovered: {}", ep_metrics.socket.paths_relay.get());
    
    // Access net-report diagnostics
    println!("Net-reports executed: {}", ep_metrics.net_report.reports.get());
    println!("Full reports: {}", ep_metrics.net_report.reports_full.get());
    println!("Port-map attempts: {}", ep_metrics.net_report.portmap_attempts.get());
    
    Ok(())
}

All counter values are accessed via the .get() method on their respective metric fields, returning the current atomic count.

Interpreting Key Performance Indicators

When monitoring iroh endpoint performance metrics in production, focus on these critical relationships:

Transport efficiency: Compare send_ipv4/send_ipv6 against send_relay. A high ratio of direct transport to relay traffic indicates successful NAT traversal and optimal path selection.

Connection stability: Correlate num_conns_opened with num_conns_closed. Rapid oscillation between these values suggests network instability or aggressive peer churn.

Path distribution: Calculate the ratio of paths_direct to paths_relay. A declining direct path percentage over time may signal deteriorating network conditions or increased firewall restrictions.

Actor loop saturation: Monitor actor_tick_msg versus actor_tick_main. If the main loop ticks significantly exceed message processing ticks, the endpoint may be handling excessive timer events or network interface changes, potentially requiring tuning.

Hole-punching frequency: Elevated holepunch_attempts combined with low paths_direct counts indicate persistent NAT traversal difficulties, possibly requiring relay infrastructure adjustments or port-mapping configuration.

Summary

• EndpointMetrics aggregates all performance counters for an iroh endpoint, accessible via endpoint.metrics() returning Arc<EndpointMetrics>.
• Socket-level metrics in iroh/src/socket/metrics.rs track transport throughput (send_*, recv_*), path types (paths_*), connection lifecycle (num_conns_*), and actor loop activity (actor_tick_*).
• Net-report metrics in iroh/src/net_report/metrics.rs monitor network discovery executions (reports, reports_full) and port-mapping success (portmap_attempts, portmap_external_address_updated).
• Operational insights derive from comparing direct vs. relay traffic ratios, monitoring connection churn, and tracking hole-punching attempt frequencies to diagnose NAT traversal health.

Frequently Asked Questions

How do I access performance metrics from an iroh Endpoint?

Call the metrics() method on your Endpoint instance. This returns an Arc<EndpointMetrics> containing the socket and net_report sub-structures. All counters are atomically updated and accessible via the .get() method on each metric field, as implemented in iroh/src/endpoint.rs.

What is the difference between send_ipv4 and transport_ip_paths_added?

send_ipv4 counts total bytes transmitted over IPv4 transports (cumulative bandwidth), while transport_ip_paths_added counts discrete IP transport path creation events (lifecycle operations). Use the former to monitor throughput and the latter to diagnose path churn or connection instability.

Why is my holepunch_attempts counter high but paths_direct remains low?

This indicates that NAT traversal attempts are failing to establish direct connections. The endpoint initiates hole-punching when direct connectivity seems possible, but if paths_direct does not increment, the punches are unsuccessful. This typically occurs with symmetric NAT configurations or restrictive firewalls, forcing continued reliance on relayed paths despite traversal attempts.

What does recv_gro_datagrams indicate about system performance?

recv_gro_datagrams tracks how many times the socket received batched datagrams via Generic Receive Offload (GRO). A high value relative to recv_datagrams indicates the kernel is successfully batching multiple QUIC packets into single receive operations, significantly reducing per-packet CPU overhead and improving throughput efficiency. Low values may suggest the network interface or operating system lacks GRO support for QUIC.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}