iroh Metrics and Statistics for Monitoring Endpoint and Connection Health

iroh exposes a comprehensive telemetry system via endpoint.metrics() that tracks data transfer volumes, connection lifecycles, path types, hole-punching attempts, and actor loop activity through atomic counters defined in iroh/src/socket/metrics.rs.

The iroh networking library provides built-in observability for production deployments through a structured metrics collection system. All telemetry data is aggregated in the Metrics struct located in iroh/src/socket/metrics.rs and accessible through the public EndpointMetrics API wrapper in iroh/src/socket.rs. These counters enable real-time monitoring of bandwidth utilization, NAT traversal success rates, and connection stability across direct and relayed paths.

Data Transfer and Transport Metrics

The telemetry system captures granular statistics about bytes sent and received across different transport types. These data-transfer counters help identify bandwidth bottlenecks and relay dependency patterns.

IP Version and Relay Bandwidth

• send_ipv4 and send_ipv6: Track the number of bytes sent over IPv4 and IPv6 transports respectively. These counters reveal protocol-specific throughput and help diagnose network-path preferences.
• send_relay: Measures bytes sent over the relay transport. Spikes in this counter indicate heavy reliance on relay nodes rather than direct connections.
• recv_data_ipv4 and recv_data_ipv6: Mirror the send counters for inbound traffic, showing received bytes per IP version.
• recv_data_relay: Tracks bytes received from the relay transport. Elevated values may signal NAT traversal failures forcing traffic through relay nodes.
• recv_data_custom: Counts bytes received over any user-supplied custom transport, enabling monitoring of bespoke networking layers.

Datagram-Level Statistics

• recv_datagrams: Records the total number of datagrams received (QUIC packets), providing a packet-level view of traffic volume independent of byte counts.
• recv_gro_datagrams: Tracks GRO (generic-receive-offload) batches received. This metric shows how often the kernel coalesces multiple packets, which can significantly affect latency characteristics.

Connection and Path Health

Connection stability and path diversity are critical indicators of endpoint health. The metrics expose both instantaneous state and lifecycle events.

Connection Lifecycle Counters

• num_conns_opened: Incremented for every handshaked connection opened. This core health metric indicates the endpoint's ability to establish new peer relationships.
• num_conns_closed: Tracks total handshaked connections closed. When analyzed alongside num_conns_opened, this reveals the connection churn rate.
• num_conns_direct: Counts connections that were established directly (recorded once per connection). High values relative to total connections indicate successful NAT traversal and lower-latency paths.

Path Distribution Metrics

• paths_direct: The current number of direct network paths to peers. A higher value indicates robust direct connectivity without relay intermediaries.
• paths_relay: The number of relayed network paths to peers. This reveals how many connections currently depend on relay infrastructure.
• paths_custom: The number of user-defined paths to peers, useful when operating with custom transport implementations.

Transport Path Lifecycle

The system tracks the creation and destruction of specific transport paths:

• transport_ip_paths_added and transport_ip_paths_removed: Monitor the lifecycle of direct IP transport paths.
• transport_relay_paths_added and transport_relay_paths_removed: Track relay transport path stability and churn.
• transport_custom_paths_added and transport_custom_paths_removed: Monitor usage and teardown of user-supplied transports.

NAT Traversal and Relay Stability

Specialized counters help diagnose NAT traversal behavior and relay infrastructure health.

Hole-Punching Activity

• holepunch_attempts: Counts the times hole-punching was started on a connection (client-side only). High values imply frequent NAT traversal attempts; comparing this against successful direct connections helps infer hole-punching failure rates.

Relay Health Indicators

• relay_home_change: Tracks how many times the home relay changed to a different relay, including the initial assignment. Frequent changes may indicate instability in the chosen relay node or network conditions forcing relay migration.

Actor Loop and Socket Health

The socket actor's event loop publishes internal telemetry that helps diagnose control-plane load and network interface changes.

• actor_tick_main: Iterations of the main socket actor loop. This metric provides a baseline for event-loop activity levels.
• actor_tick_msg: Messages processed by the socket actor. High values may indicate heavy traffic load or excessive control-message overhead.
• actor_tick_re_stun: Periodic re-STUN timer ticks handled. This shows how frequently the system refreshes NAT mappings.
• actor_tick_portmap_changed: Port-mapping change events handled. This is useful when operating in environments with dynamic NAT port-mapping.
• actor_link_change: Local network interface (link) change events. This detects network adapter up/down events that might disrupt connections.
• actor_tick_other: Miscellaneous input-watcher or receiver close events not captured by specific counters.
• update_direct_addrs and actor_tick_direct_addr_heartbeat: Currently unused placeholders reserved for future address-discovery diagnostics and liveness checks.

Accessing Metrics Programmatically

All counters are iroh_metrics::Counter objects exposed through the MetricsGroupSet aggregated by the endpoint. You can read current values using the get() method on any counter.

The following example demonstrates how to obtain a health snapshot from an iroh::Endpoint:

use iroh::Endpoint;

async fn log_endpoint_health() -> anyhow::Result<()> {
    let endpoint = Endpoint::builder().bind().await?;
    let metrics = endpoint.metrics();
    
    println!("--- iroh socket health snapshot ---");
    println!("bytes sent (IPv4):   {}", metrics.socket.send_ipv4.get());
    println!("bytes sent (IPv6):   {}", metrics.socket.send_ipv6.get());
    println!("bytes sent (relay):  {}", metrics.socket.send_relay.get());
    
    println!("bytes recv (IPv4):   {}", metrics.socket.recv_data_ipv4.get());
    println!("bytes recv (IPv6):   {}", metrics.socket.recv_data_ipv6.get());
    println!("bytes recv (relay):  {}", metrics.socket.recv_data_relay.get());
    
    println!("connections opened: {}", metrics.socket.num_conns_opened.get());
    println!("connections closed: {}", metrics.socket.num_conns_closed.get());
    
    println!("holepunch attempts: {}", metrics.socket.holepunch_attempts.get());
    println!("direct paths:       {}", metrics.socket.paths_direct.get());
    println!("relay paths:        {}", metrics.socket.paths_relay.get());
    
    println!("actor ticks (main): {}", metrics.socket.actor_tick_main.get());
    
    Ok(())
}

For continuous monitoring, reference iroh/examples/monitor-connections.rs, which implements periodic logging of these counters and can be adapted to feed Prometheus exporters or structured logging systems.

Key Source Files

File	Purpose
iroh/src/socket/metrics.rs	Defines the Metrics struct containing all counters described above.
iroh/src/socket.rs	Implements the EndpointMetrics wrapper that exposes socket metrics through the public API.
iroh/examples/monitor-connections.rs	Example program demonstrating periodic metrics logging, serving as a template for custom monitoring solutions.

Summary

• iroh provides atomic counters for every critical aspect of endpoint health, including bandwidth utilization, connection churn, path diversity, and NAT traversal attempts.
• The Metrics struct in iroh/src/socket/metrics.rs organizes counters into logical groups covering data transfer, relay activity, hole-punching, and actor loop performance.
• Access metrics at runtime via endpoint.metrics(), which returns a MetricsGroupSet containing the socket metrics group.
• All counters implement iroh_metrics::Counter and expose current values through the get() method.
• Unused placeholders like update_direct_addrs indicate future expansion points for address-discovery diagnostics.

Frequently Asked Questions

How do I access iroh endpoint metrics in my application?

Call the metrics() method on your Endpoint instance after binding. This returns a MetricsGroupSet containing the socket field, which holds all Counter instances defined in iroh/src/socket/metrics.rs. Each counter provides a get() method returning the current u64 value.

What is the difference between paths_direct and num_conns_direct counters?

paths_direct tracks the current number of active direct network paths to peers, which can exceed the number of connections since a single connection may maintain multiple paths. num_conns_direct increments once per connection that achieved direct establishment, serving as a cumulative lifetime counter of successful direct handshakes.

How can I detect relay dependency issues using iroh metrics?

Monitor the ratio of send_relay and recv_data_relay to their IPv4/IPv6 counterparts. If relay bytes constitute the majority of traffic while holepunch_attempts is high and paths_direct remains low, your endpoint is failing NAT traversal and relying excessively on relay infrastructure. Frequent relay_home_change increments may also indicate relay instability.

Why are some actor tick counters marked as unused?

Counters like update_direct_addrs and actor_tick_direct_addr_heartbeat are reserved placeholders for future direct-address liveness checking and discovery diagnostics. They currently remain at zero but are defined in the metrics struct to maintain API stability when these features are implemented in subsequent releases.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}