Monitoring iroh Connection Metrics and Stats: Real-Time Network Observability

Iroh's built-in metrics system exposes lock-free counters for every socket, transport, and connection via the Endpoint::metrics() API, enabling real-time monitoring of traffic, paths, and connection lifecycles without external dependencies.

The n0-computer/iroh repository ships with a comprehensive, zero-cost abstraction for monitoring iroh connection metrics and stats directly from your application code. This system captures fine-grained telemetry from the socket layer up to the connection manager, providing visibility into IPv4/IPv6 traffic splits, relay utilization, hole-punching success rates, and actor loop performance.

Architecture of the iroh Metrics System

Iroh implements a two-tier metrics hierarchy that separates socket-level I/O statistics from endpoint-level connection management counters. All metrics use monotonic Counter types from the iroh_metrics crate, which are incremented via lock-free operations and safely shared across threads using Arc.

Socket-Level Metrics

The Metrics struct defined in iroh/src/socket/metrics.rs holds the foundational counters for raw network activity. This includes bytes sent and received over IPv4, IPv6, and relay paths, as well as path discovery events and hole-punching attempts. Each transport layer receives a shared Arc<SocketMetrics> and updates counters directly in the data path.

Endpoint-Level Aggregation

The EndpointMetrics struct in iroh/src/metrics.rs wraps the socket-level metrics and adds connection lifecycle counters. It tracks num_conns_opened and num_conns_closed (incremented only after successful TLS handshakes), along with aggregated path statistics like paths_direct and paths_relay. This structure is what the public API exposes via Endpoint::metrics().

Transport Layer Integration

Transport implementations in iroh/src/socket/transports/ip.rs increment counters such as transport_ip_paths_added whenever a new direct path is established. Similarly, the relay actor in iroh/src/socket/transports/relay/actor.rs updates relay-specific counters and reports actor loop statistics including actor_tick_main and actor_tick_msg.

Accessing Connection Metrics via the Endpoint API

The Endpoint type provides the metrics() method as the primary entry point for querying statistics. This method returns a reference to the endpoint's EndpointMetrics, which remains valid for the lifetime of the endpoint.

use iroh::Endpoint;
use std::time::Duration;
use tokio::time;

// Build endpoint with metrics enabled by default
let endpoint = Endpoint::builder()
    .bind_default()
    .await
    .expect("failed to start iroh endpoint");

// Spawn a task to periodically log metrics
tokio::spawn(async move {
    loop {
        let metrics = endpoint.metrics();
        
        println!("IPv4 sent: {} bytes", metrics.socket.send_ipv4.get());
        println!("IPv6 sent: {} bytes", metrics.socket.send_ipv6.get());
        println!("Relay sent: {} bytes", metrics.socket.send_relay.get());
        println!("Connections active: {}", 
            metrics.num_conns_opened.get() - metrics.num_conns_closed.get());
        
        time::sleep(Duration::from_secs(10)).await;
    }
});

Key Metrics Categories

Understanding the available counters helps you diagnose network topology and performance bottlenecks.

Traffic Counters

The socket metrics track raw byte counts for send_ipv4, send_ipv6, and send_relay, along with their receive counterparts. These counters are incremented atomically in the transport layer immediately after successful socket operations.

Path Discovery and Hole-Punching

Counters like holepunch_attempts, paths_direct, paths_relay, and paths_custom indicate how the endpoint is establishing connectivity. Increments to paths_direct indicate successful hole-punching, while paths_relay tracks fallback connections through relay servers.

Connection Lifecycle

The endpoint metrics expose num_conns_opened and num_conns_closed, which reflect the actual number of active QUIC connections after TLS handshakes complete. These differ from socket-level counters by tracking logical connections rather than raw packets.

Actor Loop Statistics

The internal socket actor reports loop performance via counters like actor_tick_main (main loop iterations), actor_tick_msg (messages processed), actor_tick_re_stun (STUN re-checks), and actor_tick_portmap_changed (port mapping updates).

Exporting Metrics for External Monitoring

Because all metrics implement serde::Serialize, you can export snapshots to external observability systems without custom adapters.

JSON Serialization

Convert live metrics to JSON for HTTP endpoints or log aggregation:

use serde_json::json;

let snapshot = endpoint.metrics();
let payload = json!({
    "send_ipv4": snapshot.socket.send_ipv4.get(),
    "send_ipv6": snapshot.socket.send_ipv6.get(),
    "send_relay": snapshot.socket.send_relay.get(),
    "connections_opened": snapshot.num_conns_opened.get(),
    "connections_closed": snapshot.num_conns_closed.get(),
    "direct_paths": snapshot.paths_direct.get(),
    "relay_paths": snapshot.paths_relay.get(),
});

println!("{}", serde_json::to_string_pretty(&payload).unwrap());

Integration with Tracing

Iroh emits structured tracing events for metric updates. Enable a subscriber to capture these in your existing logging infrastructure:

use tracing_subscriber::FmtSubscriber;

let subscriber = FmtSubscriber::builder()
    .with_max_level(tracing::Level::INFO)
    .finish();
tracing::subscriber::set_global_default(subscriber)
    .expect("setting default subscriber failed");

Summary

• Lock-free collection: All counters use atomic operations from the iroh_metrics crate, ensuring zero overhead for high-throughput connections.
• Two-tier design: iroh/src/socket/metrics.rs handles socket-level I/O, while iroh/src/metrics.rs aggregates connection lifecycle data.
• Public API: Access live metrics anytime via Endpoint::metrics() defined in iroh/src/endpoint.rs.
• Transport coverage: IP and relay transports in iroh/src/socket/transports/ip.rs and iroh/src/socket/transports/relay/actor.rs update path-specific counters.
• Export ready: Built-in Serialize support enables JSON export without custom glue code.

Frequently Asked Questions

How do I access iroh connection metrics in real-time?

Call endpoint.metrics() on any live Endpoint instance. This returns a reference to EndpointMetrics containing all current counters. The method is non-blocking and returns immediately with the latest values, as the counters are updated atomically by the internal networking tasks.

Are iroh metrics thread-safe?

Yes. All metrics use Arc-shared Counter types from the iroh_metrics crate that implement lock-free atomic operations. You can safely clone and access metrics handles across multiple threads or async tasks without synchronization overhead.

Can I export iroh metrics to Prometheus?

While iroh does not provide a dedicated Prometheus exporter, the EndpointMetrics struct implements serde::Serialize. You can serialize the metrics to JSON and expose them via an HTTP endpoint, then use a converter like json_exporter or manually map the fields to Prometheus format in your application code.

What is the performance impact of enabling metrics?

The metrics system is designed as zero-cost when not observed, though in practice they are always collected. The Counter type uses relaxed atomic operations that typically compile to simple increment instructions on x86_64 and ARM64 architectures. For most applications, the overhead is negligible compared to the cost of syscall and crypto operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}