How to Monitor Iroh Connection Statistics: Real-Time Telemetry in n0-computer/iroh

Iroh provides a built-in, lock-free metrics collection system accessible via Endpoint::metrics() that exposes monotonic counters for socket traffic, transport paths, and connection lifecycle events without requiring external observability tools.

The n0-computer/iroh repository ships with a comprehensive telemetry stack designed for production P2P networking. The iroh::metrics module exposes fine-grained statistics that track every byte transmitted, path discovered, and handshake completed, all accessible through a zero-cost public API on the Endpoint struct.

Architecture of the Iroh Metrics System

The monitoring architecture separates concerns between socket-level telemetry and endpoint-wide aggregation, using atomic counters from the external iroh_metrics crate.

Socket-Level Metrics (iroh/src/socket/metrics.rs)

The core Metrics struct defined in iroh/src/socket/metrics.rs implements the MetricsGroup trait and contains atomic counters for raw socket operations. These include send_ipv4 and send_ipv6 for direct traffic, send_relay for relayed packets, and path discovery counters like paths_direct, paths_relay, and paths_custom. The struct utilizes the Counter type from iroh_metrics, which provides lock-free, monotonic increments safe for concurrent access across threads via Arc<SocketMetrics>.

Endpoint Metrics Aggregation (iroh/src/metrics.rs)

The EndpointMetrics struct in iroh/src/metrics.rs wraps the socket-level Metrics and adds connection lifecycle counters. Specifically, num_conns_opened and num_conns_closed update only after successful TLS handshakes, providing accurate connection turnover statistics. This aggregation point serves as the public interface returned by Endpoint::metrics().

Transport Layer Integration

Each transport implementation receives a shared Arc<SocketMetrics> reference during initialization. In iroh/src/socket/transports/ip.rs, the IP transport calls self.metrics.send_ipv4.inc_by(bytes) or self.metrics.send_ipv6.inc_by(bytes) directly in the send path. Similarly, the relay transport in iroh/src/socket/transports/relay/actor.rs increments counters like transport_relay_paths_removed when paths are torn down.

Socket Actor Loop Statistics

The socket actor driving the main event loop tracks operational health through counters like actor_tick_main, actor_tick_msg, actor_tick_re_stun, and actor_tick_portmap_changed. These metrics, defined in the relay actor implementation, measure loop iterations, message processing volume, and timer events for STUN rebinding and port mapping changes. The actor also increments holepunch_attempts during NAT traversal procedures.

Querying Connection Statistics at Runtime

To monitor Iroh connection statistics, call the metrics() method on any live Endpoint handle. This returns a reference to the EndpointMetrics struct containing live, atomically updated counters.

use iroh::Endpoint;
use std::time::Duration;
use tokio::time;

// Build a normal endpoint (metrics are enabled by default)
let endpoint = Endpoint::builder()
    .bind_default()
    .await
    .expect("failed to start iroh endpoint");

// Periodically dump the current counters
tokio::spawn(async move {
    loop {
        // Grab a snapshot of the metrics struct
        let metrics = endpoint.metrics();

        // Log traffic counters
        println!("📡 Sent IPv4: {} bytes", metrics.socket.send_ipv4.get());
        println!("📡 Sent IPv6: {} bytes", metrics.socket.send_ipv6.get());
        println!("🔁 Sent via relay: {} bytes", metrics.socket.send_relay.get());

        // Connection lifecycle counters
        println!("🔗 Connections opened: {}", metrics.num_conns_opened.get());
        println!("🔗 Connections closed: {}", metrics.num_conns_closed.get());

        // Path statistics
        println!("🚀 Direct paths: {}", metrics.paths_direct.get());
        println!("🕸️ Relay paths: {}", metrics.paths_relay.get());

        time::sleep(Duration::from_secs(10)).await;
    }
});

Exporting Metrics to External Systems

Because the metrics structs implement serde::Serialize, you can serialize snapshots for ingestion into Prometheus, Grafana, or custom log aggregators.

use serde_json::json;

let snapshot = endpoint.metrics();
let json = json!({
    "send_ipv4": snapshot.socket.send_ipv4.get(),
    "send_ipv6": snapshot.socket.send_ipv6.get(),
    "send_relay": snapshot.socket.send_relay.get(),
    "connections_opened": snapshot.num_conns_opened.get(),
    "connections_closed": snapshot.num_conns_closed.get(),
    "direct_paths": snapshot.paths_direct.get(),
    "relay_paths": snapshot.paths_relay.get(),
});
println!("{}", serde_json::to_string_pretty(&json).unwrap());

Integrating with Tracing

The Iroh crate emits tracing events for internal metrics that you can capture by configuring a subscriber. This enables correlating metrics with structured logs.

use tracing_subscriber::FmtSubscriber;

let subscriber = FmtSubscriber::builder()
    .with_max_level(tracing::Level::INFO)
    .finish();
tracing::subscriber::set_global_default(subscriber).expect("setting default subscriber failed");

Key Source Files for Metric Implementation

• iroh/src/socket/metrics.rs: Defines the Metrics struct with socket-level counters for traffic and path discovery.
• iroh/src/metrics.rs: Contains EndpointMetrics, the aggregation layer combining socket metrics with connection counters.
• iroh/src/endpoint.rs: Implements Endpoint::metrics(), the public API entry point for accessing statistics.
• iroh/src/socket/transports/ip.rs: IP transport implementation that increments byte counters on every send operation.
• iroh/src/socket/transports/relay/actor.rs: Relay actor that tracks hole-punching attempts, actor loop ticks, and relay-specific path metrics.

Summary

• Lock-free collection: Iroh uses atomic Counter types from the iroh_metrics crate to avoid contention in hot paths.
• Hierarchical structure: Socket-level Metrics aggregate into EndpointMetrics, accessible via Endpoint::metrics().
• Comprehensive coverage: Counters track IPv4/IPv6 bytes, relay traffic, connection opens/closes, path discovery, and actor loop events.
• Serialization ready: All metrics implement serde::Serialize for JSON export and remote diagnostics.
• Zero configuration: Metrics collection is enabled by default when building an Endpoint and requires no additional setup.

Frequently Asked Questions

How do I enable metrics collection in Iroh?

Metrics collection is active by default when you create an Endpoint using Endpoint::builder(). There is no configuration flag required; the EndpointMetrics and underlying SocketMetrics initialize automatically during the bind process in iroh/src/endpoint.rs.

What is the performance overhead of the metrics system?

The overhead is negligible for application code. The system uses lock-free atomic counters from the iroh_metrics crate that are incremented with single atomic operations. The counters are shared via Arc across threads, and because they are monotonic, no synchronization locks are required for updates.

Can I monitor per-connection statistics separately?

The current implementation aggregates connection statistics at the endpoint level via num_conns_opened and num_conns_closed. For per-connection telemetry, you would need to track individual connection handles in your application logic, as the built-in metrics focus on endpoint-wide and socket-level telemetry rather than individual connection state machines.

How do I reset the counters?

The Counter type is monotonic and does not support reset operations. To track deltas over time, capture periodic snapshots using endpoint.metrics() and calculate differences in your application code. This pattern aligns with standard Prometheus-style monitoring where counters are expected to increase until rollover or restart.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}