How to Debug iroh Network Connections: A Practical Guide

To debug iroh network connections, enable tracing logs with RUST_LOG=iroh=debug, monitor the Socket address watchers for NetReport updates, and inspect Prometheus metrics for packet flow and path selection.

iroh is a peer-to-peer networking library from n0-computer/iroh that builds its transport layer on QUIC and supports direct UDP, relay, and custom paths. When connections fail or perform poorly, the issue typically lies in the interaction between the Socket, Transports, and NetReport components. This guide shows you how to diagnose connectivity problems using the actual source code implementation.

Understanding iroh's Network Architecture

The networking stack centers on the Socket struct in iroh/src/socket.rs, which manages all transport bindings and path selection. Understanding these core components helps you identify where data flow stops.

• Socket – The central connectivity layer that manages transports, address discovery, and path selection. It handles incoming/outgoing datagrams and drives the RemoteStateActor for each peer.
• Transports – Defined in iroh/src/socket/transports.rs, this abstraction creates concrete UDP, relay (WebSocket), and custom sockets while watching for network interface changes.
• RemoteMap – Located in iroh/src/socket/remote_map.rs, this maps synthetic "mapped" addresses to real transport addresses, enabling multiplexing across multiple paths.
• NetReport – Implemented in iroh/src/net_report.rs, this component probes the network to discover NAT mappings and reachable IPs via QUIC Address Discovery (QAD).
• Metrics – Found in iroh/src/socket/metrics.rs, this provides Prometheus-style counters for socket operations, net-reports, and path state changes.

Common Failure Points and Diagnostic Targets

Symptom	Likely Cause	Source Location
No packets received	Transport not bound or network monitor disabled	Transports::bind (line ~4400) and LocalAddrsWatch
Direct UDP path never appears	NAT traversal probe timed out or empty relay map	DirectAddrUpdateState::run (~8800), check relay_map.is_empty()
Relay works but direct fails	RelayOnly mode forced or missing IP transports	Socket::my_relay (~8890) and transport config
Stale addresses advertised	publish_my_addr not called after changes	Socket::store_direct_addresses (~5110) → publish_my_addr (~6730)
High latency	Sub-optimal path selected	biased_rtt_path_selector.rs PathSelector implementation

Step-by-Step Debugging Workflow

Enable Tracing

Set the RUST_LOG environment variable before running your application to capture detailed logs from the socket and transport layers:

RUST_LOG=iroh=debug,iroh::socket=trace cargo run

This outputs logs from Socket::process_datagrams, DirectAddrUpdateState::run, and net-report timeouts. Look for messages indicating datagram processing and address updates.

Inspect Metrics

The Metrics struct exposes counters that reveal runtime behavior. Dump them locally to verify packet flow:

println!("{}", iroh::metrics::registry().gather());

Watch for socket_recv_datagrams, socket_recv_gro_datagrams, and net_report_portmap_attempts. Zero values for reception counters indicate the Socket is not receiving data from the Transports layer.

Watch Address Changes

The Socket exposes async watchers for direct addresses and net-reports. Use these to confirm discovery is working:

let mut addr_watcher = endpoint.socket().ip_addrs();
while let Some(addrs) = addr_watcher.next().await {
    println!("Direct addresses updated: {:?}", addrs);
}

If this stream never yields values, the DirectAddrUpdateState has not discovered any IPv4/IPv6 sockets, pointing to a binding or NAT traversal failure.

Force Network Changes

Trigger a manual network change to force a fresh net-report and address refresh:

endpoint.inner().network_change().await;

This sends a NetworkChange actor message that restarts the NetReport probe and updates the Socket address list.

Check Relay Configuration

Verify that a relay is present and reachable:

let relay = endpoint.socket().my_relay();
println!("Current relay: {:?}", relay);

If this prints None, check the transport configuration passed to EndpointBuilder::bind and ensure options.transports includes relay definitions.

Validate Address Lookup

If using PKARR or custom address lookup, confirm published data matches peer expectations:

let lookup = endpoint.socket().address_lookup();
lookup.publish(&iroh::address_lookup::EndpointData::new(vec![]));

Verify remote peers see these updates by checking their remote_info or the lookup service logs.

Practical Code Examples

Basic Listener with Debug Logging

This example enables detailed tracing and watches for direct address discoveries:

use iroh::EndpointBuilder;
use tracing_subscriber::fmt::Subscriber;

#[tokio::main]
async fn main() {
    // Enable detailed tracing
    Subscriber::builder()
        .with_env_filter("iroh=debug,iroh::socket=trace")
        .init();

    let endpoint = EndpointBuilder::default()
        .bind()
        .await
        .expect("failed to bind");

    // Watch direct address changes
    let mut watcher = endpoint.socket().ip_addrs();
    tokio::spawn(async move {
        while let Some(addrs) = watcher.next().await {
            eprintln!("Direct addresses: {:?}", addrs);
        }
    });

    // Keep the process alive
    endpoint.wait_for_shutdown().await;
}

The trace logs show each call to process_datagrams, store_direct_addresses, and publish_my_addr, while the watcher prints discovered UDP sockets as soon as the net-report succeeds.

Forcing Net-Report Probes

Use this pattern to verify that NetReport is successfully discovering NAT mappings:

use iroh::EndpointBuilder;

#[tokio::main]
async fn main() {
    let endpoint = EndpointBuilder::default().bind().await.unwrap();

    // Trigger a network change (forces a net-report)
    endpoint.inner().network_change().await;

    // Await the next net-report value
    let mut report_watcher = endpoint.socket().net_report();
    if let Some(report) = report_watcher.next().await {
        println!("Net-report: {:?}", report);
    }
}

The net_report watcher yields a Report struct containing NAT mappings, best relay, and RTT estimates. If None is returned, the probe timed out and you should investigate DirectAddrUpdateState::run.

Monitoring Path Selection

When latency seems incorrect, inspect the path selector's internal state:

use iroh::EndpointBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    // Use the built-in biased RTT selector
    let selector = Arc::new(iroh::socket::biased_rtt_path_selector::BiasedRttPathSelector::default());

    let endpoint = EndpointBuilder::default()
        .path_selector(selector.clone())
        .bind()
        .await
        .unwrap();

    // After a few seconds, dump the selector's internal state
    tokio::time::sleep(std::time::Duration::from_secs(5)).await;
    println!("Selector state: {:?}", selector);
}

This reveals RTT samples per path and shows whether direct paths are being ignored due to high latency or bias settings.

Key Source Files for Debugging

• iroh/src/socket.rs – Central socket implementation and EndpointInner glue
• iroh/src/socket/transports.rs – Transport creation, binding, and network-change signaling
• iroh/src/socket/remote_map.rs – Address mapping and multiplexing logic
• iroh/src/net_report.rs – QAD probing and NAT discovery
• iroh/src/address_lookup.rs – PKARR and external address publishing
• iroh/src/socket/metrics.rs – Runtime counters and Prometheus integration
• iroh/src/socket/biased_rtt_path_selector.rs – Default path-selection algorithm

Summary

• Enable tracing with RUST_LOG=iroh=debug,iroh::socket=trace to see datagram processing and address updates in real-time.
• Monitor watchers for ip_addrs() and net_report() to confirm the NetReport probe is discovering direct paths.
• Check metrics for socket_recv_datagrams and net_report_portmap_attempts to verify packet flow and probing activity.
• Force updates with endpoint.inner().network_change().await when testing network transitions.
• Inspect relays via socket().my_relay() and verify RelayOnly mode is not forced in the transport config.
• Validate lookups when using PKARR to ensure addresses are published and consumed correctly.

Frequently Asked Questions

Why are my iroh nodes not connecting directly?

Direct connections fail when the NetReport probe cannot discover NAT mappings or when the RemoteMap has no valid paths. Check DirectAddrUpdateState::run in iroh/src/socket.rs to see if the probe timed out, and verify the relay map is not empty. If behind restrictive NAT, ensure the relay path is available as a fallback.

How do I enable detailed logging for iroh network debugging?

Set the RUST_LOG environment variable to iroh=debug,iroh::socket=trace before starting your application. This activates tracing spans for Socket::process_datagrams, transport bindings, and net-report timeouts. Combine this with tracing_subscriber in your code to format output to stdout or stderr.

What does the NetReport probe do in iroh?

NetReport runs QUIC Address Discovery (QAD) probes periodically to determine the node's public IP addresses, NAT mappings, and the best available relay. It publishes results to the Socket via the net_report() watcher. If this probe fails, the node only knows its local addresses and relies entirely on relay traffic, impacting performance.

How can I force iroh to refresh its network state?

Call endpoint.inner().network_change().await to trigger a NetworkChange actor message. This forces the Socket to re-run the NetReport probe, re-check LocalAddrsWatch, and update the RemoteMap with any new addresses. Use this after network interface changes or when testing path selection in controlled environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}