Debugging Iroh Connection Issues: A Complete Guide to Network Diagnostics

Debugging Iroh connection issues requires tracing datagrams through the Socket abstraction, monitoring the NetReport probe for NAT traversal status, and inspecting path selection decisions via runtime metrics and tracing logs.

The iroh crate from n0-computer implements a QUIC-based networking layer that dynamically switches between direct UDP, relay, and custom transports at runtime. When connectivity fails, understanding the interaction between the Socket, Transports, and NetReport components is essential for diagnosing whether the issue stems from NAT mapping, relay configuration, or path selection logic.

Core Architecture Components

Iroh’s networking stack consists of several tightly integrated modules that handle address discovery, transport abstraction, and path selection.

The Socket (iroh/src/socket.rs) serves as the central connectivity layer, managing transports, address discovery, and path selection. It handles incoming and outgoing datagrams, maintains local and remote address lists, and drives the RemoteStateActor for each peer.

The Transports subsystem (iroh/src/socket/transports.rs) abstracts over UDP, relay (WebSocket), and custom transports. It creates concrete sockets, watches for network changes, and forwards packets to the Socket.

The RemoteMap (iroh/src/socket/remote_map.rs) maps synthetic “mapped” addresses to real transport addresses, enabling multiplexing across multiple paths (direct, relay, custom) and helping the Socket translate between them.

The NetReport module (iroh/src/net_report.rs) probes the network to discover NAT mappings, reachable IPs, and the best relay. It periodically runs a QAD (QUIC address discovery) probe and publishes results to the Socket.

Additional components include AddressLookup (iroh/src/address_lookup.rs) for optional external services like PKARR, Metrics (iroh/src/socket/metrics.rs) for Prometheus-style counters, and EndpointInner (iroh/src/socket.rs) which glues the Socket to the underlying noq::Endpoint.

Common Failure Points and Diagnosis

Connection failures in Iroh typically manifest through specific symptoms that map to architectural constraints.

No packets received on either side indicates the Socket is not receiving datagrams, usually because the transport failed to bind or the network monitor is disabled. Check Transports::bind (around line 4400) and the LocalAddrsWatch in socket.rs.

Direct UDP path never appears suggests the NAT traversal probe timed out or the relay map is empty, meaning NetReport never started. Inspect DirectAddrUpdateState::run (around line 8800) and verify relay_map.is_empty() and NET_REPORT_TIMEOUT.

Relay traffic works but direct fails indicates relay-only mode (RelayOnly) may be forced, or the relay map is missing IP transports. Review Socket::my_relay (around line 8890) and the RelayOnly flag in the transport configuration.

Stale addresses advertised occurs when publish_my_addr is not called after network changes. Examine Socket::store_direct_addresses (around line 5110) to ensure it triggers publish_my_addr (around line 6730).

High latency or excessive retransmits suggests the path selector prefers a sub-optimal route. Review biased_rtt_path_selector.rs to inspect the PathSelector implementation.

Step-by-Step Debugging Workflow

Follow this systematic approach to isolate Iroh connection issues using tracing, metrics, and runtime introspection.

1. Enable tracing logs. Iroh uses the tracing crate. Set the RUST_LOG environment variable to capture detailed logs from Socket::process_datagrams, DirectAddrUpdateState::run, and the net-report timeout.

   RUST_LOG=iroh=debug,iroh::socket=trace cargo run --example listen

2. Inspect Prometheus metrics. The Metrics struct registers counters exportable to Prometheus. For local debugging, dump the registry to stdout:

   println!("{}", iroh::metrics::registry().gather());

   Monitor socket_recv_datagrams, socket_recv_gro_datagrams, and net_report_portmap_attempts to verify probes are executing.

3. Watch address changes. The Socket exposes watchers for direct addresses and net-report status. Use these to confirm DirectAddrUpdateState has discovered IPv4/IPv6 sockets:

   let mut addr_watcher = endpoint.socket().ip_addrs();
   while let Some(addrs) = addr_watcher.next().await {
       println!("Direct addresses updated: {:?}", addrs);
   }

4. Force a network change. Trigger a re-probe by sending a NetworkChange actor message:

   endpoint.inner().network_change().await;

   This forces the socket to re-run the net-report and refresh the address list.

5. Verify relay configuration. Ensure a relay is present and reachable:

   let relay = endpoint.socket().my_relay();
   println!("Current relay: {:?}", relay);

   If None is returned, verify the transport configuration passed to EndpointBuilder::bind.

6. Validate address lookup. For PKARR or custom lookup services, confirm published data matches peer expectations:

   let lookup = endpoint.socket().address_lookup();
   lookup.publish(&iroh::address_lookup::EndpointData::new(vec![]));

Practical Debugging Code Examples

Running a Listener with Debug Output

This example enables detailed tracing and watches for direct address updates, revealing calls to process_datagrams, store_direct_addresses, and publish_my_addr.

use iroh::EndpointBuilder;
use tracing_subscriber::fmt::Subscriber;

#[tokio::main]
async fn main() {
    Subscriber::builder()
        .with_env_filter("iroh=debug,iroh::socket=trace")
        .init();

    let endpoint = EndpointBuilder::default()
        .bind()
        .await
        .expect("failed to bind");

    let mut watcher = endpoint.socket().ip_addrs();
    tokio::spawn(async move {
        while let Some(addrs) = watcher.next().await {
            eprintln!("Direct addresses: {:?}", addrs);
        }
    });

    endpoint.wait_for_shutdown().await;
}

Forcing a Net-Report Probe

Trigger a network change and await the report to verify NAT traversal is functioning. The net_report watcher yields a Report struct containing NAT mappings, best relay, and RTT estimates.

use iroh::EndpointBuilder;

#[tokio::main]
async fn main() {
    let endpoint = EndpointBuilder::default().bind().await.unwrap();

    endpoint.inner().network_change().await;

    let mut report_watcher = endpoint.socket().net_report();
    if let Some(report) = report_watcher.next().await {
        println!("Net-report: {:?}", report);
    }
}

Inspecting Path Selector Decisions

Use the biased RTT path selector to diagnose latency issues. The selector tracks RTT samples per path; printing its internal state reveals whether direct paths are being ignored.

use iroh::endpoint::{EndpointBuilder, PathSelector};
use std::sync::Arc;

#[tokio::main]
async fn main() {
    let selector = Arc::new(
        iroh::socket::biased_rtt_path_selector::BiasedRttPathSelector::default()
    );

    let endpoint = EndpointBuilder::default()
        .path_selector(selector.clone())
        .bind()
        .await
        .unwrap();

    tokio::time::sleep(std::time::Duration::from_secs(5)).await;
    println!("Selector state: {:?}", selector);
}

Key Source Files for Debugging

When tracing connection issues, focus on these files in the n0-computer/iroh repository:

• iroh/src/socket.rs – Central socket implementation and entry point for packet processing.
• iroh/src/socket/transports.rs – Transport creation, binding, and network-change signaling.
• iroh/src/socket/remote_map.rs – Address mapping between synthetic and real transport addresses.
• iroh/src/net_report.rs – QAD probing logic for NAT traversal and relay discovery.
• iroh/src/address_lookup.rs – PKARR integration and endpoint address publication.
• iroh/src/socket/metrics.rs – Prometheus counters for socket and transport diagnostics.
• iroh/src/socket/biased_rtt_path_selector.rs – Default path selection algorithm.

Summary

• Enable tracing with RUST_LOG=iroh=debug to observe Socket::process_datagrams and DirectAddrUpdateState::run.
• Monitor metrics for socket_recv_datagrams and net_report_portmap_attempts to verify probe execution.
• Watch address streams using endpoint.socket().ip_addrs() and net_report() to confirm NAT traversal results.
• Force re-probes via endpoint.inner().network_change().await when network conditions change.
• Check relay status with endpoint.socket().my_relay() to ensure fallback connectivity is available.
• Inspect path selection using the biased RTT selector when experiencing high latency or retransmits.

Frequently Asked Questions

How do I enable debug logging for Iroh connection issues?

Set the RUST_LOG environment variable to iroh=debug,iroh::socket=trace before running your application. This activates the tracing subscriber and emits logs from Socket::process_datagrams, transport binding operations, and the net-report timeout handlers, allowing you to trace datagram flow through the system.

Why is my direct UDP path not appearing in Iroh?

A missing direct path typically indicates the NetReport probe failed or timed out. Check DirectAddrUpdateState::run (around line 8800 in iroh/src/socket.rs) to verify the relay map is not empty and that NET_REPORT_TIMEOUT has not expired. Also ensure the RelayOnly flag is not forced in your transport configuration.

How do I force a network re-probe in Iroh?

Call endpoint.inner().network_change().await to trigger a NetworkChange actor message. This forces the Socket to re-run the net-report probe, refresh the address list via store_direct_addresses, and republish new endpoints via publish_my_addr, which is useful when switching networks or diagnosing stale address issues.

What metrics should I monitor to diagnose Iroh connectivity problems?

Key metrics include socket_recv_datagrams and socket_recv_gro_datagrams to confirm packet reception, net_report_portmap_attempts to verify NAT traversal probes are executing, and counters from iroh/src/socket/metrics.rs tracking path state changes. Export these to Prometheus or dump them locally via iroh::metrics::registry().gather().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}