How iroh Memory Address Lookup Works for Local Peer Discovery

iroh uses a pluggable MemoryLookup implementation that stores peer addresses in a thread-safe BTreeMap, enabling zero-latency local discovery without network traffic.

The n0-computer/iroh project provides a modular address lookup system that allows an Endpoint to locate network addresses of remote peers. The built-in memory-based implementation serves as the simplest mechanism for supplying locally-known address information, making it ideal for testing, simulations, and out-of-band address distribution via tickets or configuration files.

The AddressLookup Trait Architecture

Core Interface Definitions

All address lookup mechanisms in iroh implement the AddressLookup trait defined in iroh/src/address_lookup.rs. This trait specifies two fundamental operations:

• publish(&self, data: &EndpointData) – Notifies the lookup of local endpoint data (noop for memory-based implementations).
• resolve(&self, endpoint_id) -> Option<BoxStream<Item>> – Retrieves address information for a specific peer, returning an async stream of discovery items.

The trait abstracts over various discovery backends, allowing the Endpoint to treat memory lookups, DNS resolution, and QUIC address discovery uniformly.

MemoryLookup Implementation Details

The MemoryLookup struct in iroh/src/address_lookup/memory.rs provides a concrete implementation using an Arc<RwLock<BTreeMap<EndpointId, StoredEndpointInfo>>> (lines 75-80). This thread-safe hash map stores StoredEndpointInfo entries, which wrap EndpointData alongside a last_updated timestamp and provenance metadata.

How MemoryLookup Works in Practice

Construction and Registration

Applications instantiate MemoryLookup before binding an Endpoint. The lookup is passed to the builder via the address_lookup() method:

let address_lookup = MemoryLookup::new();
let ep = Endpoint::builder(presets::N0)
    .address_lookup(address_lookup.clone())
    .bind()
    .await?;

This pattern appears in memory.rs (lines 51-57) and demonstrates how the lookup becomes integrated into the endpoint's discovery pipeline.

Adding Endpoint Information

To populate the local registry, applications call add_endpoint_info with an EndpointAddr or EndpointInfo. This method:

1. Records the current system time as a timestamp.
2. Merges the new address set with any existing entry for that EndpointId.
3. Attaches the provenance string "memory_lookup" for debugging purposes.

The implementation spans lines 84-95 in memory.rs, ensuring thread-safe updates via the RwLock guard.

Resolving Peer Addresses

When the endpoint needs to contact a remote peer, it invokes resolve(peer_id). The MemoryLookup implementation:

• Acquires a read lock on the underlying BTreeMap.
• Retrieves the StoredEndpointInfo for the requested EndpointId.
• Constructs an Item containing the endpoint info, provenance, and timestamp (converted to microseconds).
• Returns the item wrapped in a one-element async stream: stream::iter(Some(Ok(item))).boxed().

If the peer ID is unknown, the method returns None, triggering fallback to other discovery mechanisms like QUIC address discovery. This logic appears in lines 22-38 of memory.rs.

Integration with the Endpoint

The Endpoint integration occurs in iroh/src/endpoint.rs (lines 251-262), where the builder stores the lookup implementation. When the endpoint needs to establish a connection, it calls lookup.resolve(id) and processes the resulting stream identically to other lookup implementations.

Because the in-memory store operates synchronously under read locks, local peer discovery achieves effectively zero latency. This characteristic makes MemoryLookup essential for test harnesses and simulations where network traffic must be minimized.

Practical Usage Example

The following example demonstrates the complete lifecycle: creating the lookup, registering a remote peer, and resolving its address:

use iroh::{
    Endpoint, EndpointAddr, TransportAddr,
    address_lookup::memory::MemoryLookup,
    endpoint::presets,
};
use iroh_base::SecretKey;

// Create a memory lookup and an endpoint that uses it
let lookup = MemoryLookup::new();
let ep = Endpoint::builder(presets::Minimal)
    .address_lookup(lookup.clone())
    .bind()
    .await?;

// Insert a remote peer's address obtained out-of-band
let key = SecretKey::from_bytes(&[0u8; 32]);
let remote = EndpointAddr::from_parts(
    key.public(),
    [TransportAddr::Relay("https://example.com".parse()?)],
);
lookup.add_endpoint_info(remote.clone());

// Resolve the remote peer
if let Some(mut stream) = lookup.resolve(key.public()) {
    let item = stream.next().await.unwrap()?;
    println!("Discovered via {}: {:?}", item.provenance(), item);
}

This pattern mirrors the unit tests in memory.rs (lines 52-70 and 88-99) and appears in iroh/examples/custom-transport.rs for custom transport scenarios. Additionally, iroh/src/socket.rs (lines 2337-2344) uses MemoryLookup to create paired endpoints sharing the same lookup instance.

Summary

• iroh memory address lookup operates through the AddressLookup trait, enabling pluggable discovery mechanisms.
• MemoryLookup stores peer addresses in an Arc<RwLock<BTreeMap<EndpointId, StoredEndpointInfo>>>, providing thread-safe access.
• The add_endpoint_info method registers out-of-band address data with provenance tracking.
• resolve returns Option<BoxStream<Item>>, allowing the Endpoint to treat memory lookups identically to network-based discovery.
• This implementation enables zero-latency local peer discovery, making it ideal for testing, CI pipelines, and controlled deployments.

Frequently Asked Questions

What is the primary use case for iroh's MemoryLookup?

MemoryLookup is designed for scenarios where peer addresses are already known locally, such as in unit tests, simulations, or when addresses are distributed via configuration files or tickets. It eliminates network traffic during discovery, providing deterministic, instantaneous address resolution that is not possible with DNS or QUIC-based discovery.

How does MemoryLookup handle concurrent access?

The implementation uses an Arc<RwLock<BTreeMap<...>>> pattern that allows multiple readers or a single writer. Read operations during resolve acquire read locks briefly to clone the stored endpoint info, while add_endpoint_info acquires write locks to update the map. This ensures thread safety without blocking the async runtime during resolution.

Can MemoryLookup be combined with other discovery methods?

Yes. The Endpoint treats all lookup implementations uniformly through the AddressLookup trait. When resolve returns None for an unknown peer, the endpoint automatically falls back to other configured discovery mechanisms. Production applications typically use MemoryLookup alongside DNS or QUIC discovery, with the memory lookup serving as a fast-path cache for recently-seen peers.

Where is MemoryLookup used in the iroh codebase?

Beyond the core implementation in iroh/src/address_lookup/memory.rs, MemoryLookup appears in iroh/examples/custom-transport.rs for custom transport demonstrations and in iroh/src/socket.rs (lines 2337-2344) to create paired endpoints that share address information. It is also extensively used in the project's test suite to create isolated network environments without actual TCP/UDP binding.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}