How to Implement Custom Protocol Handlers in iroh: A Complete Guide

Implementing custom protocol handlers in iroh requires creating a Rust type that implements the ProtocolHandler trait, registering it with a unique ALPN string via RouterBuilder::accept, and spawning the router to dispatch incoming QUIC connections to your handler.

The iroh library from n0-computer provides a QUIC-based networking stack that routes connections by their Application-Layer Protocol Negotiation (ALPN) strings. To implement custom protocol handlers in iroh, you define a struct implementing the ProtocolHandler trait and register it with the router, enabling you to build custom peer-to-peer protocols while leveraging iroh's NAT traversal and connection management.

The ProtocolHandler Architecture

At the core of iroh's extensibility is the ProtocolHandler trait defined in iroh/src/protocol.rs#L27-L31. This trait defines the asynchronous callbacks the router invokes when a matching ALPN is detected during the TLS handshake.

Core Components

The routing system relies on three key components:

• ProtocolHandler trait – Defines the accept method (required) and optional on_accepting and shutdown callbacks. Located in iroh/src/protocol.rs#L27-L31.
• RouterBuilder::accept – Registers a handler for a specific ALPN value, storing the mapping in an internal ProtocolMap. Found at iroh/src/protocol.rs#L84-L92.
• Endpoint::set_alpns – Updates the endpoint's advertised ALPN list so clients can negotiate the custom protocol. Implemented in iroh/src/endpoint.rs#L49-L59.

Connection Flow

When you spawn a router using Router::spawn, the accept loop in iroh/src/protocol.rs#L124-L142 extracts the ALPN from each incoming TLS handshake and dispatches to the matching handler. The router automatically calls Endpoint::set_alpns internally, ensuring your endpoint advertises all registered protocols without manual configuration.

Implementing a Custom Echo Protocol

Below is a complete, runnable example demonstrating how to implement a custom protocol handler in iroh. This "echo" protocol reads data from the client and writes it back unchanged.

use iroh::{
    endpoint::{Endpoint, presets},
    protocol::{ProtocolHandler, Router},
};
use n0_error::Result;

// Define the handler struct
#[derive(Debug, Clone)]
struct Echo;

impl ProtocolHandler for Echo {
    // Required: handle the incoming connection
    async fn accept(
        &self,
        connection: iroh::endpoint::Connection,
    ) -> Result<(), iroh::protocol::AcceptError> {
        let (mut send, mut recv) = connection.accept_bi().await?;
        
        // Echo all received data back to the peer
        tokio::io::copy(&mut recv, &mut send).await?;
        
        // Gracefully close the stream and wait for peer acknowledgment
        send.finish()?;
        connection.closed().await;
        Ok(())
    }
}

#[tokio::main]
async fn main() -> Result<()> {
    // Bind a local endpoint using production-grade defaults
    let endpoint = Endpoint::bind(presets::N0).await?;

    // Define a unique ALPN identifier for this protocol
    const ECHO_ALPN: &[u8] = b"/iroh/echo/1";

    // Build the router and register the handler
    let router = Router::builder(endpoint)
        .accept(ECHO_ALPN, Echo)   // Registration happens here
        .spawn();

    println!("Server address: {}", router.endpoint().addr());

    // Demonstrate a client connection (normally in a separate process)
    let client = Endpoint::bind(presets::N0).await?;
    let conn = client.connect(router.endpoint().addr(), ECHO_ALPN).await?;
    let (mut send, mut recv) = conn.accept_bi().await?;

    // Send test payload
    let payload = b"hello iroh!";
    send.write_all(payload).await?;
    send.finish()?;

    // Read the echo response
    let mut echoed = Vec::new();
    tokio::io::copy(&mut recv, &mut echoed).await?;
    println!("Echoed back: {}", String::from_utf8_lossy(&echoed));

    // Cleanup
    router.shutdown().await?;
    client.close().await;
    Ok(())
}

Registering and Configuring Your Protocol

The registration process follows a specific pattern to ensure proper ALPN negotiation:

1. Choose a unique ALPN – Select a byte string that won't conflict with built-in protocols. The example uses b"/iroh/echo/1".

2. Implement the trait – Your struct must implement ProtocolHandler. Only the accept method is mandatory; on_accepting (for early connection inspection) and shutdown (for cleanup) are optional.

3. Register with the builder – Call RouterBuilder::accept(alpn, handler) as shown in iroh/src/protocol.rs#L84-L92. This stores the handler in the router's internal ProtocolMap.

4. Spawn the router – The spawn() method starts the accept loop and automatically invokes Endpoint::set_alpns to advertise your protocol to connecting peers.

Source Code References

You can examine the implementation details in the following locations:

• iroh/src/protocol.rs – Contains the ProtocolHandler trait definition, RouterBuilder::accept registration logic, and the dispatch mechanism in handle_connection (L124-L142).
• iroh/src/endpoint.rs – Implements set_alpns (L49-L59) which updates the QUIC server configuration with registered ALPNs.
• iroh/examples/echo.rs – A canonical reference implementation matching the echo pattern above.
• iroh/tests/patchbay/ – Integration tests demonstrating custom protocol registration and graceful shutdown behavior.

Best Practices for Custom Protocols

When implementing custom protocol handlers in production iroh applications, keep these considerations in mind:

• ALPN uniqueness – Avoid collisions with standard protocols by using namespaced identifiers like /myapp/protocol/1.
• Thread safety – Handlers must be Send + Sync. Wrap shared mutable state in Arc<Mutex<T>> or similar synchronization primitives.
• Back-pressure handling – The accept future runs on a spawned task, so performing long-running work inside it won't block other connections.
• Graceful shutdown – Implement the shutdown method if your protocol holds resources that require cleanup. The router calls this method before aborting pending connections during shutdown.

Summary

• ProtocolHandler trait – The core interface at iroh/src/protocol.rs requiring only the accept method to handle incoming connections.
• ALPN registration – Use RouterBuilder::accept with a unique byte string to register your handler; the router automatically advertises it via Endpoint::set_alpns.
• Connection dispatch – The router's accept loop matches incoming QUIC connections by ALPN and routes them to your handler's accept method.
• Optional lifecycle hooks – Implement on_accepting for early connection inspection and shutdown for resource cleanup.
• Reference examples – Study iroh/examples/echo.rs and the integration tests in iroh/tests/patchbay/ for production patterns.

Frequently Asked Questions

What is ALPN and why does iroh use it for protocol routing?

Application-Layer Protocol Negotiation (ALPN) is a standard TLS extension that allows clients and servers to negotiate the application protocol during the handshake. iroh uses ALPN strings to identify which ProtocolHandler should receive an incoming QUIC connection, similar to how HTTP/3 or SSH negotiate protocols. This occurs in the router's handle_connection method at iroh/src/protocol.rs#L124-L142.

Do I need to implement all methods on the ProtocolHandler trait?

No, only the accept method is required. The ProtocolHandler trait provides default implementations for on_accepting (which simply returns the connection unchanged) and shutdown (which performs no cleanup). Implement these optional methods only if you need to inspect connections before full acceptance or require specific teardown logic.

How do I handle multiple concurrent connections in my custom protocol?

Each incoming connection spawns its own task running your accept method. Because ProtocolHandler requires Send + Sync, you can safely share state between connections using Arc<Mutex<T>> or atomic types. The router handles the concurrency management, so your handler code can focus on protocol logic without worrying about blocking the accept loop.

Can I register multiple custom protocols on a single iroh endpoint?

Yes, you can register unlimited protocols by calling RouterBuilder::accept multiple times with different ALPN strings before spawning the router. The router maintains an internal ProtocolMap (as seen in iroh/src/protocol.rs) and dispatches each incoming connection to the appropriate handler based on the client's requested ALPN.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}