How iroh AfterHandshakeOutcome Connection Hooks Control Incoming QUIC Connections

AfterHandshakeOutcome is the result type returned by the after_handshake hook in iroh's EndpointHooks trait, determining whether a QUIC connection is accepted or immediately rejected with a specific error code after the TLS handshake completes.

The iroh distributed systems toolkit provides fine-grained control over connection establishment through its endpoint hooks API. When implementing custom authorization or inspection logic in iroh, understanding the AfterHandshakeOutcome type and the connection hooks workflow is essential for rejecting unwanted connections before they reach your application logic.

AfterHandshakeOutcome Enum and Variants

The AfterHandshakeOutcome enum is defined in iroh/src/endpoint/hooks.rs and represents the decision point after a TLS handshake finishes. According to the source code at lines 18-34 and 36-44, the enum has two variants:

• Accept – Signals that the connection should proceed normally
• Reject { error_code, reason } – Signals that the connection should be closed immediately with the specified QUIC error code and reason string

The Reject variant includes a reject helper method that constructs the rejection outcome with the appropriate error code and reason bytes. When a hook decides to reject, it returns AfterHandshakeOutcome::Reject { error_code, reason }, which the endpoint uses to terminate the connection before any application-level communication occurs.

The EndpointHooks Trait Interface

Connection hooks in iroh are implemented through the EndpointHooks trait, also located in iroh/src/endpoint/hooks.rs (lines 87-106). This trait defines the after_handshake method signature:

fn after_handshake<'a>(
    &'a self,
    conn: &'a Connection,
) -> impl Future<Output = AfterHandshakeOutcome> + Send + 'a;

The default implementation provided by the trait simply returns AfterHandshakeOutcome::Accept, allowing all connections through. Custom implementations can inspect the Connection reference to examine remote endpoint IDs, ALPN protocols, or other connection metadata before returning either Accept or Reject.

Hook Execution Flow in EndpointHooksList

Iroh supports multiple registered hooks through the EndpointHooksList struct. The after_handshake implementation in EndpointHooksList (lines 60-70 in iroh/src/endpoint/hooks.rs) processes hooks sequentially:

1. Iterates through each registered hook in the order they were added
2. Awaits the outcome of each hook's after_handshake future
3. If a hook returns AfterHandshakeOutcome::Accept, continues to the next hook
4. If any hook returns AfterHandshakeOutcome::Reject, immediately stops processing and returns the rejection

This short-circuit behavior ensures that the first rejection wins, preventing subsequent hooks from executing once a connection has been flagged for rejection.

Connection Handling and Rejection Logic

The concrete integration between the handshake outcome and connection lifecycle appears in iroh/src/endpoint/connection.rs at lines 52-57. After the TLS handshake completes, the endpoint invokes hooks.after_handshake. If the result is Reject, the endpoint:

1. Closes the connection immediately using the provided error_code and reason
2. Returns ConnectingError::LocallyRejected to the caller
3. Prevents the connection from being handed off to application code

This ensures rejected connections never consume application-level resources, with the peer receiving the QUIC error code and reason string for debugging.

Practical Example: ALPN-Based Rejection

The following example demonstrates implementing EndpointHooks to reject connections that advertise an unexpected ALPN protocol:

use iroh::endpoint::{Endpoint, EndpointHooks, AfterHandshakeOutcome, Connection};
use iroh::endpoint::hooks::BeforeConnectOutcome;

#[derive(Debug)]
struct RejectLargeFiles;

impl EndpointHooks for RejectLargeFiles {
    // The default `before_connect` is fine; we only care about post‑handshake.
    fn after_handshake<'a>(
        &'a self,
        conn: &'a Connection,
    ) -> impl std::future::Future<Output = AfterHandshakeOutcome> + Send + 'a {
        async move {
            // Example: reject connections that advertise an unexpected ALPN.
            if conn.alpn() != b"iroh/1" {
                // 0x0b is the QUIC "CryptoError" code; choose any appropriate code.
                AfterHandshakeOutcome::Reject {
                    error_code: 0x0b.into(),
                    reason: b"Unsupported ALPN".to_vec(),
                }
            } else {
                AfterHandshakeOutcome::Accept
            }
        }
    }
}

#[tokio::main]
async fn main() {
    // Build an endpoint with the custom hook.
    let ep = iroh::Endpoint::builder()
        .hooks(RejectLargeFiles)
        .listen()
        .await
        .unwrap();

    // Normal usage – incoming connections will be examined by the hook.
    // If the hook rejects, the peer receives the error code/reason and the
    // connection is not handed to the application.
}

In this implementation, the hook inspects the connection's ALPN identifier after the TLS handshake completes. Connections not using the expected protocol are rejected with QUIC error code 0x0b (CryptoError) and the reason string "Unsupported ALPN".

Summary

• AfterHandshakeOutcome is the decision enum returned by after_handshake hooks in iroh/src/endpoint/hooks.rs, with Accept and Reject variants.
• Hooks are processed sequentially via EndpointHooksList::after_handshake, short-circuiting on the first rejection.
• Rejections immediately close the connection with the specified QUIC error code and return ConnectingError::LocallyRejected from iroh/src/endpoint/connection.rs.
• The Connection reference passed to hooks allows inspection of ALPN, remote endpoint IDs, and other metadata before making authorization decisions.

Frequently Asked Questions

What happens when multiple after_handshake hooks are registered?

Iroh processes hooks in the order they were added to the EndpointHooksList. If any hook returns AfterHandshakeOutcome::Reject, the loop terminates immediately and the connection closes without executing remaining hooks. If all hooks return Accept, the connection proceeds to the application.

Can I inspect connection metadata before deciding to reject?

Yes. The after_handshake hook receives a reference to the Connection struct, allowing inspection of the ALPN protocol, remote endpoint ID, and other connection parameters defined in the established TLS context. This metadata inspection occurs after the cryptographic handshake but before application protocols begin.

What QUIC error code should I use when rejecting a connection?

The choice depends on your rejection reason. The example in iroh/src/endpoint/hooks.rs and related files uses 0x0b (representing a cryptographic error), but you may use any valid QUIC error code appropriate for your application's protocol. The reason field accepts a byte vector for human-readable debugging information.

How is AfterHandshakeOutcome different from BeforeConnectOutcome?

BeforeConnectOutcome controls whether to initiate an outgoing connection attempt, while AfterHandshakeOutcome controls whether to accept an incoming connection after the TLS handshake completes. The former runs before network activity begins, whereas the latter runs after cryptographic verification but before application data flows, making it suitable for post-authentication authorization decisions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}