# How to Implement Custom Authentication Logic in Iroh: 3 Proven Methods

> Implement custom authentication logic in Iroh by extending handshake protocols or using the AccessControl trait for fine-grained authorization. Discover 3 proven methods.

- Repository: [number zero/iroh](https://github.com/n0-computer/iroh)
- Tags: how-to-guide
- Published: 2026-07-15

---

**To implement custom authentication logic in Iroh, extend the handshake protocol in [`iroh-relay/src/protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/protos/handshake.rs) or implement the `AccessControl` trait from [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs) to enforce fine-grained authorization policies after cryptographic verification.**

Iroh's relay layer authenticates clients during the TLS or QUIC handshake and authorizes them before allowing traffic. The n0-computer/iroh repository provides two primary extension points for customizing this behavior: replacing the default challenge-response handshake or implementing post-authentication access controls. This guide covers three methods to implement custom authentication logic in Iroh, from low-level protocol changes to high-level policy enforcement.

## Understanding Iroh's Authentication Architecture

Iroh separates identity verification (authentication) from permission grants (authorization). The default flow uses cryptographic challenge-response to prove identity, then checks permissions via a pluggable trait.

### The Handshake Protocol

In [`iroh-relay/src/protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/protos/handshake.rs), the server handles two authentication mechanisms:

- **Challenge-Response**: The server generates a random challenge that the client signs with the secret key belonging to its `EndpointId`. The server verifies this using `ClientAuth::verify` (lines 456-470).
- **Key-Material Authentication**: If the client sends a `KeyMaterialClientAuth` header, the server validates TLS keying material instead of the challenge (lines 425-444).

After successful cryptographic verification, the server creates a `SuccessfulAuthentication` instance and calls `authorize_with`, which delegates to the `AccessControl` trait implementation (defined in [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs), line 285).

### The Access Control Layer

Authentication proves *who* the client is. Authorization decides *what* they may do. The `AccessControl` trait provides the hook to implement custom authorization logic after the cryptographic handshake succeeds.

## Method 1: Extend the Handshake Protocol

Replace or extend the default handshake to add custom authentication mechanisms like device certificates or hardware security modules.

**Hook location**: `handshake::serverside` constructs the `SuccessfulAuthentication` instance after verifying the client auth header (lines 418-442 in [`iroh-relay/src/protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/protos/handshake.rs)).

**Implementation steps**:

1. Create a new struct (e.g., `CustomDeviceAuth`) implementing `serde::Deserialize` and a `verify` method:

```rust
use serde::Deserialize;

#[derive(Deserialize)]
struct CustomDeviceAuth {
    device_cert: Vec<u8>,
    signature: Vec<u8>,
}

impl CustomDeviceAuth {
    async fn verify(&self, io: &mut impl AsyncRead + AsyncWrite) -> anyhow::Result<()> {
        // Verify hardware-backed device certificate
        // Return Ok(()) if valid, Err otherwise
        todo!()
    }
}

```

2. Modify `serverside` to attempt deserialization into your struct before falling back to `ClientAuth`:

```rust
// In iroh-relay/src/protos/handshake.rs serverside function
if let Ok(custom_auth) = postcard::from_bytes::<CustomDeviceAuth>(&client_auth_header) {
    custom_auth.verify(&mut io).await?;
    // Extract public key from custom_auth
    return Ok(SuccessfulAuthentication::new(public_key));
}
// Fall back to default ClientAuth

```

3. Return a `SuccessfulAuthentication` containing the extracted public key.

## Method 2: Implement Custom AccessControl

Implement fine-grained authorization policies without changing the cryptographic handshake. This method validates JWTs, queries databases, or checks allow-lists after the client proves possession of their private key.

The `AccessControl` trait resides in [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs) (lines 285-306):

```rust
pub trait AccessControl: Send + Sync + 'static {
    fn authorize(
        &self,
        client_key: &PublicKey,
        request: &ClientRequest,
    ) -> Result<(), Error>;
}

```

**Implementation example**:

```rust
use iroh_base::key::PublicKey;
use iroh_relay::server::{AccessControl, ClientRequest, Error};
use std::collections::HashSet;

#[derive(Clone)]
struct DatabaseAccessControl {
    allowed_keys: HashSet<PublicKey>,
    // Add DB pool or API client here
}

impl AccessControl for DatabaseAccessControl {
    fn authorize(
        &self,
        client_key: &PublicKey,
        request: &ClientRequest,
    ) -> Result<(), Error> {
        // Check if key exists in organization database
        if !self.allowed_keys.contains(client_key) {
            return Err(Error::AccessDenied("Key not in organization".into()));
        }
        
        // Additional checks: validate JWT in request.auth_token(), etc.
        Ok(())
    }
}

```

**Wire it into the server** using `ServerBuilder` (from [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs)):

```rust
use iroh_relay::server::ServerBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let allowed_keys = load_keys_from_database().await?;
    let access = Arc::new(DatabaseAccessControl { allowed_keys });
    
    let server = ServerBuilder::new()
        .with_access_control(access)  // Inject custom logic
        .bind(([0, 0, 0, 0], 443))
        .await?;
        
    server.run().await?;
    Ok(())
}

```

## Method 3: Static Token-Based Authentication (Shortcut)

For simple use cases requiring a shared secret across all clients, use the built-in static token support in [`iroh-relay/src/relay_map.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/relay_map.rs) (lines 266-277).

Configure the client with `RelayConfig::with_auth_token`:

```rust
use iroh_relay::relay_map::RelayConfig;

let config = RelayConfig::new(url)
    .with_auth_token("secret-token-123".to_string());

```

The server extracts this token via `Server::auth_token()` (lines 252-264 in [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs)). Combine this with a custom `AccessControl` implementation to validate the token against an external authority:

```rust
impl AccessControl for TokenValidator {
    fn authorize(&self, client_key: &PublicKey, request: &ClientRequest) -> Result<(), Error> {
        if let Some(token) = request.auth_token() {
            if validate_against_oauth(token)? {
                return Ok(());
            }
        }
        Err(Error::AccessDenied("Invalid token".into()))
    }
}

```

## Complete Minimal Example

This example demonstrates custom `AccessControl` with an in-memory allow-list:

```rust
use iroh_base::key::PublicKey;
use iroh_relay::server::{ServerBuilder, AccessControl, ClientRequest, Error};
use std::collections::HashSet;
use std::sync::Arc;

#[derive(Clone)]
struct AllowList {
    allowed: HashSet<PublicKey>,
}

impl AccessControl for AllowList {
    fn authorize(&self, client_key: &PublicKey, _req: &ClientRequest) -> Result<(), Error> {
        if self.allowed.contains(client_key) {
            Ok(())
        } else {
            Err(Error::AccessDenied("unknown client".into()))
        }
    }
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let mut allowed = HashSet::new();
    allowed.insert(PublicKey::from_hex("0123…")?);
    
    let access = Arc::new(AllowList { allowed });
    let server = ServerBuilder::new()
        .with_access_control(access)
        .bind(([0, 0, 0, 0], 0))
        .await?;
        
    server.run().await?;
    Ok(())
}

```

## Summary

- **Handshake extension**: Modify [`iroh-relay/src/protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/protos/handshake.rs) to replace `ClientAuth` with custom challenge-response mechanisms or certificate validation.
- **AccessControl traits**: Implement the `AccessControl` trait from [`iroh-relay/src/server.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/server.rs) to enforce policies after cryptographic authentication, rejecting clients based on external databases, OAuth tokens, or IP restrictions.
- **Static tokens**: Use `RelayConfig::with_auth_token` for simple shared-secret scenarios, validating the token inside your `AccessControl` implementation.
- **Server construction**: Pass custom implementations via `ServerBuilder::with_access_control` to inject your logic into the relay server.

## Frequently Asked Questions

### What is the difference between authentication and authorization in Iroh?

Authentication verifies that a client possesses the private key corresponding to their public identity, typically through the challenge-response handshake in [`iroh-relay/src/protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/iroh-relay/src/protos/handshake.rs). Authorization occurs afterward via the `AccessControl` trait, determining whether that authenticated identity has permission to use the relay.

### Can I use JWT or OAuth tokens with Iroh's relay server?

Yes. Implement the `AccessControl` trait and extract the token from `ClientRequest::auth_token()` within the `authorize` method. You can then validate the JWT against your identity provider or OAuth server before returning `Ok(())` or `Error::AccessDenied`.

### Do I need to modify client code when implementing custom server authentication?

If you extend the handshake protocol in [`protos/handshake.rs`](https://github.com/n0-computer/iroh/blob/main/protos/handshake.rs), you must update the client to send your custom authentication header. If you only implement custom `AccessControl` or use static tokens via `RelayConfig`, standard Iroh clients work without modification.

### Where should I store allowed public keys for my AccessControl implementation?

Store them in any external system accessible from your `AccessControl` implementation, such as a PostgreSQL database, Redis cache, or LDAP directory. Initialize your struct with connection pools in `main()` and pass it to `ServerBuilder::with_access_control`.