# WebSocket Connection Lifecycle and Cleanup Process for the Buzz Relay: A Deep Dive

> Explore the Buzz relay's WebSocket connection lifecycle and cleanup process. Discover its semaphore admission control, NIP-42 auth, concurrent Tokio loops, and cancellation token cleanup for reliable resource management.

- Repository: [Block Open Source/buzz](https://github.com/block/buzz)
- Tags: deep-dive
- Published: 2026-08-29

---

**The Buzz relay implements a robust WebSocket lifecycle using a semaphore-based admission controller, NIP-42 authentication challenges, three concurrent Tokio loops for send/receive/heartbeat operations, and cancellation token-driven cleanup that guarantees resource deallocation even under high load or abusive conditions.**

Every WebSocket connection in the Block Buzz relay follows a strictly defined lifecycle from admission to cleanup, designed to prevent resource exhaustion and ensure protocol compliance. This article examines the implementation in [`crates/buzz-relay/src/connection.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/connection.rs), explaining how the system manages concurrent connections, enforces NIP-42 authentication, handles back-pressure, and ensures graceful resource reclamation through coordinated task cancellation.

## Connection Admission and Rate Limiting

The lifecycle begins at `handle_connection`, the entry point defined in [`crates/buzz-relay/src/connection.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/connection.rs). Before any WebSocket upgrade occurs, the system attempts to acquire a slot from `state.conn_semaphore.try_acquire_owned()`. If the semaphore is exhausted, the connection is immediately rejected with a log entry, protecting the relay from overload.

Upon successful admission, the function generates a unique connection ID, creates a `CancellationToken` for coordinated shutdown, and increments the active connections gauge via `metrics::gauge!`. The connection is then registered with the global manager using `state.conn_manager.register`, enabling broadcast delivery to this specific client.

## NIP-42 Authentication Flow

Before accepting traffic, the relay enforces **NIP-42 authentication**. The `generate_challenge` function creates a random challenge string sent immediately to the client via the WebSocket. The connection enters an authentication window where it must respond within 5 seconds.

If authentication fails or times out, the connection aborts. The authentication state is tracked in `ConnectionState.auth` as an `RwLock<AuthState>` that transitions from `Pending` to `Authenticated` only after valid signature verification handled in [`crates/buzz-relay/src/handlers/auth.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/handlers/auth.rs).

## Concurrent Loop Architecture

Once authenticated, `handle_active_connection` splits the socket into `ws_send` and `ws_recv` halves and establishes three concurrent Tokio channels:

- **`send_tx/rx`** – Buffered data frames sized by `state.config.send_buffer_size`
- **`ctrl_tx/rx`** – Small priority channel for Pong and Close control frames
- **`restart_tx/rx`** – One-shot channel for graceful restart signaling

Three async tasks are spawned to manage the connection:

### The Send Loop

The send loop drains the priority `ctrl_tx` channel before processing the main `send_tx` queue. It batches data frames up to `MAX_WS_SEND_BATCH` before flushing to the socket, ensuring efficient throughput while giving control frames immediate precedence. If the buffer exceeds `grace_limit`, the connection self-cancels to prevent memory exhaustion attacks.

### The Heartbeat Loop

A dedicated heartbeat task sends WebSocket Ping frames every 30 seconds. The loop maintains a missed-pong counter; after three consecutive missed Pongs, it triggers cancellation. This mechanism detects stale connections without relying on TCP timeouts alone.

### The Receive Loop

The receive loop in `recv_loop` continuously polls `ws_recv`, handling frame types as follows:

- **Text** – Parsed as NIP-29 client messages via `handle_text_message`
- **Binary** – UTF-8 decoded then processed as text
- **Ping** – Replied immediately via the priority `ctrl_tx` channel
- **Pong** – Resets the missed-pong counter in the heartbeat tracker
- **Close/Error** – Breaks the loop, initiating shutdown

## Graceful Shutdown and Resource Cleanup

When any loop encounters an error, receives a close frame, or detects a timeout, it fires the shared `CancellationToken` via `cancel.cancel()`. This signal propagates to all three loops, ensuring coordinated termination without resource leaks.

The cleanup sequence executes in strict order:

1. **Subscription removal** – Calls `state.sub_registry.remove_connection` to purge all client subscriptions
2. **Pub-sub topic release** – Releases global and per-channel pub-sub resources
3. **Manager deregistration** – Removes the connection from `state.conn_manager`
4. **Presence cleanup** – If this was the client's last connection, clears their presence entry
5. **Metrics and semaphore** – Decrements the active connections gauge and drops the semaphore permit

For operational maintenance, the `restart_tx` channel supports graceful restarts by broadcasting a 1012 "restart" close code, flushing pending data, and confirming shutdown via the oneshot sender.

## Code Examples

### Wiring the WebSocket Route in Axum

```rust
use axum::{
    extract::ws::{WebSocketUpgrade, WebSocket},
    routing::get,
    Router,
};
use std::sync::Arc;
use buzz_relay::state::AppState;
use buzz_relay::connection::handle_connection;

async fn ws_handler(
    ws: WebSocketUpgrade,
    state: Arc<AppState>,
    addr: std::net::SocketAddr,
    tenant: buzz_core::tenant::TenantContext,
) -> axum::response::Response {
    ws.on_upgrade(|socket| handle_connection(socket, state, addr, tenant))
}

let app = Router::new().route("/ws", get(ws_handler));

```

### Sending a NOTICE to a Specific Client

```rust
use buzz_relay::state::AppState;
use buzz_relay::connection::ConnectionState;
use buzz_relay::protocol::RelayMessage;

async fn send_notice(state: &Arc<AppState>, conn_id: uuid::Uuid, text: &str) {
    if let Some(conn) = state.conn_manager.get(conn_id) {
        let _ = conn.send(RelayMessage::notice(text));
    }
}

```

### Disconnecting All Clients for a Deleted Community

```rust
use buzz_relay::state::AppState;
use buzz_relay::state::CommunityDisconnectReason;

fn delete_community(state: &Arc<AppState>, community_id: uuid::Uuid) {
    state
        .community_connections
        .notify_all(community_id, CommunityDisconnectReason::CommunityDeleted);
}

```

## Summary

- **Admission control** uses a semaphore to hard-limit concurrent connections, rejecting excess load immediately at `handle_connection`.
- **NIP-42 authentication** requires a valid challenge response within 5 seconds, enforced before message processing begins.
- **Three concurrent loops** (send, heartbeat, receive) operate independently, with control frames receiving priority delivery through a dedicated channel.
- **Back-pressure protection** cancels connections that exceed `grace_limit` buffered messages, preventing OOM conditions.
- **Graceful cleanup** follows a five-step sequence using `CancellationToken` coordination, ensuring subscriptions, presence, and semaphore permits are always released.

## Frequently Asked Questions

### How does the Buzz relay prevent WebSocket connection overload?

The relay initializes a semaphore with a fixed number of permits. In [`crates/buzz-relay/src/connection.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/connection.rs), the `handle_connection` function attempts `state.conn_semaphore.try_acquire_owned()` before processing. If no permits remain, the function logs the rejection and returns immediately, preventing resource allocation for excess connections.

### What happens if a client fails to authenticate within the timeout window?

The relay spawns an authentication timeout task alongside the main loops. If the client's `AuthState` does not transition to `Authenticated` within 5 seconds, the task fires the `CancellationToken`, triggering the standard cleanup sequence and closing the socket. This prevents unauthenticated connections from consuming resources indefinitely.

### How does the relay handle back-pressure from slow consumers?

The send loop monitors channel depth against `state.config.send_buffer_size`. When buffered messages exceed the configured `grace_limit`, the connection self-cancels. This design protects the relay from memory exhaustion caused by clients that accept data slower than the broadcast rate.

### What is the difference between the `send_tx` and `ctrl_tx` channels?

The `send_tx` channel carries Nostr data messages with a configurable buffer size, while `ctrl_tx` is a small, high-priority channel reserved for WebSocket control frames like Pong and Close. The send loop always drains `ctrl_tx` before processing `send_tx`, ensuring timely protocol responses even when the data pipeline is saturated.