WebSocket Connection Lifecycle and Cleanup Process for the Buzz Relay: A Deep Dive

The Buzz relay implements a robust WebSocket lifecycle using a semaphore-based admission controller, NIP-42 authentication challenges, three concurrent Tokio loops for send/receive/heartbeat operations, and cancellation token-driven cleanup that guarantees resource deallocation even under high load or abusive conditions.

Every WebSocket connection in the Block Buzz relay follows a strictly defined lifecycle from admission to cleanup, designed to prevent resource exhaustion and ensure protocol compliance. This article examines the implementation in crates/buzz-relay/src/connection.rs, explaining how the system manages concurrent connections, enforces NIP-42 authentication, handles back-pressure, and ensures graceful resource reclamation through coordinated task cancellation.

Connection Admission and Rate Limiting

The lifecycle begins at handle_connection, the entry point defined in crates/buzz-relay/src/connection.rs. Before any WebSocket upgrade occurs, the system attempts to acquire a slot from state.conn_semaphore.try_acquire_owned(). If the semaphore is exhausted, the connection is immediately rejected with a log entry, protecting the relay from overload.

Upon successful admission, the function generates a unique connection ID, creates a CancellationToken for coordinated shutdown, and increments the active connections gauge via metrics::gauge!. The connection is then registered with the global manager using state.conn_manager.register, enabling broadcast delivery to this specific client.

NIP-42 Authentication Flow

Before accepting traffic, the relay enforces NIP-42 authentication. The generate_challenge function creates a random challenge string sent immediately to the client via the WebSocket. The connection enters an authentication window where it must respond within 5 seconds.

If authentication fails or times out, the connection aborts. The authentication state is tracked in ConnectionState.auth as an RwLock<AuthState> that transitions from Pending to Authenticated only after valid signature verification handled in crates/buzz-relay/src/handlers/auth.rs.

Concurrent Loop Architecture

Once authenticated, handle_active_connection splits the socket into ws_send and ws_recv halves and establishes three concurrent Tokio channels:

• send_tx/rx – Buffered data frames sized by state.config.send_buffer_size
• ctrl_tx/rx – Small priority channel for Pong and Close control frames
• restart_tx/rx – One-shot channel for graceful restart signaling

Three async tasks are spawned to manage the connection:

The Send Loop

The send loop drains the priority ctrl_tx channel before processing the main send_tx queue. It batches data frames up to MAX_WS_SEND_BATCH before flushing to the socket, ensuring efficient throughput while giving control frames immediate precedence. If the buffer exceeds grace_limit, the connection self-cancels to prevent memory exhaustion attacks.

The Heartbeat Loop

A dedicated heartbeat task sends WebSocket Ping frames every 30 seconds. The loop maintains a missed-pong counter; after three consecutive missed Pongs, it triggers cancellation. This mechanism detects stale connections without relying on TCP timeouts alone.

The Receive Loop

The receive loop in recv_loop continuously polls ws_recv, handling frame types as follows:

• Text – Parsed as NIP-29 client messages via handle_text_message
• Binary – UTF-8 decoded then processed as text
• Ping – Replied immediately via the priority ctrl_tx channel
• Pong – Resets the missed-pong counter in the heartbeat tracker
• Close/Error – Breaks the loop, initiating shutdown

Graceful Shutdown and Resource Cleanup

When any loop encounters an error, receives a close frame, or detects a timeout, it fires the shared CancellationToken via cancel.cancel(). This signal propagates to all three loops, ensuring coordinated termination without resource leaks.

The cleanup sequence executes in strict order:

1. Subscription removal – Calls state.sub_registry.remove_connection to purge all client subscriptions
2. Pub-sub topic release – Releases global and per-channel pub-sub resources
3. Manager deregistration – Removes the connection from state.conn_manager
4. Presence cleanup – If this was the client's last connection, clears their presence entry
5. Metrics and semaphore – Decrements the active connections gauge and drops the semaphore permit

For operational maintenance, the restart_tx channel supports graceful restarts by broadcasting a 1012 "restart" close code, flushing pending data, and confirming shutdown via the oneshot sender.

Code Examples

Wiring the WebSocket Route in Axum

use axum::{
    extract::ws::{WebSocketUpgrade, WebSocket},
    routing::get,
    Router,
};
use std::sync::Arc;
use buzz_relay::state::AppState;
use buzz_relay::connection::handle_connection;

async fn ws_handler(
    ws: WebSocketUpgrade,
    state: Arc<AppState>,
    addr: std::net::SocketAddr,
    tenant: buzz_core::tenant::TenantContext,
) -> axum::response::Response {
    ws.on_upgrade(|socket| handle_connection(socket, state, addr, tenant))
}

let app = Router::new().route("/ws", get(ws_handler));

Sending a NOTICE to a Specific Client

use buzz_relay::state::AppState;
use buzz_relay::connection::ConnectionState;
use buzz_relay::protocol::RelayMessage;

async fn send_notice(state: &Arc<AppState>, conn_id: uuid::Uuid, text: &str) {
    if let Some(conn) = state.conn_manager.get(conn_id) {
        let _ = conn.send(RelayMessage::notice(text));
    }
}

Disconnecting All Clients for a Deleted Community

use buzz_relay::state::AppState;
use buzz_relay::state::CommunityDisconnectReason;

fn delete_community(state: &Arc<AppState>, community_id: uuid::Uuid) {
    state
        .community_connections
        .notify_all(community_id, CommunityDisconnectReason::CommunityDeleted);
}

Summary

• Admission control uses a semaphore to hard-limit concurrent connections, rejecting excess load immediately at handle_connection.
• NIP-42 authentication requires a valid challenge response within 5 seconds, enforced before message processing begins.
• Three concurrent loops (send, heartbeat, receive) operate independently, with control frames receiving priority delivery through a dedicated channel.
• Back-pressure protection cancels connections that exceed grace_limit buffered messages, preventing OOM conditions.
• Graceful cleanup follows a five-step sequence using CancellationToken coordination, ensuring subscriptions, presence, and semaphore permits are always released.

Frequently Asked Questions

How does the Buzz relay prevent WebSocket connection overload?

The relay initializes a semaphore with a fixed number of permits. In crates/buzz-relay/src/connection.rs, the handle_connection function attempts state.conn_semaphore.try_acquire_owned() before processing. If no permits remain, the function logs the rejection and returns immediately, preventing resource allocation for excess connections.

What happens if a client fails to authenticate within the timeout window?

The relay spawns an authentication timeout task alongside the main loops. If the client's AuthState does not transition to Authenticated within 5 seconds, the task fires the CancellationToken, triggering the standard cleanup sequence and closing the socket. This prevents unauthenticated connections from consuming resources indefinitely.

How does the relay handle back-pressure from slow consumers?

The send loop monitors channel depth against state.config.send_buffer_size. When buffered messages exceed the configured grace_limit, the connection self-cancels. This design protects the relay from memory exhaustion caused by clients that accept data slower than the broadcast rate.

What is the difference between the send_tx and ctrl_tx channels?

The send_tx channel carries Nostr data messages with a configurable buffer size, while ctrl_tx is a small, high-priority channel reserved for WebSocket control frames like Pong and Close. The send loop always drains ctrl_tx before processing send_tx, ensuring timely protocol responses even when the data pipeline is saturated.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}