How the Buzz Relay Implements Its Three-Tier Subscription Fan-Out System

The Buzz relay achieves scalable real-time event delivery through a three-tier subscription fan-out system that combines in-memory WebSocket pushes, Redis Pub/Sub cluster propagation, and asynchronous post-persistence side effects to ensure deterministic, low-latency distribution across distributed relay instances.

The block/buzz Nostr relay must deliver every new event to all interested subscribers while maintaining horizontal scalability across many relay processes and isolated communities. Its three-tier subscription fan-out system solves this by layering immediate in-memory delivery for live connections, cluster-wide synchronization via Redis, and guaranteed asynchronous processing for indexing and audit trails.

Tier 1: In-Memory WebSocket Fan-Out

The first tier provides O(1) deterministic delivery to live WebSocket connections. When an event is persisted, the relay immediately pushes it to all matching local subscribers before attempting any network overhead.

SubscriptionRegistry and fan_out Implementation

At the core of this tier sits the SubscriptionRegistry in crates/buzz-relay/src/subscription.rs. The registry maintains per-connection indexes that map filters to connection identifiers, enabling constant-time lookups during the fan-out process.

When an event enters the system, the registry's fan_out or fan_out_scoped methods enumerate all matching (ConnId, SubId) pairs—tuples representing the connection ID and subscription ID—then write the event directly to each corresponding WebSocket stream. This ensures that active clients receive events with minimal latency, independent of cluster size or Redis latency.

// Conceptual usage within the relay
let matches = registry.fan_out(&event);
// matches contains (ConnId, SubId) pairs for immediate delivery

The implementation at lines 379-426 handles the filter matching logic, ensuring that only subscribers whose filters intersect with the event's metadata receive the payload.

Tier 2: Redis Pub/Sub Cluster Propagation

The second tier solves horizontal scaling by propagating events to all relay instances that may hold connections for the same community or channel.

Community-Scoped Channel Naming

After completing the in-memory push (Tier 1), the relay publishes a lightweight "fan-out" message to Redis using a hierarchical channel naming convention:

• buzz:{community}:channel:{uuid} for channel-specific events
• buzz:{community}:global for events with no specific channel target

Other relay processes subscribe to these Redis keys and execute the same in-memory fan-out logic on their local SubscriptionRegistry instances. This design ensures that a client connected to Relay A receives events posted via Relay B, provided both relays serve the same community.

The Redis publishing logic appears at lines 401-426 in crates/buzz-relay/src/subscription.rs, where the relay determines whether to scope the message to a specific channel or broadcast it globally within the community boundary.

Tier 3: Post-Persist Side Effects

The third tier handles guaranteed asynchronous processing that must occur after live delivery but regardless of client connectivity status.

WorkflowSink and dispatch_persistent_event

Once the event is persisted and the live fan-out completes, the WorkflowSink (located in crates/buzz-relay/src/workflow_sink.rs) delegates post-persistence work to dispatch_persistent_event. This helper triggers:

• Search indexing: Updating PostgreSQL full-text search indexes
• Audit-log creation: Appending to cryptographic hash-chains for compliance
• Downstream pipelines: Media storage processing and webhook triggers

Notably, the Redis fan-out (Tier 2) itself is invoked as part of these side effects, ensuring that cluster propagation only occurs for successfully persisted events. This ordering guarantees that transient delivery failures on one relay do not propagate incomplete data to the cluster.

Community Scoping and Security Boundaries

Each tier of the fan-out system enforces community isolation. The fan_out_scoped method and Redis channel naming both incorporate community_id, ensuring events never cross community boundaries during distribution. This multi-tenant design prevents data leakage between isolated communities while allowing a single relay cluster to serve multiple organizations.

Practical Examples: Triggering the Fan-Out Pipeline

Publishing Events via CLI

You can trigger the complete three-tier pipeline using the Buzz CLI:

# Create a kind-1 text note in channel "general"

buzz messages post --channel $(buzz channels list | jq -r '.[] | select(.name=="general") | .uuid') \
  --content "Hello from the three-tier fan-out demo"

When this command executes, the relay performs the following sequence:

1. Persists the event to the database
2. Calls SubscriptionRegistry::fan_out(&event) for Tier 1 delivery
3. Publishes the fan-out message to the appropriate Redis channel for Tier 2 propagation
4. Executes dispatch_persistent_event for Tier 3 side effects (search indexing and audit logging)

Verifying Fan-Out in Tests

The integration test suite demonstrates how to verify the fan-out mechanism:

let event = make_stored_event(Kind::TextNote, Some(channel_uuid));
let matches = registry.fan_out(&event);
assert!(!matches.is_empty(), "live fan-out delivered to at least one connection");

These tests reside in crates/buzz-test-client/tests/e2e_* and validate fan-out isolation, live delivery guarantees, and side-effect execution across the three tiers.

Summary

• Tier 1 (In-Memory): The SubscriptionRegistry provides O(1) WebSocket delivery to local subscribers via fan_out and fan_out_scoped methods in crates/buzz-relay/src/subscription.rs
• Tier 2 (Redis Pub/Sub): Cluster-wide propagation uses community-scoped Redis channels (buzz:{community}:channel:{uuid}) to synchronize events across relay instances
• Tier 3 (Side Effects): The WorkflowSink delegates post-persistence work including search indexing and audit logging through dispatch_persistent_event in crates/buzz-relay/src/workflow_sink.rs
• Security Model: All tiers respect community_id boundaries, ensuring strict isolation between communities in multi-tenant deployments

Frequently Asked Questions

How does the Buzz relay handle fan-out when running multiple relay instances?

The relay uses Redis Pub/Sub as the coordination mechanism between instances. When one relay receives an event, it publishes a fan-out message to a community-specific Redis channel after completing local in-memory delivery. All other relay instances subscribe to these channels and execute the same SubscriptionRegistry::fan_out logic on their local connections, ensuring clients connected to any instance receive the event.

What happens if the Redis connection fails during Tier 2 fan-out?

The Tier 1 in-memory fan-out completes before Tier 2 begins, so local subscribers still receive the event immediately. However, cluster-wide propagation pauses for that specific event. Because Tier 3 post-persistence processing (including the Redis publish) is handled asynchronously by WorkflowSink, the system can retry the Redis publish or log the failure without blocking the initial WebSocket delivery to local clients.

How does the three-tier system prevent events from leaking between communities?

Community isolation is enforced at every tier. The SubscriptionRegistry uses fan_out_scoped methods that filter by community_id during in-memory delivery. Redis channels are prefixed with the community identifier (buzz:{community}...), ensuring that relay instances only subscribe to channels relevant to their served communities. The post-persistence side effects also respect these boundaries when indexing or logging events.

What is the latency impact of the three-tier pipeline?

Tier 1 provides sub-millisecond delivery to local WebSocket connections due to O(1) registry lookups. Tier 2 adds network latency equivalent to one Redis round-trip between relay instances, typically under 5ms in co-located deployments. Tier 3 operates asynchronously and does not block the initial response to publishing clients, ensuring that the critical path for live event delivery remains deterministic and fast regardless of indexing or audit workload.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}