# How the Buzz Relay Implements Its Three-Tier Subscription Fan-Out System

> Discover the Buzz relay's three-tier subscription fan-out system. Learn how it uses WebSockets, Redis Pub/Sub, and async side effects for scalable, low-latency event delivery across distributed instances.

- Repository: [Block Open Source/buzz](https://github.com/block/buzz)
- Tags: architecture
- Published: 2026-08-29

---

**The Buzz relay achieves scalable real-time event delivery through a three-tier subscription fan-out system that combines in-memory WebSocket pushes, Redis Pub/Sub cluster propagation, and asynchronous post-persistence side effects to ensure deterministic, low-latency distribution across distributed relay instances.**

The [block/buzz](https://github.com/block/buzz) Nostr relay must deliver every new event to all interested subscribers while maintaining horizontal scalability across many relay processes and isolated communities. Its **three-tier subscription fan-out system** solves this by layering immediate in-memory delivery for live connections, cluster-wide synchronization via Redis, and guaranteed asynchronous processing for indexing and audit trails.

## Tier 1: In-Memory WebSocket Fan-Out

The first tier provides **O(1)** deterministic delivery to live WebSocket connections. When an event is persisted, the relay immediately pushes it to all matching local subscribers before attempting any network overhead.

### SubscriptionRegistry and fan_out Implementation

At the core of this tier sits the `SubscriptionRegistry` in [`crates/buzz-relay/src/subscription.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/subscription.rs). The registry maintains per-connection indexes that map filters to connection identifiers, enabling constant-time lookups during the fan-out process.

When an event enters the system, the registry's `fan_out` or `fan_out_scoped` methods enumerate all matching `(ConnId, SubId)` pairs—tuples representing the connection ID and subscription ID—then write the event directly to each corresponding WebSocket stream. This ensures that active clients receive events with minimal latency, independent of cluster size or Redis latency.

```rust
// Conceptual usage within the relay
let matches = registry.fan_out(&event);
// matches contains (ConnId, SubId) pairs for immediate delivery

```

The implementation at lines 379-426 handles the filter matching logic, ensuring that only subscribers whose filters intersect with the event's metadata receive the payload.

## Tier 2: Redis Pub/Sub Cluster Propagation

The second tier solves horizontal scaling by propagating events to **all relay instances** that may hold connections for the same community or channel.

### Community-Scoped Channel Naming

After completing the in-memory push (Tier 1), the relay publishes a lightweight "fan-out" message to Redis using a hierarchical channel naming convention:

- `buzz:{community}:channel:{uuid}` for channel-specific events
- `buzz:{community}:global` for events with no specific channel target

Other relay processes subscribe to these Redis keys and execute the same in-memory fan-out logic on their local `SubscriptionRegistry` instances. This design ensures that a client connected to Relay A receives events posted via Relay B, provided both relays serve the same community.

The Redis publishing logic appears at lines 401-426 in [`crates/buzz-relay/src/subscription.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/subscription.rs), where the relay determines whether to scope the message to a specific channel or broadcast it globally within the community boundary.

## Tier 3: Post-Persist Side Effects

The third tier handles **guaranteed asynchronous processing** that must occur after live delivery but regardless of client connectivity status.

### WorkflowSink and dispatch_persistent_event

Once the event is persisted and the live fan-out completes, the `WorkflowSink` (located in [`crates/buzz-relay/src/workflow_sink.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/workflow_sink.rs)) delegates post-persistence work to `dispatch_persistent_event`. This helper triggers:

- **Search indexing**: Updating PostgreSQL full-text search indexes
- **Audit-log creation**: Appending to cryptographic hash-chains for compliance
- **Downstream pipelines**: Media storage processing and webhook triggers

Notably, the Redis fan-out (Tier 2) itself is invoked as part of these side effects, ensuring that cluster propagation only occurs for successfully persisted events. This ordering guarantees that transient delivery failures on one relay do not propagate incomplete data to the cluster.

## Community Scoping and Security Boundaries

Each tier of the fan-out system enforces **community isolation**. The `fan_out_scoped` method and Redis channel naming both incorporate `community_id`, ensuring events never cross community boundaries during distribution. This multi-tenant design prevents data leakage between isolated communities while allowing a single relay cluster to serve multiple organizations.

## Practical Examples: Triggering the Fan-Out Pipeline

### Publishing Events via CLI

You can trigger the complete three-tier pipeline using the Buzz CLI:

```bash

# Create a kind-1 text note in channel "general"

buzz messages post --channel $(buzz channels list | jq -r '.[] | select(.name=="general") | .uuid') \
  --content "Hello from the three-tier fan-out demo"

```

When this command executes, the relay performs the following sequence:
1. Persists the event to the database
2. Calls `SubscriptionRegistry::fan_out(&event)` for Tier 1 delivery
3. Publishes the fan-out message to the appropriate Redis channel for Tier 2 propagation
4. Executes `dispatch_persistent_event` for Tier 3 side effects (search indexing and audit logging)

### Verifying Fan-Out in Tests

The integration test suite demonstrates how to verify the fan-out mechanism:

```rust
let event = make_stored_event(Kind::TextNote, Some(channel_uuid));
let matches = registry.fan_out(&event);
assert!(!matches.is_empty(), "live fan-out delivered to at least one connection");

```

These tests reside in `crates/buzz-test-client/tests/e2e_*` and validate fan-out isolation, live delivery guarantees, and side-effect execution across the three tiers.

## Summary

- **Tier 1 (In-Memory)**: The `SubscriptionRegistry` provides O(1) WebSocket delivery to local subscribers via `fan_out` and `fan_out_scoped` methods in [`crates/buzz-relay/src/subscription.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/subscription.rs)
- **Tier 2 (Redis Pub/Sub)**: Cluster-wide propagation uses community-scoped Redis channels (`buzz:{community}:channel:{uuid}`) to synchronize events across relay instances
- **Tier 3 (Side Effects)**: The `WorkflowSink` delegates post-persistence work including search indexing and audit logging through `dispatch_persistent_event` in [`crates/buzz-relay/src/workflow_sink.rs`](https://github.com/block/buzz/blob/main/crates/buzz-relay/src/workflow_sink.rs)
- **Security Model**: All tiers respect `community_id` boundaries, ensuring strict isolation between communities in multi-tenant deployments

## Frequently Asked Questions

### How does the Buzz relay handle fan-out when running multiple relay instances?

The relay uses **Redis Pub/Sub** as the coordination mechanism between instances. When one relay receives an event, it publishes a fan-out message to a community-specific Redis channel after completing local in-memory delivery. All other relay instances subscribe to these channels and execute the same `SubscriptionRegistry::fan_out` logic on their local connections, ensuring clients connected to any instance receive the event.

### What happens if the Redis connection fails during Tier 2 fan-out?

The **Tier 1 in-memory fan-out** completes before Tier 2 begins, so local subscribers still receive the event immediately. However, cluster-wide propagation pauses for that specific event. Because Tier 3 post-persistence processing (including the Redis publish) is handled asynchronously by `WorkflowSink`, the system can retry the Redis publish or log the failure without blocking the initial WebSocket delivery to local clients.

### How does the three-tier system prevent events from leaking between communities?

Community isolation is enforced at every tier. The `SubscriptionRegistry` uses `fan_out_scoped` methods that filter by `community_id` during in-memory delivery. Redis channels are prefixed with the community identifier (`buzz:{community}...`), ensuring that relay instances only subscribe to channels relevant to their served communities. The post-persistence side effects also respect these boundaries when indexing or logging events.

### What is the latency impact of the three-tier pipeline?

**Tier 1** provides sub-millisecond delivery to local WebSocket connections due to O(1) registry lookups. **Tier 2** adds network latency equivalent to one Redis round-trip between relay instances, typically under 5ms in co-located deployments. **Tier 3** operates asynchronously and does not block the initial response to publishing clients, ensuring that the critical path for live event delivery remains deterministic and fast regardless of indexing or audit workload.