How AFFiNE Handles Real-Time Collaboration and Data Synchronization: Yjs CRDT and WebSocket Architecture

AFFiNE enables real-time collaboration by combining Yjs CRDTs for conflict-free document merging with Socket.IO WebSocket gateways for low-latency update distribution, supported by local IndexedDB/SQLite persistence for offline functionality.

AFFiNE is an open-source knowledge base that unifies docs and whiteboards in a collaborative workspace. To power its real-time collaboration and data synchronization capabilities across web, desktop, and mobile clients, the platform implements a layered architecture centered on operational CRDTs rather than traditional operational transform algorithms. This design ensures that concurrent edits from multiple users converge to identical document states without manual conflict resolution, even during network partitions or client reconnections.

Core Architecture Components

CRDT Engine and Document State

At the heart of AFFiNE's collaboration system lies Yjs, a proven CRDT (Conflict-free Replicated Data Type) library that represents documents as immutable data structures. When a user opens a workspace, the client instantiates a YDoc object that maintains the authoritative state of the document.

According to the source code in packages/frontend/core/src/modules/workspace-engine/impls/local.ts, the workspace engine creates a fresh CRDT document using:

import { Doc as YDoc, encodeStateAsUpdate } from 'yjs';

const ydoc = new YDoc();                     // create a fresh CRDT document
const stateUpdate = encodeStateAsUpdate(ydoc); // binary delta to sync

The encodeStateAsUpdate function generates binary deltas that represent incremental changes, allowing the system to transmit only modified bytes rather than full document snapshots. This approach minimizes bandwidth usage and enables efficient synchronization.

Local Persistence Layer

Before any network transmission occurs, AFFiNE persists document state locally to support offline editing. The platform uses different storage backends depending on the environment:

• Web clients: Store snapshots in IndexedDB via IndexedDBDocStorage and IndexedDBBlobStorage
• Electron desktop: Utilizes SQLite through SqliteDocStorage for durable local storage

Additionally, the system employs BroadcastChannel('affine-local-workspace-changed') to notify other browser tabs when a local workspace changes, ensuring cross-tab synchronization without server round-trips.

WebSocket Sync Gateway

The server-side synchronization logic resides in packages/backend/server/src/core/sync/gateway.ts, implemented as a NestJS SpaceSyncGateway. This gateway exposes Socket.IO endpoints that handle protocol negotiation, space membership validation, and update broadcasting.

The gateway supports protocol versioning to maintain backward compatibility. Clients identify themselves with a clientVersion parameter, allowing older clients to fall back to the sync-025 protocol while newer clients use sync-026. This versioning ensures smooth migrations without breaking existing collaboration sessions.

Awareness and Presence Tracking

Beyond document content, AFFiNE synchronizes user presence information—cursor positions, selections, and user names—through the Awareness protocol. Implemented in blocksuite/playground/apps/_common/sync/websocket/awareness.ts, the WebSocketAwarenessSource class forwards awareness updates through the same WebSocket channel used for document synchronization:

export class WebSocketAwarenessSource implements AwarenessSource {
  constructor(readonly ws: WebSocket) {}
  
  send(update: Uint8Array) {
    this.ws.send(JSON.stringify({ type: 'awareness', payload: update }));
  }
}

Data Synchronization Flow

The end-to-end synchronization process follows a precise sequence that guarantees eventual consistency:

1. Document Initialization: When opening a workspace, the client either creates a new YDoc or loads a persisted snapshot from IndexedDB/SQLite.

2. Local Edit Generation: User inputs trigger Yjs to produce binary updates via encodeStateAsUpdate, capturing only the differential changes.

3. Worker Transmission: The out-worker.ts implementation (located in packages/frontend/core/src/modules/workspace-engine/impls/out-worker.ts) forwards these updates to the server through a Socket.IO connection authenticated with user tokens.

4. Server Processing: The SpaceSyncGateway receives the pushDocUpdate event, validates the client's permissions, and merges the incoming update with the stored document using Yjs's native merge algorithms.

5. Broadcast Distribution: The gateway rebroadcasts the merged update to all connected clients in the specific workspace room, using the appropriate protocol room type based on the client's version.

6. Client Application: Remote clients receive the broadcastDocUpdate event and apply the changes instantly using ydoc.applyUpdate, rendering the merged state without page refreshes.

Because Yjs CRDTs are operation-based and commutative, the system tolerates out-of-order delivery and network partitions without requiring complex locking mechanisms.

Implementation Examples

Creating a Collaborative Workspace

To initialize a new collaborative workspace on the client side, the application uses the Workspace Engine with explicit Yjs document attachment:

import { WorkspaceEngine } from '@affine/workspace-engine';
import { Doc as YDoc } from 'yjs';

// Initialise the engine (local storage provider already configured)
const engine = new WorkspaceEngine();

// Create a fresh workspace
const workspace = await engine.createWorkspace({
  flavour: 'local',
  name: 'My Collaboration Space',
});

// Attach a Y-document
const ydoc = new YDoc();
workspace.attachDoc(ydoc);

Transmitting Updates to the Server

When local changes occur, the client encodes and transmits updates through the out-worker:

// packages/frontend/core/src/modules/workspace-engine/impls/out-worker.ts
import { io } from 'socket.io-client';

const socket = io(`${process.env.AFFINE_WS_URL}/ws`, {
  transports: ['websocket'],
  auth: { token: userToken },
});

// Join a workspace room with version negotiation
socket.emit('joinSpace', {
  spaceType: 'workspace',
  spaceId,
  clientVersion: AFFiNE_VERSION,
});

// Push an encoded update
socket.emit('pushDocUpdate', {
  spaceType: 'workspace',
  spaceId,
  docId,
  update: Buffer.from(update).toString('base64'),
});

Server-Side Update Handling

The NestJS gateway processes incoming updates and manages room broadcasting:

// packages/backend/server/src/core/sync/gateway.ts
@WebSocketGateway()
export class SpaceSyncGateway {
  @SubscribeMessage('pushDocUpdate')
  async handlePush(
    @ConnectedSocket() client: Socket,
    @MessageBody() msg: PushDocUpdateMessage,
  ) {
    const doc = await this.docReader.read(msg.spaceId, msg.docId);
    const update = Buffer.from(msg.update, 'base64');
    const merged = mergeUpdatesInApplyWay(doc, update);
    
    // Broadcast to peers in the same space
    client.to(Room(msg.spaceId, this.getSyncProtocolRoomType(msg.clientVersion)))
          .emit('broadcastDocUpdate', {...msg, update: merged});
  }
}

Applying Remote Updates

On the receiving end, clients apply incoming binary updates to their local Yjs document:

// Inside out-worker.ts
socket.on('broadcastDocUpdate', async msg => {
  const { update } = msg;
  const binary = Buffer.from(update, 'base64');
  ydoc.applyUpdate(binary);   // merge remote changes instantly
});

Summary

• Yjs CRDTs provide the mathematical foundation for conflict-free merging of concurrent edits in AFFiNE, eliminating the need for operational transform complexity.
• Socket.IO WebSocket gateways enable sub-second latency for update distribution, with the SpaceSyncGateway in packages/backend/server/src/core/sync/gateway.ts handling room management and protocol versioning.
• Dual storage strategy combines IndexedDB (web) and SQLite (desktop) for offline-capable local persistence, synchronized via local.ts and cloud.ts implementations.
• Awareness protocol transmits cursor positions and user presence through the same WebSocket channel used for document data.
• Backward compatibility is maintained through explicit protocol version checking (sync-025 vs sync-026), allowing heterogeneous client versions to collaborate within the same workspace.

Frequently Asked Questions

What CRDT library does AFFiNE use for real-time collaboration?

AFFiNE uses Yjs, a battle-tested CRDT library that provides conflict-free data structures including maps, arrays, and text types. The codebase imports Doc as YDoc and encodeStateAsUpdate from the yjs package to create documents and generate incremental binary updates. This choice enables operational-transform-free synchronization that tolerates out-of-order message delivery.

How does AFFiNE handle offline editing and data synchronization?

When offline, AFFiNE stores document snapshots in IndexedDB (browser environments) or SQLite (Electron desktop) via the IndexedDBDocStorage and SqliteDocStorage classes. Local edits generate Yjs updates that queue for transmission. Upon reconnection, the out-worker.ts synchronizes the accumulated changes with the cloud backend, merging updates using the CRDT's automatic convergence properties to ensure consistency.

What is the role of the SpaceSyncGateway in AFFiNE's architecture?

The SpaceSyncGateway (defined in packages/backend/server/src/core/sync/gateway.ts) acts as the central WebSocket hub for real-time collaboration. It handles client authentication, validates space membership permissions, merges incoming document updates using Yjs algorithms, and broadcasts changes to all connected clients in a workspace room. It also manages protocol version negotiation to support backward compatibility.

How does AFFiNE manage user presence and cursor positions?

User presence data flows through the Awareness subsystem, implemented in blocksuite/playground/apps/_common/sync/websocket/awareness.ts. The WebSocketAwarenessSource class transmits cursor positions, text selections, and user identification metadata through the same Socket.IO connection used for document synchronization. This ensures that remote cursors and selections appear instantly to collaborators without interfering with document content updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}