Building Real-Time Collaborative Editing with Yjs CRDT in Agent-Native

Agent-native implements real-time collaborative editing by layering the Yjs CRDT on top of server-side SQL persistence, using a client-side hook for document synchronization and a server-side LRU cache with optimistic concurrency control.

The BuilderIO/agent-native repository provides a production-ready collaborative editing stack that combines Yjs conflict-free replicated data types (CRDTs) with SQL storage. This architecture enables multiple users and AI agents to edit documents simultaneously without conflicts, leveraging leader election and visibility-aware polling to ensure consistent state across all clients.

Architecture Overview

The collaborative editing system in agent-native consists of four tightly-coupled layers working together to synchronize document state:

• Client Hook (packages/core/src/collab/client.ts): Creates and maintains a stable Y.Doc per document, synchronizes updates via HTTP polling or SSE, and manages cursor presence through the Yjs Awareness protocol.
• Server-Side Doc Manager (packages/core/src/collab/ydoc-manager.ts): LRU-caches Y.Doc instances, loads and saves binary Yjs state from the database, applies incoming updates, and performs CAS-based persistence with optional compaction.
• SQL Storage (packages/core/src/collab/storage.ts): Defines the _collab_docs table schema, stores base64-encoded Yjs updates with optimistic concurrency control via a version column.
• HTTP Routes (packages/core/src/collab/routes.ts): Exposes /state, /update, /text, and /search-replace endpoints used by the client hook, emitting change events for SSE polling.

Client-Side Collaboration with useCollaborativeDoc

The useCollaborativeDoc hook in packages/core/src/collab/client.ts builds a persistent Y.Doc for each docId. When a component mounts, the hook executes a strict synchronization sequence:

1. Creates a new Y.Doc() instance.
2. Loads initial state from the /state endpoint (or requests a delta if the client already possesses a state vector).
3. Applies server updates using Y.applyUpdate to merge remote changes.
4. Debounces local updates for approximately 80ms before POSTing to /update.
5. Synchronizes cursor presence using the Yjs Awareness protocol, transported via polling or SSE.

Stable Document Identity and Leader Election

Agent-native maintains document stability by ensuring the same Y.Doc object lives for the lifetime of the browser tab. This allows TipTap's Collaboration extension to bind once and remain connected throughout the session.

The system implements leader election to prevent duplicate full-document rewrites. Only the client with the lowest Yjs clientID (excluding the AI agent) applies complete document snapshots. The isReconcileLeadClient function in client.ts determines leadership by comparing client IDs, ensuring that when multiple clients rewrite the same region, the operation executes exactly once.

Visibility-Aware Polling

When a browser tab becomes hidden, the client automatically pauses polling and yields its leader role. This ensures the user-visible tab receives the latest changes while background tabs conserve resources and avoid contention.

Server-Side Document Management

The ydoc-manager.ts file maintains an in-memory LRU cache with a maximum capacity of 50 documents (MAX_CACHE = 50). On cache misses, the system loads the stored binary Yjs state via loadYDocState and reconstructs the document using Y.applyUpdate.

Update Processing and Persistence

When the server receives an update through the applyUpdate function, it follows a strict serialization pattern:

1. Acquires a per-document write lock using withDocWriteLock to serialize mutations.
2. Applies the binary update to the cached Y.Doc instance.
3. Persists the merged state via persistMergedState, which implements optimistic concurrency control.

The persistence layer reads the current version column only once, then attempts to save using trySaveYDocState. If the stored blob exceeds four times the size of a fresh encoding, the system triggers compaction to clean up tombstones. After five CAS (Compare-And-Swap) retry failures, the system falls back to an unconditional save.

Text and XML Utilities

The manager exports applyTextToYDoc for diffing plain text against Yjs documents (used by AI agents) and searchAndReplaceInYXml for performing XML-based search-replace operations while preserving cursor positions.

Persistent SQL Storage

The _collab_docs table defined in packages/core/src/collab/storage.ts provides the durability layer for both SQLite (development) and PostgreSQL (production) environments:

• doc_id: Primary key identifying the document.
• yjs_state: Base64-encoded binary Yjs update representing the document state.
• text_snapshot: Plain-text snapshot for quick preview without Yjs decoding.
• version: Integer used for optimistic concurrency checks in trySaveYDocState.

Both loadYDocRecord and trySaveYDocState manipulate the version column to guarantee that concurrent writers cannot overwrite each other's changes, effectively preventing lost updates.

HTTP API Endpoints

The collaborative editing routes exposed in packages/core/src/collab/routes.ts operate under the /_agent-native/collab/:docId/* namespace with a default payload limit of 2 MiB (DEFAULT_MAX_BYTES):

Method	Path	Purpose
GET	/state	Returns full Yjs state as base64; accepts optional stateVector query parameter to fetch deltas only.
POST	/update	Receives base64 Yjs updates from clients, applies them via applyUpdate.
POST	/text	Accepts raw text, diffs it against the current Yjs document, and applies minimal operations (used by AI agents).
POST	/search-replace	Performs search-and-replace within the ProseMirror XML fragment while preserving cursor positions.

All endpoints emit change events that trigger SSE notifications to connected clients.

End-to-End Integration Example

The following example demonstrates how to enable collaborative editing in a React component using the client hook, and how the server processes incoming updates:

// src/app/components/CollabEditor.tsx
import { useCollaborativeDoc } from "@agent-native/core/collab/client";
import { useEditor, EditorContent } from "@tiptap/react";
import Collaboration from "@tiptap/extension-collaboration";

export function CollabEditor({ docId }: { docId: string }) {
  const { ydoc, isLoading, isSynced, activeUsers } = useCollaborativeDoc({
    docId,
    pollInterval: 2000,
    pollIntervalWithSse: 12000,
    user: { name: "Alice", email: "alice@example.com", color: "" },
  });

  const editor = useEditor({
    extensions: [
      Collaboration.configure({ document: ydoc?.getXmlFragment("content") })
    ],
  });

  if (isLoading || !editor) return <p>Loading…</p>;

  return (
    <>
      <EditorContent editor={editor} />
      <p>Online: {activeUsers.map(u => u.name).join(", ")}</p>
    </>
  );
}

// Server-side update handler
import * as manager from "packages/core/src/collab/ydoc-manager.js";

export async function postCollabUpdate(event) {
  const docId = getRouterParam(event, "docId");
  const { update } = await readBody(event);
  const binary = base64ToUint8Array(update);
  
  // Merges the binary delta, persists to SQL, and notifies other clients
  await manager.applyUpdate(docId, binary, "client");
  return { ok: true };
}

Summary

• Agent-native uses Yjs CRDTs layered over SQL persistence to provide conflict-free real-time collaboration.
• The useCollaborativeDoc hook in packages/core/src/collab/client.ts manages stable Y.Doc instances, leader election, and debounced updates.
• Server-side LRU caching (MAX_CACHE = 50) and per-document write locks in ydoc-manager.ts ensure high performance under load.
• Optimistic concurrency control via the version column in _collab_docs prevents data loss during concurrent writes.
• The HTTP API supports standard CRDT operations plus AI-specific endpoints like /text for diff-based updates.

Frequently Asked Questions

How does agent-native handle concurrent edits from multiple users?

Agent-native uses Yjs CRDTs to automatically merge concurrent edits without conflicts. At the storage layer, the version column in _collab_docs implements optimistic concurrency control. When persistMergedState detects a version mismatch during save, it retries the CAS operation up to five times before falling back to an unconditional save, ensuring data consistency even under high contention.

What is the purpose of leader election in the client hook?

The isReconcileLeadClient function in packages/core/src/collab/client.ts identifies the client with the lowest Yjs clientID as the leader. This client is solely responsible for applying full-document snapshots, preventing duplicate insertions when multiple clients rewrite the same content region. When a tab becomes hidden, it yields leadership to ensure the active visible client maintains authority.

How does the server optimize storage for large collaborative documents?

The persistMergedState function in packages/core/src/collab/ydoc-manager.ts monitors the size of stored Yjs updates. When the stored blob grows larger than four times the size of a fresh encoding, the system triggers automatic compaction to remove tombstones and merge operations. This keeps the _collab_docs table efficient while preserving full edit history within the Yjs document.

Can AI agents edit documents collaboratively without disrupting user cursors?

Yes. The /text endpoint in packages/core/src/collab/routes.ts accepts raw text and uses applyTextToYDoc to diff the input against the current document state, applying only the minimal necessary changes. For more complex operations, the /search-replace endpoint performs XML-aware replacements that preserve cursor positions, ensuring AI edits integrate smoothly with human editing sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}