How Cloudflare OS Handles User Data Persistence Using Typed-Storage

Cloudflare OS provides a type-safe persistence layer called Typed-Storage that wraps the Durable Object KV store with schema-driven collections, automatic indexing, atomic transactions, and change notifications.

Cloudflare OS implements a robust edge data persistence mechanism through its Typed-Storage package, offering developers a strongly-typed alternative to raw KV operations. This system builds directly on top of Durable Object storage in the cloudflare/cloudflare-os repository, adding structured schema definitions, transactional guarantees, and real-time subscriptions. Understanding how Cloudflare OS handles user data persistence using typed-storage reveals a sophisticated approach to managing structured data at the edge.

Core Architecture of Typed-Storage

Typed-Storage abstracts the underlying Durable Object KV store into a schema-driven API that enforces type safety at compile time while maintaining runtime consistency through transactions.

Schema Definition with the collection() Helper

Developers declare data models using the collection<T>() helper function defined in packages/typed-storage/src/index.ts at lines 77-89. This function accepts a configuration object specifying the primary key, optional unique indexes, and non-unique indexes.

const User = collection<{ id: string; email: string; tags: string[] }>()({
  primaryKey: "id",
  uniqueIndexes: { byEmail: (r) => r.email },
  nonUniqueIndexes: { byTag: (r) => r.tags },
});

The schema definition generates a Collection<T> interface (lines 23-32) that exposes get, list, put, delete, and subscription methods. For single-value storage, the Singleton<T> interface (lines 38-44) provides a simplified API for configuration data or global state.

The TypedStorage Interface and createTypedStorage

The TypedStorage type (lines 46-49) exposes a transaction() method for atomic operations and dynamically injects properties for each declared collection or singleton. You instantiate storage by calling createTypedStorage() with the Durable Object's storage context and your schema definition:

const storage = createTypedStorage(this.ctx.storage, {
  collections: { users: User },
});

This factory function returns a typed interface where storage.users provides full CRUD capabilities backed by the Durable Object's transactional KV store.

Persistence Mechanisms and Data Flow

Typed-Storage handles user data persistence through three coordinated mechanisms: prefixed KV storage, atomic transactions, and automatic index maintenance.

KV Prefixed Views and Key Structure

All data resides in a KvPrefixedView<T> class (lines 23-34) that transparently prefixes every key with the collection name. This isolation prevents key collisions between collections while maintaining the flat KV structure required by the underlying storage. The implementation encodes numeric values to ensure lexical ordering is preserved when iterating keys.

When you call storage.users.put(record), the system writes to a key formatted as users:${id}, ensuring collection-scoped storage without manual key management.

Atomic Transactions with transactionSync

Mutating operations wrap all writes, deletions, and index updates inside storage.transactionSync() blocks. According to the implementation in lines 66-84 and 90-104, this guarantees that a failure during index rebuilding rolls back the entire operation, preventing partial writes that would corrupt the unique or non-unique indexes.

The transaction boundary ensures that when you update a record, both the primary data write and all associated index updates succeed or fail atomically.

Automatic Index Maintenance through Subscriptions

Typed-Storage implements UniqueIndex and NonUniqueIndex definitions (lines 59-75) that automatically track record fields. When a record changes, the collection's internal subscriber logic (implemented around lines 224-260) invokes addIndexSubscriber to update indexes via the Subscriber<T> interface (lines 14-22).

Unique indexes enforce constraints by mapping a field value to exactly one primary key, while non-unique indexes maintain sets of primary keys for each field value, enabling efficient many-to-many lookups without full table scans.

Change Notifications and Real-Time Updates

Collections and singletons maintain sets of Subscriber objects that receive add, update, or remove callbacks when records mutate. This subscription model allows reactive UI updates or cascading business logic without polling the KV store.

The notification system operates within the same transaction boundary as the data modifications, ensuring subscribers receive events only for successfully committed changes.

Practical Implementation Example

The following example demonstrates the complete persistence flow in a Durable Object:

import { collection, createTypedStorage } from "@cloudflare/typed-storage";

// Define schema with primary key and indexes
const User = collection<{ id: string; email: string; tags: string[] }>()({
  primaryKey: "id",
  uniqueIndexes: { byEmail: (r) => r.email },
  nonUniqueIndexes: { byTag: (r) => r.tags },
});

export class UserDO {
  #storage = createTypedStorage(this.ctx.storage, {
    collections: { users: User },
  });

  // Atomic write with automatic index updates
  async addUser(user: { id: string; email: string; tags: string[] }) {
    await this.#storage.users.put(user);
  }

  // Primary key lookup
  async getUser(id: string) {
    return this.#storage.users.get(id);
  }

  // Query via non-unique index
  async usersWithTag(tag: string) {
    const results = [];
    for (const rec of this.#storage.users.byTag.get(tag)) {
      results.push(rec);
    }
    return results;
  }
}

This implementation mirrors the createCollection logic (lines 335-445) and index subscriber handling (lines 224-260) found in packages/typed-storage/src/index.ts.

Summary

• Schema-First API: Typed-Storage uses the collection() helper in packages/typed-storage/src/index.ts to define typed collections with primary keys and indexes.
• Atomic Consistency: All writes execute within storage.transactionSync() blocks, ensuring that data and index updates remain consistent even during failures.
• Transparent Prefixing: The KvPrefixedView class automatically scopes keys by collection name, providing isolation without manual key management.
• Automatic Indexing: Unique and non-unique indexes update automatically via the subscriber pattern, maintaining query performance without manual intervention.
• Type Safety: The entire API is generic over TypeScript types, providing compile-time guarantees for data stored in the Durable Object KV backend.

Frequently Asked Questions

What is the difference between Collections and Singletons in Typed-Storage?

Collections map multiple records keyed by a primary key and support list operations, while Singletons store exactly one value for configuration or global state. According to the source code in packages/typed-storage/src/index.ts (lines 23-32 for Collections, lines 38-44 for Singletons), Collections provide get, list, put, and delete methods, whereas Singletons offer only get and put with simpler subscription hooks.

How does Typed-Storage ensure data consistency during concurrent writes?

Typed-Storage wraps all mutating operations inside storage.transactionSync() as implemented in lines 66-84 and 90-104 of the main source file. This transactional boundary ensures that when you update a record, the primary write and all associated index modifications succeed or fail atomically, preventing partial updates that would leave indexes out of sync with the data.

Can I use Typed-Storage outside of Durable Objects?

No. Typed-Storage is specifically designed to operate within Cloudflare Durable Objects, as evidenced by the createTypedStorage function requiring this.ctx.storage (the Durable Object storage API) as its first parameter. The library depends on the transactional guarantees and KV storage interface provided exclusively by the Durable Object environment.

How are indexes maintained when records are updated?

When a record is added, updated, or removed, the collection's internal Subscriber logic (lines 14-22) triggers addIndexSubscriber functions that rebuild the affected indexes. This happens automatically within the same transaction as the data modification, as shown in the index subscriber implementation around lines 224-260. Unique indexes validate constraints before writing, while non-unique indexes maintain sets of primary keys for efficient lookups.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}