How denoland/celld Handles Data: SQLite-Backed Durable Object Storage Explained

celld stores all Durable Object data in dedicated per-cell SQLite databases, with prepared statement caching, thread-local isolation, and LRU eviction to deliver fast, consistent KV operations.

The denoland/celld runtime implements Deno Land's Durable Object (DO) storage API using a Rust core that maps JavaScript async calls to synchronous SQLite operations. Every cell receives its own isolated database file, while the runtime manages connection pooling, statement caching, and cache eviction to maintain performance under load.

Cell-Scoped Identity and Safe Path Handling

Each Durable Object instance in celld is identified by a scope string in the format Class:instance. Before any storage operation, celld validates this scope to prevent directory traversal attacks and ensure safe filesystem paths.

In crates/logic/cell.rs, the valid_cell_scope function enforces these constraints:

• Maximum length of 400 bytes
• Forbidden characters that could escape the data directory
• Guaranteed valid UTF-8

// From cell.rs – scope validation (lines 43-51)
// Returns true if the scope is safe to use as a path component
pub fn valid_cell_scope(scope: &str) -> bool {
    // Implementation rejects paths containing "..", "//", or control characters
    // and enforces the MAX_SCOPE_LENGTH constant (400)
}

This validation ensures that even malicious cell identifiers cannot read or write outside their designated storage directory.

Per-Cell SQLite Database Architecture

Every active cell maintains an exclusive SQLite connection that is opened on activation and closed on eviction. The Cells struct in crates/celld/storage.rs (lines 34-50) tracks all state for currently open cells:

// Conceptual structure from storage.rs
pub struct Cells {
    open: HashMap<String, OpenCell>,   // scope -> database connection
    alarms: AlarmMap,                   // scheduled timers per cell
    cursors: CursorMap,                 // streaming result handles
    stmt_cache: StmtCache,              // shared prepared statements
}

The OpenCell struct (lines 52-60) pairs this SQLite connection with critical metadata:

pub struct OpenCell {
    pub connection: Connection,         // rusqlite handle
    pub scope: String,
    pub epoch: u64,                     // ownership epoch for fencing
}

The epoch fencing mechanism prevents race conditions during cell migration: if a cell's ownership transfers to another node, the epoch increments, and any stale OpenCell with a mismatched epoch is rejected.

KV Operations via Precompiled SQL Statements

The DO storage API (ctx.storage.get, put, delete) compiles to three static SQL statements defined in crates/celld/storage.rs (lines 91-99):

// Prepared once per process, reused for all KV operations
const KV_GET_SQL: &str = "SELECT value FROM kv WHERE scope = ?1 AND key = ?2";
const KV_PUT_SQL: &str = "INSERT INTO kv(scope, key, value) VALUES(?1, ?2, ?3) \
                          ON CONFLICT(scope, key) DO UPDATE SET value = excluded.value";
const KV_DELETE_SQL: &str = "DELETE FROM kv WHERE scope = ?1 AND key = ?2";

These statements are prepared using SQLite's cached statement API, eliminating per-request parse and plan overhead. Benchmarks on similar workloads show 2-5x throughput improvement versus unprepared queries.

JavaScript Usage Example

// examples/kv/index.js – Durable Object handler
export default class Counter {
  async fetch(request) {
    const key = "count";
    
    // Async JS API maps to synchronous SQLite on the Rust side
    const value = await this.storage.get(key);
    const count = value ? Number(value) : 0;
    
    await this.storage.put(key, (count + 1).toString());
    return new Response(`Count is now ${count + 1}`);
  }
}

The JavaScript async/await surface preserves compatibility with the Cloudflare Durable Objects API, while the Rust implementation executes blocking SQLite calls without thread hops.

Direct Rust Storage Access

For internal operations or native extensions, the storage layer exposes synchronous functions:

use celld::storage::{dbs, KV_GET_SQL};

// Synchronous read—usable from within a running turn
fn read_counter(scope: &str) -> Option<i64> {
    dbs(|dbs| {
        let cell = dbs.borrow().get(scope)?;
        let conn = &cell.connection;
        
        conn.prepare_cached(KV_GET_SQL)
            .ok()?
            .query_row([scope, "count"], |row| row.get::<_, i64>(0))
            .optional()
            .ok()
            .flatten()
    })
}

The dbs helper wraps thread-local access to the current isolate's Cells instance.

Thread-Local Isolation and Turn-Based Access

celld uses a thread-local pointer pattern to guarantee single-threaded access to cell state. In crates/celld/storage.rs (lines 14-22):

thread_local! {
    static CURRENT_CELLS: RefCell<Option<Rc<RefCell<Cells>>>> = const { RefCell::new(None) };
}

Before executing any JavaScript turn, the runtime:

1. Sets CURRENT_CELLS to the isolate's Cells instance
2. Runs the turn (all storage operations resolve to the thread-local)
3. Clears CURRENT_CELLS before the next turn begins

This design eliminates cross-thread synchronization overhead while maintaining Rust's borrow-checker guarantees. No two turns access the same Cells maps concurrently.

Snapshot Cache and LRU Eviction

To reduce redundant database round-trips, celld maintains a snapshot cache of recent cell state. When memory pressure increases, plan_eviction in crates/logic/cache.rs (lines 26-48) selects entries to drop:

// From cache.rs – LRU eviction with byte-limit awareness
pub fn plan_eviction(cache: &Cache, target_bytes: usize) -> Vec<CacheKey> {
    // Sorts entries by last_access time, oldest first
    // Accumulates entries until target_bytes would be freed
    // Returns keys to evict; their database connections are closed
}

The eviction policy prioritizes oldest-accessed, largest-footprint entries, keeping hot data resident while bounding memory consumption.

Consistency Guarantees and Epoch Fencing

The combination of SQLite, prepared statements, and epoch tracking provides strong per-key atomicity with session consistency. Operations within a single turn observe their own writes immediately. Across turns, celld guarantees:

• Linearizable writes for a given cell (single writer per epoch)
• Read-your-writes within a turn
• Automatic fencing preventing stale reads after ownership transfer

The epoch field in OpenCell is checked before every operation; a mismatch triggers automatic re-activation with the current epoch.

Summary

denoland/celld handles Durable Object data through a layered architecture optimized for latency and correctness:

• Per-cell SQLite files provide durable, isolated storage with full SQL expressiveness
• Prepared statement caching at KV_GET_SQL, KV_PUT_SQL, KV_DELETE_SQL eliminates parse overhead
• Thread-local CURRENT_CELLS enables lock-free, synchronous Rust core operation
• Epoch fencing in OpenCell prevents split-brain during cell migration
• LRU snapshot cache with plan_eviction bounds memory while retaining hot data

The JavaScript async API wraps this synchronous core, matching the Cloudflare Durable Objects contract without cross-thread overhead.

Frequently Asked Questions

What database does celld use for Durable Object storage?

celld uses SQLite as its sole storage engine. Each Durable Object cell receives a dedicated .sqlite file on the node's local filesystem, created in a subdirectory determined by the validated cell scope. This design prioritizes operational simplicity and single-node performance over distributed replication.

How does celld prevent data corruption during cell migration?

The epoch fencing mechanism in OpenCell stores the ownership epoch that authorized a cell's activation. Before any storage operation, celld verifies the cell's epoch matches the current cluster state. If a cell migrates to another node, its epoch increments, and any in-flight connection with a stale epoch is rejected, forcing re-activation with fresh state.

What is the performance cost of the async-to-sync bridge in celld?

There is no cross-thread cost for storage operations. The JavaScript async API resolves to a synchronous Rust callback within the same thread; the await merely yields to the event loop. The actual SQLite work executes without thread hops, providing near-native SQLite latency (typically 10-100μs for cached pages) while preserving the ergonomic async interface.

How does statement caching work in celld's storage layer?

celld prepares the three KV statements (KV_GET_SQL, KV_PUT_SQL, KV_DELETE_SQL) once per process using SQLite's prepare_cached API. These cached statements are reused across all operations on all cells, eliminating SQL parsing and query planning overhead. The cache is thread-local and cleared only on process restart.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}