# How the LTX Format Handles Cell Replication to Object Storage in Celld

> Learn how the LTX format streams SQLite cell transactions for efficient replication to object storage like S3 and MinIO. Explore Celld's approach to seamless data backup.

- Repository: [Deno/celld](https://github.com/denoland/celld)
- Tags: internals
- Published: 2026-08-15

---

**The LTX format implements a streaming replication pipeline that serializes SQLite cell transactions into self-contained LTX files and uploads them to object storage backends like S3, MinIO, or local filesystems.**

The `celld-ltx` crate in the [denoland/celld](https://github.com/denoland/celld) repository provides embedded, continuous replication for SQLite databases managed within the Celld runtime. This article examines how the **LTX (Litestream-compatible transaction)** format captures, transmits, and restores cell state to remote object storage.

## How Cell Replication Works in LTX Format

The replication architecture separates concerns into three layers: **local capture**, **transfer orchestration**, and **remote storage abstraction**. Each committed transaction generates an immutable, incremental snapshot that can reconstruct the database at any point in time.

### Local Capture: WAL to LTX File Conversion

When a managed `Db` commits a transaction, the capture loop serializes the write-ahead log (WAL) changes into a **Level-0 (L0) LTX file**. In [`crates/ltx/src/db.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/db.rs), the `Db::ltx_path` method determines the local destination:

```

ltx/0/{txid}.ltx

```

These files contain:

- A structured header with transaction metadata
- Page-level incremental changes or full snapshots
- Cryptographic checksums for integrity verification

The exact encoding logic resides in [`crates/ltx/src/ltx.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/ltx.rs) and [`crates/ltx/src/codec.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/codec.rs), with the `encode_file` function handling serialization.

## The Replica Struct: Orchestrating Object Storage Sync

The `Replica<C>` struct in [`crates/ltx/src/replica.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/replica.rs) serves as the primary interface for **LTX format cell replication to object storage**. It couples a local `Db` with a generic `ReplicaClient` capable of remote operations.

```rust
pub struct Replica<C: ReplicaClient> {
    db: Option<Db>,
    pub client: C,
    pos: Pos,
    pos_known: bool,
} // https://github.com/denoland/celld/blob/main/crates/ltx/src/replica.rs#L91-L98

```

- **`db`** — The managed SQLite cell database
- **`client`** — Storage backend implementation (S3, file, etc.)
- **`pos`** — Last successfully replicated transaction ID
- **`pos_known`** — Whether `pos` reflects confirmed remote state

This design allows the same replication logic to operate across wildly different storage backends through trait abstraction.

## Uploading LTX Files with Replica::sync

The `sync` method drives incremental replication by identifying and uploading any LTX files newer than the current position. From [`crates/ltx/src/replica.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/replica.rs) lines 250-282:

```rust
let tx_id = TXID(self.pos().txid.0 + 1);
self.upload_ltx_file(0, tx_id, tx_id).await?;
self.set_pos(Pos::new(tx_id, 0));

```

The method:

1. **Determines the next transaction** to replicate based on `pos`
2. **Invokes `upload_ltx_file`** to transfer the LTX file via `client.write_ltx_file`
3. **Atomically updates `pos`** only after successful upload confirmation

This **cursor-based approach** ensures exactly-once semantics: interrupted syncs resume from the last confirmed position without re-uploading or skipping transactions.

## ReplicaClient Abstraction: Supporting Multiple Object Storage Backends

The `ReplicaClient` trait abstracts storage operations, enabling the LTX format to replicate cells to diverse object storage systems. Two primary implementations exist:

### FileReplicaClient (Local Development)

Located in [`crates/ltx/src/client/file.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/client/file.rs), this client stores LTX files in a local directory. It uses `clean_replica_url_path` to normalize paths and mirrors the remote storage semantics for testing.

### ObjectStoreReplicaClient (S3-Compatible Production)

In [`crates/ltx/src/client/object_store.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/client/object_store.rs), this implementation targets production object storage:

- **Multipart uploads** for large LTX files
- **Range GET requests** for selective page retrieval during restore
- **Credential handling** via `StorageCredentials` for access key and endpoint configuration

URL parsing and endpoint detection logic live in [`crates/ltx/src/replica_url.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/replica_url.rs). Functions like `parse_replica_url_with_query` (lines 21-78) and `ensure_endpoint_scheme` (lines 26-40) normalize MinIO, Cloudflare R2, and AWS S3 endpoints, automatically selecting HTTP/HTTPS as appropriate.

## Restoring Cells: Downloading and Merging LTX Files

Cell restoration reverses the replication flow. `Replica::restore` in [`crates/ltx/src/replica.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/replica.rs) lines 311-389:

1. **Builds a restore plan** via `calc_restore_plan` — lists required LTX files in TXID order
2. **Downloads concurrently** — uses `RESTORE_DOWNLOAD_CONCURRENCY` parallel streams
3. **Merges pages** — `build_database_image` reconstructs the exact SQLite database image at the target TXID

The restore process fetches only the LTX files necessary to reach the desired state, minimizing bandwidth and latency.

## Fault Tolerance and Position Recovery

The replication system handles failures through position state management. The `pos_untrustworthy` method (lines 67-83) clears the cached position when:

- Checksum mismatches indicate data corruption
- Missing LTX files suggest state divergence
- Other invariant violations occur

Transient network errors **preserve** the position, preventing expensive remote listing operations during intermittent connectivity issues.

## Practical Example: Replicating a Cell to S3

```rust
use celld_ltx::db::Db;
use celld_ltx::replica::Replica;
use celld_ltx::client::object_store::ObjectStoreReplicaClient;
use celld_ltx::replication::StorageCredentials;

// Open the local cell database
let db = Db::open(std::path::PathBuf::from("production_cell.db"))?;

// Configure S3-compatible credentials
let creds = StorageCredentials::new(
    "https://s3.us-east-1.amazonaws.com".into(),
    Some(("AKIA...".into(), "wJalr...".into())),
    None,
);

// Create the object storage client
let client = ObjectStoreReplicaClient::new("my-replication-bucket".into(), creds);

// Initialize replica and sync
let mut replica = Replica::new(db, client);
tokio_runtime.block_on(replica.sync())?;

```

## Summary

- **LTX files** provide immutable, self-contained transaction snapshots of SQLite cells
- **`Replica<C>`** orchestrates sync and restore operations with cursor-based position tracking
- **`ReplicaClient`** abstraction enables seamless support for files, S3, MinIO, and R2
- **`sync`** uploads incremental LTX files; **`restore`** reconstructs databases via parallel download and merge
- **Fault tolerance** distinguishes permanent errors (position reset) from transient failures (retry with preserved cursor)

## Frequently Asked Questions

### What object storage backends does Celld LTX support?

Celld LTX supports **any S3-compatible API** including AWS S3, MinIO, Cloudflare R2, and DigitalOcean Spaces, plus **local filesystem** replication via `FileReplicaClient`. Backend selection occurs at runtime through URL parsing in [`replica_url.rs`](https://github.com/denoland/celld/blob/main/replica_url.rs), which automatically detects endpoint types and adjusts scheme handling.

### How does LTX replication handle network interruptions?

Transient failures preserve the replication cursor (`pos`), allowing resume without re-listing remote objects. Permanent failures like checksum mismatches trigger `pos_untrustworthy`, clearing the position and forcing a fresh remote state comparison on the next sync attempt.

### What is the difference between Level-0 and higher LTX levels?

Level-0 (L0) files contain raw transaction data from the WAL. Higher levels (L1, L2+) result from compaction — merging multiple L0 files into larger, more efficient archives. The `celld-ltx` crate currently focuses on L0 replication; compaction logic operates separately from the core replication path.

### Can I restore a cell to a specific point in time?

Yes. The `restore` method accepts a target TXID, and `calc_restore_plan` determines the minimal LTX file set needed to reconstruct that exact state. Downloads proceed in parallel with configurable concurrency, then `build_database_image` merges pages to produce the final SQLite database.