# How celld Supports Both S3 and Google Cloud Storage Buckets

> Discover how celld seamlessly integrates S3 and Google Cloud Storage buckets. celld uses s3:// or gs:// prefixes for a unified API, simplifying your cloud storage management.

- Repository: [Deno/celld](https://github.com/denoland/celld)
- Tags: how-to-guide
- Published: 2026-08-15

---

**celld abstracts storage behind a unified bucket specification using `s3://` or `gs://` prefixes, automatically selecting the appropriate backend while exposing a consistent API for reads and writes.**

The `celld` project from Deno Land Inc. provides a distributed SQLite replication system that delegates persistent storage to external object stores. Rather than maintaining separate code paths for different providers, celld implements a single **bucket abstraction** in [`crates/celld/bucket.rs`](https://github.com/denoland/celld/blob/main/crates/celld/bucket.rs) that transparently handles both S3-compatible services and Google Cloud Storage.

## How Bucket Specifications Work

celld uses URL-style prefixes to determine which storage backend to initialize. This design keeps configuration explicit and user-facing interfaces minimal.

### Syntax and Parsing

The entry point is `Bucket::parse_spec` in [`crates/celld/bucket.rs`](https://github.com/denoland/celld/blob/main/crates/celld/bucket.rs). It accepts strings in two forms:

- `s3://BUCKET_NAME[/PREFIX]` — S3-compatible endpoints (AWS S3, MinIO, Cloudflare R2, etc.)
- `gs://BUCKET_NAME[/PREFIX]` — Google Cloud Storage buckets

```rust
// From crates/celld/bucket.rs
let bucket = Bucket::open("s3://my-bucket/backups").await?;
// or
let bucket = Bucket::open("gs://project-bucket/data").await?;

```

The parser extracts the scheme, validates mutual exclusivity of options, and dispatches to the corresponding builder.

## Backend Selection and Implementation

celld defines a `StorageBackend` enum with two variants: `S3` and `Gcs`. Each wraps provider-specific authentication and transport logic while presenting identical operation semantics.

### S3 Backend: `StorageBackend::S3`

When the `s3://` prefix is detected, celld constructs an `object_store::aws::AmazonS3Builder` via [`ltx/src/client/object_store.rs`](https://github.com/denoland/celld/blob/main/ltx/src/client/object_store.rs). The implementation respects:

- Standard AWS credential chain (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`)
- Optional `S3_ENDPOINT` for custom endpoints (MinIO, R2, etc.)
- Conditional PUT semantics using ETag verification for safe concurrent writes

```rust
// S3 configuration flow
let backend = StorageBackend::S3 {
    endpoint: std::env::var("S3_ENDPOINT").ok(),
    // ... other AWS-specific options
};

```

### GCS Backend: `StorageBackend::Gcs`

For `gs://` prefixes, celld bypasses all S3-specific configuration. The GCS backend:

- Authenticates via Google Application Default Credentials (ADC)
- Accepts `GOOGLE_APPLICATION_CREDENTIALS` for service account keys
- Ignores `S3_ENDPOINT`, `AWS_*` environment variables, and path-style endpoint logic

This separation is documented in [`docs/limitations.md`](https://github.com/denoland/celld/blob/main/docs/limitations.md) to prevent configuration errors.

## Unified API Across Backends

Both backends implement the same `Bucket` trait methods, enabling celld's replication, fencing, and restore operations to remain storage-agnostic. The unified interface includes:

- `get(key)` — streaming reads
- `put(key, data, options)` — conditional writes with ETag / generation preconditions
- `delete(key)` — removal with verification

```rust
// Same code works regardless of s3:// or gs://
let data = bucket.get("snapshot/0001.ltx").await?;
bucket.put("snapshot/0002.ltx", &new_data, PutOptions {
    if_match: Some(current_etag),
}).await?;

```

## CLI Validation and User Experience

The `deploy` subcommand in [`crates/celld/deploy.rs`](https://github.com/denoland/celld/blob/main/crates/celld/deploy.rs) enforces valid flag combinations at parse time:

- `--bucket gs://...` rejects `--endpoint` (S3-specific)
- `--bucket gs://...` rejects static AWS credential flags
- Help text documents required environment variables for each backend

This prevents runtime authentication failures from misconfigured credential mixing.

## Configuration Examples

| Provider | Bucket Spec | Required Environment |
|----------|-------------|----------------------|
| AWS S3 | `s3://my-bucket/prefix` | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` |
| MinIO | `s3://my-bucket/prefix` | `S3_ENDPOINT=http://localhost:9000`, AWS credentials |
| Cloudflare R2 | `s3://my-bucket/prefix` | `S3_ENDPOINT=https://<account>.r2.cloudflarestorage.com`, R2 tokens |
| GCS | `gs://my-bucket/prefix` | `GOOGLE_APPLICATION_CREDENTIALS` or ADC |

## Key Source Files

- [`crates/celld/bucket.rs`](https://github.com/denoland/celld/blob/main/crates/celld/bucket.rs) — `Bucket::parse_spec`, `StorageBackend` enum, backend dispatch
- [`crates/ltx/src/client/object_store.rs`](https://github.com/denoland/celld/blob/main/crates/ltx/src/client/object_store.rs) — S3 client construction via `object_store` crate
- [`crates/celld/deploy.rs`](https://github.com/denoland/celld/blob/main/crates/celld/deploy.rs) — CLI argument parsing and validation
- [`docs/README.md`](https://github.com/denoland/celld/blob/main/docs/README.md) — User-facing bucket configuration documentation
- [`docs/limitations.md`](https://github.com/denoland/celld/blob/main/docs/limitations.md) — Authentication requirements and unsupported combinations

## Summary

- **Single specification format** — `s3://` or `gs://` prefixes trigger automatic backend selection
- **S3 backend** — Uses `object_store` crate with full AWS credential chain and custom endpoint support via `S3_ENDPOINT`
- **GCS backend** — Relies on Google Application Default Credentials, explicitly ignoring S3 configuration
- **Unified interface** — `Bucket` trait methods abstract storage details from replication logic
- **CLI safeguards** — [`deploy.rs`](https://github.com/denoland/celld/blob/main/deploy.rs) validates that GCS buckets cannot combine with S3-specific flags

## Frequently Asked Questions

### Can I use GCS with S3-compatible interoperability mode?

No. celld's GCS backend (`gs://` prefix) uses native Google Cloud Storage authentication and APIs. It does not support GCS's S3 interoperability credentials. For S3-compatible access to GCS, use a third-party gateway or choose the S3 backend with a compatible endpoint.

### What happens if I set both `S3_ENDPOINT` and use `gs://`?

The CLI parser in [`crates/celld/deploy.rs`](https://github.com/denoland/celld/blob/main/crates/celld/deploy.rs) rejects this combination at startup. If bypassed, the GCS backend explicitly ignores `S3_ENDPOINT` per [`docs/limitations.md`](https://github.com/denoland/celld/blob/main/docs/limitations.md), so the setting has no effect.

### Does celld support other object storage providers?

celld officially supports S3-compatible endpoints and Google Cloud Storage. Other providers (Azure Blob, etc.) are not implemented in the current codebase. Some may work via S3-compatible gateways if they implement the standard S3 API.

### How does celld handle credentials rotation?

For S3, celld relies on the standard AWS SDK credential chain, which automatically handles IAM role assumption and credential refresh. For GCS, the `object_store` crate manages OAuth2 token refresh using ADC. No explicit credential reload code exists in celld itself.