# SpacetimeDB Scaling and Performance: Horizontal and Vertical Optimization Guide

> Discover how SpacetimeDB scales using horizontal replica sets and vertical optimization. Learn to boost performance by managing indexes for read efficiency.

- Repository: [Clockwork Labs/SpacetimeDB](https://github.com/clockworklabs/SpacetimeDB)
- Tags: performance
- Published: 2026-03-09

---

**SpacetimeDB achieves horizontal scaling through configurable replica sets with automatic leader election, while vertical scaling relies on strategic index management that trades write latency for read performance.**

SpacetimeDB is an open-source relational database optimized for real-time applications, combining the performance of in-memory processing with the durability of persistent storage. Understanding SpacetimeDB scaling and performance requires examining both its distributed architecture for horizontal growth and its indexing strategies for single-node optimization.

## Horizontal Scaling Through Replica Management

### Replica Architecture and Leader Election

The foundation of horizontal scaling in SpacetimeDB resides in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs), where the `Replica` struct defines each node in a cluster with `database_id`, `node_id`, and a critical `leader` boolean flag. The control database in [`crates/standalone/src/control_db.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/control_db.rs) tracks leadership via `get_leader_replica_by_database`, ensuring that only the leader replica processes writes while followers serve read-only traffic.

### Dynamic Replica Scheduling

The `schedule_replicas` function in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs) orchestrates cluster topology. When a database is created, the engine invokes `schedule_replicas(database_id, num_replicas)`, where `num_replicas` defaults to 1 for standalone deployments but accepts any `u8` value for clustered configurations.

During module updates, the `publish_database` function compares desired versus actual replica counts. It triggers `insert_replica` to scale up or `delete_replica` (for non-leader nodes) to scale down, emitting transparent log messages: "Scaling up database …" or "Scaling down database …".

### Client-Side Configuration

Users configure replication through the client API defined in [`crates/client-api/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/client-api/src/lib.rs). The `DatabaseDef::num_replicas` field accepts replication factors via the unstable CLI flag `--num-replicas`:

```bash
spacetime publish \
    --file my_module.wasm \
    --identity my-db \
    --num-replicas 3

```

This value propagates through the REST/WebSocket API to the backend replication logic in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs).

## Vertical Scaling with Index Optimization

### Index Architecture and Memory Tracking

Vertical scaling in SpacetimeDB depends on efficient index management within [`crates/table/src/table.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/table/src/table.rs). The `Table` struct maintains index metadata through several diagnostic methods:

- `num_indices()` returns `self.indexes.len()`
- `num_rows_in_indexes()` calculates `self.num_rows() * self.indexes.len()`
- `bytes_used_by_index_keys()` iterates all indexes summing `idx.num_key_bytes()`

These metrics enable precise memory accounting when scaling tables to millions of rows.

### Index Implementations and Trade-offs

The `crates/table/src/table_index/` directory contains specialized index structures:

- [`hash_index.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/hash_index.rs) – O(1) lookups for equality predicates
- [`unique_direct_index.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/unique_direct_index.rs) – Optimized single-element lookups with uniqueness constraints
- [`multimap.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/multimap.rs) – Many-to-many relationship support

Selecting the appropriate index shape transforms O(N) table scans into O(log N) or O(1) operations.

### Write Path Performance Implications

Every write operation traverses all table indexes. In `Table::insert` (around line 890), the engine invokes `index.check_and_insert` for each index:

```rust
// Conceptual representation of the write path
for index in &self.indexes {
    index.check_and_insert(row)?;
}

```

This means **write latency grows linearly with the number of indexes** on the table. High-throughput applications must balance read optimization against write amplification.

### Read Path Optimization

The read path leverages indexes through `find_same_row_via_unique_index` (around line 945), which bypasses full table scans when serving queries against unique indexed columns. Direct column access via `Table::project` and `RowRef::read_col` operates without index overhead when projections don't require indexed lookups.

## Observability and Runtime Metrics

### Key Performance Indicators

SpacetimeDB exposes Prometheus-compatible metrics in [`crates/core/src/worker_metrics/mod.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/core/src/worker_metrics/mod.rs). Critical indicators for scaling decisions include:

- `spacetime_total_incoming_queue_length` – WebSocket message backlog indicating client write pressure
- `spacetime_total_outgoing_queue_length` – Server-to-client message queue depth, relevant during replica synchronization
- `spacetime_replay_total_time_seconds` – Database recovery duration after restart, scaling with replica count and log size
- `spacetime_snapshot_creation_time_total` – Snapshot overhead, sensitive to index cardinality
- `spacetime_module_create_instance_time_seconds` – WASM/V8 instantiation latency during module scaling

### Monitoring Replication Health

Metrics automatically tag data with `db: Identity` and `replica_id`, enabling per-replica analysis. Grafana dashboards can track replica lag through `spacetime_total_outgoing_queue_length` per replica, while `spacetime_replay_total_time_seconds` reveals how long replicas take to catch up after restarts.

## Practical Implementation Examples

### Configuring Multi-Replica Deployment

Deploy a three-node cluster using the CLI:

```bash
spacetime publish \
    --file my_module.wasm \
    --identity production-db \
    --num-replicas 3

```

This triggers `schedule_replicas` with `num_replicas = 3`, creating two follower replicas and one leader in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs).

### Optimizing Table Indexes

Add a composite index for high-cardinality queries:

```rust
use spacetimedb::{Table, IndexAlgorithm, BTreeAlgorithm};

let mut table = MyTable::table(&db);
let algo = IndexAlgorithm::BTree(BTreeAlgorithm {
    columns: vec!["user_id".into(), "timestamp".into()],
    unique: false,
});
table.add_index(algo)?;

```

This creates a B-tree index in `crates/table/src/table_index/`, reducing query complexity from O(N) to O(log N).

### Monitoring with Prometheus

Configure scraping for the metrics endpoint:

```yaml

# prometheus.yml

scrape_configs:
  - job_name: "spacetime"
    static_configs:
      - targets: ["localhost:9090"]

```

Query for scaling bottlenecks:

```promql

# High incoming queue indicates need for more replicas or index optimization

spacetime_total_incoming_queue_length{db="production-db"}

# Snapshot duration correlates with index count

rate(spacetime_snapshot_creation_time_total_seconds[5m])

```

## Summary

- **Horizontal scaling** is achieved through the `Replica` struct and `schedule_replicas` logic in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs), supporting dynamic scaling via the `--num-replicas` CLI flag.
- **Vertical scaling** depends on strategic index management in [`crates/table/src/table.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/table/src/table.rs), where each index adds linear write overhead but provides logarithmic or constant-time read improvements.
- **Observability** is provided through Prometheus metrics in [`crates/core/src/worker_metrics/mod.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/core/src/worker_metrics/mod.rs), tracking queue depths, replay times, and snapshot durations to inform scaling decisions.
- **Trade-offs** between replication (availability/read scaling) and indexing (query performance) must be balanced based on workload characteristics observed through runtime metrics.

## Frequently Asked Questions

### How does SpacetimeDB handle leader election during horizontal scaling?

SpacetimeDB tracks leadership through the `leader` boolean flag on the `Replica` struct in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs). The control database in [`crates/standalone/src/control_db.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/control_db.rs) provides `get_leader_replica_by_database` to identify the current leader, ensuring only one replica processes writes while followers remain read-only. When scaling up, the existing leader remains unchanged until explicitly failed over.

### What is the performance impact of adding indexes in SpacetimeDB?

Each index adds linear write overhead because `Table::insert` in [`crates/table/src/table.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/table/src/table.rs) calls `index.check_and_insert` for every index on the table. However, indexes provide O(log N) or O(1) read performance for indexed columns versus O(N) table scans. The `bytes_used_by_index_keys` method helps monitor memory consumption as tables scale to millions of rows.

### How can I monitor replica lag in a SpacetimeDB cluster?

Monitor the `spacetime_total_outgoing_queue_length` metric from [`crates/core/src/worker_metrics/mod.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/core/src/worker_metrics/mod.rs), which tracks server-to-client message queues per replica. High values indicate replication lag. Additionally, track `spacetime_replay_total_time_seconds` to measure how long replicas take to catch up after restarts, which grows with replica count and transaction log size.

### Can I change the number of replicas without downtime?

Yes, SpacetimeDB supports dynamic scaling through the `publish_database` logic in [`crates/standalone/src/lib.rs`](https://github.com/clockworklabs/SpacetimeDB/blob/main/crates/standalone/src/lib.rs). When you publish a module with a different `--num-replicas` value, the engine compares the desired count with current replicas and automatically invokes `insert_replica` to scale up or `delete_replica` (for non-leaders) to scale down without requiring a full cluster restart.