# CubeCoW Snapshot Engine Architecture: Instant Cloning and Rollback in CubeSandbox

> Explore the CubeCoW snapshot engine architecture for instant cloning and rollback in CubeSandbox. Discover its lightweight Copy-on-Write mechanism for sub-100ms operations.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: architecture
- Published: 2026-07-03

---

**CubeCoW uses a lightweight Copy-on-Write architecture with kernel reflink support to enable sub-100ms cloning and rollback of sandbox root filesystems and memory states.**

The CubeCoW snapshot engine powers TencentCloud's CubeSandbox platform, providing instant cloning and rollback capabilities through a hybrid Go/Rust implementation. It represents every sandbox root filesystem and memory state as a **cubecow object** within a flat namespace, leveraging kernel-level reflinks to perform metadata-only copy operations that complete in tens of milliseconds.

## Core Architecture Components

### localStorage Singleton

The `localStorage` struct serves as the central storage coordinator for the Cubelet. Defined in `Cubelet/storage/`, it holds configuration, maintains a reference to the Cubecow manager (`cowManager`), and tracks per-sandbox `StorageInfo`. Key methods include `Engine()`, `useCowStorage()`, and `ensureCowManager()`.

### Rust-Based Cubecow Engine

At the heart of the system lies the `cubecow.Engine` implemented in Rust and exposed to Go via C-FFI. This engine handles the creation, activation, cloning, and deletion of CoW objects. Critical functions include `CreateSnapshot`, `DeleteSnapshot`, `ResolveDevPath`, `GetMetrics`, and `RollbackDeriveNewGen`.

### Snapshot Artifacts Layer

The [`cubecow_snapshot_artifacts.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cubecow_snapshot_artifacts.go) file provides the Go-level abstraction that translates Cubelet concepts (templates, sandboxes) into Cubecow objects. It handles naming conventions, kind normalization, and catalog updates through functions like `CreateTemplateRootfsFromBuild`, `ResolveSnapshotForRollback`, and `PersistSandboxRootfsAfterRollback`.

### Object Types and Catalog

The engine distinguishes between two primary object types: `CowKindSnapshot` (read-only) and `CowKindVolume` (mutable). The [`snapshot_catalog.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot_catalog.go) maintains an on-disk JSON catalog ([`catalog.json`](https://github.com/TencentCloud/CubeSandbox/blob/main/catalog.json)) recording snapshot IDs, paths, and kinds, enabling fast lookups without filesystem scans via `SetSnapshotCatalogRoots`, `AddSnapshotCatalogRoot`, and `GetCatalogEntry`.

## How Instant Cloning Works

CubeCoW achieves instant cloning through **reflink cloning** of the underlying block device. When creating a snapshot, `cubecow.Engine` creates a new CoW device that shares blocks with its parent. Subsequent writes allocate new blocks only for the child, keeping the parent immutable. Because reflinks are handled by the kernel, cloning becomes a metadata-only operation completing in tens of milliseconds regardless of actual data size.

## CubeCoW Rollback Workflow

The rollback mechanism follows a three-phase process that preserves existing snapshots while generating new active generations:

1. **Resolve**: The engine locates the rootfs and memory objects for the target snapshot using `ResolveSnapshotForRollback`.
2. **Derive**: It creates a new generation via `RollbackDeriveNewGen`, producing a thin reflink copy that becomes the active rootfs.
3. **Persist**: The new generation is committed to the sandbox's `StorageInfo` through `PersistSandboxRootfsAfterRollback`.

This design guarantees that rollback never mutates existing snapshots, preserving the ability to roll back again or branch off new clones. Cleanup operations use `CleanupCowTemplateObjects` to perform idempotent deletion of leftover objects, aggregating errors without aborting the cleanup loop.

## Implementation Examples

### Creating a Snapshot

```go
// sb is a sandbox client obtained from the SDK.
snap, err := sb.CreateSnapshot(ctx, "my-snap")
if err != nil {
    log.Fatalf("snapshot failed: %v", err)
}
fmt.Printf("snapshot ID: %s\n", snap.SnapshotID)

```

### Cloning from a Snapshot

```go
opts := cubesandbox.CloneOptions{
    SnapshotID: snap.SnapshotID,
    Count:      3, // spin up 3 clones instantly
}
clones, err := sb.CloneFromSnapshot(ctx, opts)
if err != nil {
    log.Fatalf("clone failed: %v", err)
}
for _, c := range clones {
    fmt.Printf("new sandbox: %s\n", c.SandboxID)
}

```

### Performing Rollback

```go
// RollbackSandbox is a high-level API that triggers the engine path.
rsp, err := sb.Rollback(ctx, snap.SnapshotID, cubesandbox.RollbackParams{
    NewGen:        2,               // generate a new rootfs generation
    DesiredSize:   5 << 30,         // 5 GiB target size (optional)
})
if err != nil {
    log.Fatalf("rollback failed: %v", err)
}
fmt.Printf("rollback succeeded, new rootfs: %s\n", rsp.RootfsVol)

```

### Inspecting Cubecow Objects

```go
refs := []storage.CowObjectRef{
    {Name: "tpl-abc-rootfs", Kind: "snapshot", Role: "rootfs"},
    {Name: "tpl-abc-memory", Kind: "volume", Role: "memory"},
}
status, err := storage.InspectCowObjects(context.Background(), refs)
if err != nil {
    log.Fatalf("inspect failed: %v", err)
}
fmt.Printf("%+v\n", status)

```

## Key Source Files

The CubeCoW snapshot engine spans these critical files in the TencentCloud/CubeSandbox repository:

- [`Cubelet/storage/cubecow_engine.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/cubecow_engine.go): Thin wrapper exposing the global Cubecow engine instance.
- [`Cubelet/storage/cubecow_snapshot_artifacts.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/cubecow_snapshot_artifacts.go): Core API for creating, committing, rolling back, and cleaning up Cubecow objects.
- [`Cubelet/storage/snapshot_catalog.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/snapshot_catalog.go): JSON catalog handling for persistent snapshot metadata.
- [`Cubelet/services/cubebox/rollback.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/services/cubebox/rollback.go): RPC entry point (`service.RollbackSandbox`) integrating the Cubecow engine into the Cubelet service layer.
- [`Cubelet/services/cubebox/snapshot_runtime_binding.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/services/cubebox/snapshot_runtime_binding.go): Defines snapshot lifecycle and hypervisor shim interaction.
- [`Cubelet/storage/cubecow_volume_manager.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/cubecow_volume_manager.go): Volume manager implementation for creating, deleting, and resolving paths.
- [`sdk/go/snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/go/snapshot.go): Public Go SDK exposing snapshot, clone, and rollback APIs.

## Summary

- CubeCoW uses a hybrid Go/Rust architecture with a flat namespace for snapshot and volume objects.
- **Reflink cloning** enables instant duplication by sharing block metadata rather than copying data, completing in sub-100ms.
- The rollback workflow creates new generations via `RollbackDeriveNewGen` without mutating existing snapshots.
- Persistent metadata lives in [`catalog.json`](https://github.com/TencentCloud/CubeSandbox/blob/main/catalog.json) managed by [`snapshot_catalog.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot_catalog.go).
- All operations are orchestrated from the Cubelet's RPC handlers and exposed through the Go SDK.

## Frequently Asked Questions

### What is the CubeCoW snapshot engine?

CubeCoW is the Copy-on-Write snapshot engine powering CubeSandbox's instant cloning and rollback capabilities. It represents every sandbox root filesystem and memory state as a cubecow object in a flat namespace, using reflink-based block sharing to perform metadata-only operations.

### How does instant cloning work in CubeCoW?

Instant cloning works by creating a new CoW device that shares blocks with its parent snapshot via kernel reflinks. When a clone is created, only metadata is duplicated; data blocks are shared until the clone writes, which triggers allocation of new blocks only for modified regions. This completes in tens of milliseconds regardless of dataset size.

### What happens during a CubeCoW rollback?

During rollback, the engine first resolves the target snapshot objects using `ResolveSnapshotForRollback`, then derives a new generation via `RollbackDeriveNewGen` to create a thin reflink copy of the historical state. This new generation becomes the active rootfs through `PersistSandboxRootfsAfterRollback` without modifying the original snapshot, enabling repeated rollbacks and branching.

### Where is snapshot metadata stored in CubeSandbox?

Snapshot metadata persists in an on-disk JSON catalog ([`catalog.json`](https://github.com/TencentCloud/CubeSandbox/blob/main/catalog.json)) managed by [`Cubelet/storage/snapshot_catalog.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/snapshot_catalog.go). This catalog records each snapshot's ID, path, and kind (`CowKindSnapshot` or `CowKindVolume`), allowing fast lookup via `GetCatalogEntry` without scanning the entire storage tree.