Redis in CubeSandbox's Stateless Control Plane: Architecture and Implementation

Redis serves as the single source of truth for CubeSandbox's stateless control plane, storing all metadata, lifecycle events, distributed locks, and routing tables in a centralized deployment that enables horizontal scaling and rapid failover.

CubeSandbox's architecture deliberately separates stateless control plane components from the node-local data plane. In this design, Redis assumes the critical role of maintaining the system's shared state, allowing CubeAPI and CubeMaster instances to operate without local persistent storage. Because no control plane instance stores data locally, Redis becomes the authoritative source for all coordination and lifecycle information.

Why Redis Powers the Stateless Architecture

The control plane achieves true statelessness by externalizing all coordination and persistence to Redis. This eliminates the need for local databases or configuration files on individual nodes. Adding or removing control-plane instances becomes as simple as starting or stopping a process, since Redis remains the only shared state component.

Core Functions of Redis in the Control Plane

Centralized Metadata Store

Sandbox descriptors—including IDs, node assignments, network endpoints, and template references—are stored in Redis hash tables. This allows any API gateway or scheduler to query sandbox state without maintaining local caches.

Event Stream for Lifecycle Management

Lifecycle actions such as create, pause, resume, and destroy are emitted as Redis stream entries. The cube-lifecycle-manager service consumes these events to drive auto-pause/resume logic, while CubeProxy instances discover each other through a Redis-backed registration table.

Distributed Lock Coordination

Auto-pause/resume operations rely on Redis-based mutexes to ensure exclusive access. Only one node can manipulate a sandbox's state at any given time, preventing race conditions during concurrent operations.

Dynamic Routing Table

CubeProxy queries Redis to resolve sandbox routing information (host/port to sandbox ID mappings). This enables horizontal scaling of proxies without static configuration files.

Implementation Details from the Source Code

The TencentCloud/CubeSandbox codebase demonstrates Redis integration through specific client initialization patterns and stream operations.

Creating a Redis client in cube-lifecycle-manager/cmd/cube-lifecycle-manager/main.go initializes the connection used by both CubeMaster and Cubelet components:

// CubeMaster – cube-lifecycle-manager/main.go
// https://github.com/TencentCloud/CubeSandbox/blob/master/cube-lifecycle-manager/cmd/cube-lifecycle-manager/main.go#L56
rdb := redis.NewClient(&redis.Options{
    Addr: cfg.RedisAddr,          // e.g. "10.0.0.5:6379"
    Password: cfg.RedisPass,     // optional
})
stream := redisstream.New(rdb, logger.Named("redis"))

Publishing lifecycle events occurs through the Redis stream abstraction, as shown at line 255 of the same file:

// CubeMaster – publishing an event
// https://github.com/TencentCloud/CubeSandbox/blob/master/cube-lifecycle-manager/cmd/cube-lifecycle-manager/main.go#L255
ev := redisstream.Event{
    Type:      redisstream.EventCreate,
    SandboxID: sandboxID,
    NodeID:    targetNode,
}
if err := stream.Publish(ctx, ev); err != nil {
    logger.Error("failed to publish lifecycle event", zap.Error(err))
}

Consuming events in the auto-pause/resume service follows a continuous read loop at line 273:

// cube-lifecycle-manager – event loop
// https://github.com/TencentCloud/CubeSandbox/blob/master/cube-lifecycle-manager/cmd/cube-lifecycle-manager/main.go#L273
for {
    ev, err := stream.Read(ctx)
    if err != nil { continue }
    handleEvent(ctx, ev, pushClient)
}

Reading sandbox metadata for routing lookups happens in cube-lifecycle-manager/internal/discovery/redis.go at line 36:

// CubeProxy – registration discovery
// https://github.com/TencentCloud/CubeSandbox/blob/master/cube-lifecycle-manager/internal/discovery/redis.go#L36
func (d *Discovery) GetSandboxInfo(sandboxID string) (*SandboxInfo, error) {
    key := fmt.Sprintf("sandbox:%s:info", sandboxID)
    fields, err := d.rdb.HGetAll(ctx, key).Result()
    …
}

Key Source Files and Components

Understanding Redis's role requires examining these specific files in the TencentCloud/CubeSandbox repository:

• docs/architecture/overview.md – Explains the stateless design philosophy and Redis's central role in the control plane.
• Cubelet/pkg/redisconf/redis.go – Implements the Redis client initialization wrapper used across node components.
• cube-lifecycle-manager/internal/redisstream/stream.go – Provides the publish/consume API abstraction for lifecycle events.
• cube-lifecycle-manager/internal/discovery/redis.go – Handles proxy registration and metadata lookup via Redis hash operations.
• cube-lifecycle-manager/cmd/cube-lifecycle-manager/main.go – Instantiates the Redis client and integrates it into the main event loop.

Summary

• Redis acts as the single source of truth for CubeSandbox's stateless control plane, eliminating local state in CubeAPI and CubeMaster instances.
• Hash tables store sandbox metadata such as IDs, node assignments, and network endpoints, enabling stateless query handling.
• Redis streams power the event-driven architecture for lifecycle management (create, pause, resume, destroy) consumed by the cube-lifecycle-manager.
• Distributed locks prevent race conditions during auto-pause/resume operations by ensuring single-node access to sandbox state.
• Dynamic routing tables in Redis allow CubeProxy instances to scale horizontally without static configuration.

Frequently Asked Questions

How does Redis enable horizontal scaling of CubeSandbox's control plane?

By storing all state in Redis, CubeAPI and CubeMaster instances become completely interchangeable. You can add or remove instances dynamically because they retrieve all necessary coordination data, metadata, and routing information from the centralized Redis deployment rather than local storage.

What happens if the Redis instance becomes unavailable?

If Redis becomes unavailable, the control plane loses its ability to coordinate lifecycle events, acquire distributed locks, and route requests. The stateless design means control plane instances cannot function without Redis connectivity, making Redis a critical dependency that requires high availability configurations (such as Redis Sentinel or Cluster mode) for production deployments.

Why did CubeSandbox choose Redis over a traditional relational database?

Redis provides the sub-millisecond latency necessary for distributed locking and real-time event streaming, which are critical for the auto-pause/resume functionality. The hash table and stream data structures map naturally to the sandbox metadata and lifecycle event patterns, whereas relational databases would require additional abstraction layers to achieve similar performance characteristics.

How does CubeProxy use Redis for service discovery?

CubeProxy instances query Redis hash tables to resolve sandbox routing information, specifically mapping host/port combinations to sandbox IDs. This lookup mechanism, implemented in cube-lifecycle-manager/internal/discovery/redis.go, eliminates the need for static configuration files and allows proxies to discover new sandboxes automatically as they are created.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}