How CubeProxy Handles Reverse Proxy Request Routing and Load Balancing

CubeProxy uses Redis-backed service discovery to maintain a fleet of live instances, broadcasts sandbox metadata to all proxies via the sidecar's proxypush.Client, and handles per-request routing through local meta-dictionaries with implicit load balancing via a SETNX-based state gate.

CubeProxy serves as the front-end reverse proxy in TencentCloud's CubeSandbox architecture, sitting between external clients and sandbox workloads. Unlike traditional architectures that rely on external load balancers, CubeProxy implements a self-healing distribution mechanism through Redis-coordinated discovery and fleet-wide state synchronization.

Redis-Based Service Discovery and Fleet Management

The discovery mechanism in cube-lifecycle-manager/internal/discovery/discovery.go establishes the foundation for reverse proxy request routing. Each CubeProxy instance registers its admin address in the Redis hash key cube:v1:shared:cube_proxy:registry and continuously updates its presence in the sorted set cube:v1:shared:cube_proxy:heartbeat.

The discovery module watches these keys to build a fleet of live proxies. When a new proxy appears or disappears, the system fires OnJoin or OnLeave callbacks to update the fleet membership dynamically. This ensures that the sidecar always maintains an accurate view of available proxy instances without manual configuration.

Fleet-Wide Metadata Broadcasting via Sidecar

Once the fleet is established, the cube-lifecycle-manager sidecar creates a proxypush.Client using the current fleet snapshot. As implemented in cube-lifecycle-manager/internal/proxypush/client.go, the client initializes with proxypush.NewWithFleet and distributes state changes to every live admin URL in the fleet.

The sidecar pushes sandbox metadata and state changes using three primary methods:

• UpsertMeta: Broadcasts sandbox metadata (ID, template, networking info) to all proxies
• DeleteMeta: Removes sandbox entries from proxy caches
• SetState: Updates sandbox lifecycle states (e.g., running, paused)

This broadcast approach ensures that every CubeProxy instance maintains a consistent local copy of the routing table, eliminating the need for request-time coordination between proxies.

Client Initialization Example

// in cube-lifecycle-manager/main.go
fleet := discovery.NewStatic(cfg.CubeProxyAdminURLs) // or discovery.New(...)
pushClient := proxypush.NewWithFleet(fleet, cfg.CubeAdminToken,
    cfg.HTTPTimeout, logger.Named("proxypush"))

Broadcasting Metadata Updates

// inside some lifecycle handler
meta := proxypush.Meta{
    SandboxID:  sandboxID,
    TemplateID: templateID,
    // … other fields …
}
if err := pushClient.UpsertMeta(ctx, meta); err != nil {
    logger.Error("failed to push meta to proxies", zap.Error(err))
}

State Synchronization Across the Fleet

if err := pushClient.SetState(ctx, sandboxID, proxypush.StateRunning); err != nil {
    logger.Error("failed to set state", zap.Error(err))
}

Per-Request Routing and Implicit Load Balancing

When an external request arrives at a CubeProxy instance, the proxy consults its local meta-dictionary to determine the sandbox's target address. This dictionary contains the routing information pushed by the sidecar. If the target sandbox is not found in the local cache or is marked as paused, the proxy returns an HTTP 503 Service Unavailable error.

The actual load balancing occurs implicitly through the sidecar's SETNX-based state gate implemented in cube-lifecycle-manager/internal/sweeper/sweeper.go. This mechanism ensures that only one proxy processes a given request at a time, preventing race conditions during sandbox rollouts or migrations. When multiple proxies could theoretically handle the same sandbox, the first proxy to acquire the state lock processes the request while others return 503 errors until the meta-dictionary updates.

Request Handling Implementation

func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // look up sandbox target from local meta‑dict
    target, ok := p.metaDict.Get(r.Host)
    if !ok || p.isPaused(target) {
        http.Error(w, "service unavailable", http.StatusServiceUnavailable)
        return
    }
    // forward request to the sandbox host‑proxy
    proxy := httputil.NewSingleHostReverseProxy(target.URL)
    proxy.ServeHTTP(w, r)
}

Configuration and Administrative Interfaces

The routing system relies on configuration defined in cube-lifecycle-manager/internal/config/config.go, which validates that at least one CubeProxyAdminURL is specified. Administrative endpoints exposed through cube-lifecycle-manager/internal/httpapi/server.go handle internal routes such as /_sidecar_resume, allowing CubeProxy to coordinate with the sidecar during sandbox lifecycle transitions.

Summary

• Redis-backed discovery in cube-lifecycle-manager/internal/discovery/discovery.go maintains the fleet of live proxies using heartbeat sorted-sets and registry hashes.
• Fleet-wide broadcasting via proxypush.Client ensures every proxy holds identical routing metadata, eliminating external load balancer dependencies.
• Per-request routing uses local meta-dictionaries for O(1) lookup performance, with implicit load balancing achieved through SETNX-based state gates in cube-lifecycle-manager/internal/sweeper/sweeper.go.
• Self-healing architecture automatically excludes unhealthy proxies when their Redis heartbeats expire, requiring no manual intervention.

Frequently Asked Questions

How does CubeProxy discover other proxy instances in the fleet?

CubeProxy instances register their admin addresses in the Redis hash cube:v1:shared:cube_proxy:registry and update the sorted set cube:v1:shared:cube_proxy:heartbeat continuously. The discovery module in cube-lifecycle-manager/internal/discovery/discovery.go monitors these keys and triggers OnJoin or OnLeave callbacks to maintain the current fleet membership.

What happens when a CubeProxy instance becomes unhealthy?

When a proxy stops updating its heartbeat in Redis, the discovery module detects the expiration and removes the instance from the fleet via the OnLeave callback. The sidecar stops pushing metadata to this address, and client requests naturally route to remaining healthy instances without explicit configuration changes.

How does the sidecar ensure all proxies have consistent routing information?

The sidecar creates a proxypush.Client using proxypush.NewWithFleet and broadcasts every metadata change (UpsertMeta), deletion (DeleteMeta), and state transition (SetState) to every live admin URL in the fleet simultaneously. This ensures eventual consistency across all proxy instances.

What prevents multiple proxies from handling the same sandbox request simultaneously?

The system implements a SETNX-based state gate in cube-lifecycle-manager/internal/sweeper/sweeper.go that ensures only one proxy holds the authoritative state for a sandbox at any given time. If multiple proxies receive requests for the same sandbox during a rollout, only the instance with the current state lock processes the request while others return HTTP 503 errors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}