# CubeAPI Gateway Architecture for High-Concurrency REST API Handling

> Discover the CubeAPI gateway architecture built with Rust. Handle millions of concurrent REST requests with low latency and per-key rate limiting. Explore the four-layer design for high-performance API management.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: architecture
- Published: 2026-07-11

---

**CubeSandbox implements a four-layer Rust-based gateway—comprising the Cube API (Axum), Cube Proxy (tokio), Nginx front-end, and optional sub-domain isolation—to handle millions of concurrent REST requests with per-key token-bucket rate limiting and minimal latency.**

The TencentCloud/CubeSandbox project delivers a high-performance API gateway engineered specifically for sandboxed AI agent workloads and E2B-compatible environments. This CubeAPI gateway architecture for high-concurrency REST API handling leverages Rust's async runtime and a layered reverse-proxy design to ensure strict tenant isolation, sub-millisecond latency, and horizontal scalability out of the box.

## The Four-Layer Architecture

The gateway consists of four tightly integrated layers that process requests from the public internet down to the individual sandbox instances.

### Layer 1: Cube API (E2B-Compatible REST Service)

The **Cube API** serves as the core HTTP service implementing all sandbox, template, snapshot, and cluster operations. In [`CubeAPI/src/routes.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/routes.rs), the Axum framework mounts all routes under the `/cubeapi/v1` prefix, creating a unified routing table for sandbox management endpoints.

The implementation uses **Rust** with the **Axum** web framework to maximize throughput. In [`CubeAPI/src/middleware/rate_limit.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/middleware/rate_limit.rs), a per-API-key token-bucket rate limiter inspects the `X-API-Key` header (falling back to IP-based `anonymous` tracking) and immediately returns **429 Too Many Requests** when quotas are exceeded, preventing abuse before requests reach business logic.

### Layer 2: Cube Proxy (High-Performance Reverse-Proxy)

The **Cube Proxy** operates as a minimal, high-performance reverse-proxy that exposes public endpoints and forwards them to the internal Cube API process. Defined in [`CubeAPI/src/main.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/main.rs), this component runs as a tiny Rust HTTP server using **tokio** for single-threaded, async connection multiplexing.

By default, the proxy listens on `0.0.0.0:3000` (configurable via `CUBE_API_URL`) and forwards all `/:path` traffic directly to the Cube API socket. This design avoids extra network hops, using the host's loopback interface for direct inter-process communication.

### Layer 3: Nginx Front-End (One-Click Deployment)

For production deployments, an **Nginx** container provides TLS termination, static asset serving for the Web UI, and reverse-proxy capabilities. The configuration in [`deploy/one-click/webui/nginx.conf`](https://github.com/TencentCloud/CubeSandbox/blob/main/deploy/one-click/webui/nginx.conf) exposes port **12088** for the human-facing interface while proxying `/cubeapi/` requests to the host-gateway (`host.docker.internal`).

The Nginx layer handles TLS certificate management and serves the compiled Web UI (`webui/dist`), offloading encryption overhead from the Rust API stack while maintaining a clean separation between static content and dynamic API traffic.

### Layer 4: Gateway Sub-Domain Isolation (Optional)

For advanced isolation scenarios, the architecture supports **per-sandbox sub-domains** using the format `<port>-<sandboxId>.<domain>`. When configured, the Web UI stores a `gatewayDomain` for each instance (see [`web/src/api/client.ts`](https://github.com/TencentCloud/CubeSandbox/blob/main/web/src/api/client.ts) lines 46-52), constructing URLs like `https://12088-<sandboxId>.cube.app/...`.

This pattern, configured through [`web/src/components/agents/AgentSettingsDialog.tsx`](https://github.com/TencentCloud/CubeSandbox/blob/main/web/src/components/agents/AgentSettingsDialog.tsx) (lines 56-66), requires a wildcard DNS record and TLS certificate but ensures complete traffic isolation per sandbox, preventing cross-tenant WebSocket interference and enabling per-sandbox rate limit enforcement.

## Request Flow and Rate Limiting

Understanding the request lifecycle clarifies how the CubeAPI gateway architecture maintains performance under high concurrency:

1. **Incoming request** arrives at **Nginx** (port 12088) and is reverse-proxied via `/cubeapi/` to the **Cube Proxy** (port 3000).
2. **Cube Proxy** forwards the request to the **Cube API** process over the host's loopback interface, avoiding NAT overhead.
3. **Cube API** extracts the `X-API-Key` header and checks the token-bucket rate limiter in [`CubeAPI/src/middleware/rate_limit.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/middleware/rate_limit.rs). Exceeded quotas trigger an immediate 429 response.
4. Passed requests dispatch through the Axum router in [`CubeAPI/src/routes.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/routes.rs) to the appropriate handler (e.g., sandbox creation, template listing).
5. For assistant-level gateways, the UI appends the sub-domain (`<port>-<sandboxId>.<gatewayDomain>`), allowing the backend to enforce per-sandbox limits based on the origin's API key.

## Design Rationale for High Concurrency

The architecture addresses specific high-concurrency challenges through targeted technical decisions:

- **High request volume**: **Async Rust (tokio)** combined with a single-threaded proxy eliminates thread-per-connection overhead, using non-blocking I/O to sustain millions of concurrent connections.
- **Latency reduction**: **Direct host-gateway routing** via Docker's `host.docker.internal` bridge eliminates network address translation hops, keeping internal latency sub-millisecond.
- **Tenant isolation**: **Per-sandbox sub-domains** with independent rate-limiting buckets ensure that a single tenant cannot saturate the global request queue, protecting multi-tenant workloads.
- **Operational security**: **Nginx** handles TLS termination and static file serving, offloading cryptographic operations from the high-performance Rust API layer while maintaining battle-tested SSL/TLS implementations.

## Practical Usage Examples

Access the gateway directly through the Cube Proxy for local development:

```bash

# Basic health check (no API key required)

curl http://localhost:3000/cubeapi/v1/health

# Authenticated request with rate-limit key

curl -H "X-API-Key: my-key-123" \
     http://localhost:3000/cubeapi/v1/sandboxes

```

When utilizing gateway domain isolation, route traffic through the sub-domain endpoint:

```bash
curl https://3000-<sandbox-id>.cube.app/cubeapi/v1/health

```

Both routes terminate at the same CubeAPI backend, but the sub-domain variant provides strict CORS isolation and WebSocket separation per sandbox instance.

## Summary

- **CubeSandbox's gateway** uses a four-layer architecture (Cube API, Cube Proxy, Nginx, sub-domain isolation) to handle high-concurrency REST traffic.
- **Rust and Axum** power the core API with async tokio networking, while **token-bucket rate limiting** in [`CubeAPI/src/middleware/rate_limit.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/middleware/rate_limit.rs) enforces per-key quotas.
- **Direct host-gateway routing** minimizes latency by avoiding NAT hops between the proxy and API layers.
- **Optional sub-domain isolation** (`<port>-<sandboxId>.<domain>`) enables per-sandbox origin separation and independent rate limiting.
- **Nginx front-end** handles TLS termination and static assets, offloading encryption from the high-throughput Rust stack.

## Frequently Asked Questions

### How does CubeAPI prevent a single tenant from overwhelming the gateway?

The gateway implements a **token-bucket rate limiter** in [`CubeAPI/src/middleware/rate_limit.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeAPI/src/middleware/rate_limit.rs) that tracks requests per `X-API-Key` header. When a key exceeds its configured quota, the middleware immediately returns HTTP 429 before the request reaches the Axum router, ensuring that abusive traffic never impacts other tenants or consumes compute resources.

### What makes the Cube Proxy more efficient than a traditional reverse-proxy?

The **Cube Proxy** runs as a minimal Rust service using **tokio** for single-threaded, async connection multiplexing. Unlike traditional thread-per-connection proxies, it maintains state for thousands of concurrent connections without context-switching overhead, forwarding traffic directly to the Cube API via the host's loopback interface to eliminate network hops.

### Why does the architecture include both a Cube Proxy and Nginx?

**Nginx** handles TLS termination and static file serving for the Web UI, leveraging battle-tested SSL implementations and disk I/O optimization. The **Cube Proxy** focuses solely on high-performance API request forwarding, avoiding the overhead of TLS handshakes and file system operations. This separation allows the Rust-based API layer to dedicate CPU cycles to business logic while Nginx manages edge connectivity.

### How does sub-domain isolation improve security for sandboxed environments?

Sub-domain isolation (configured via `gatewayDomain` in [`web/src/api/client.ts`](https://github.com/TencentCloud/CubeSandbox/blob/main/web/src/api/client.ts)) assigns each sandbox a unique origin (`<port>-<sandboxId>.<domain>`). This prevents cross-origin resource sharing (CORS) conflicts and ensures that WebSocket sessions for different sandboxes never share a domain, blocking cross-site scripting (XSS) vectors and enabling per-sandbox rate limit enforcement based on the unique sub-domain's API key.