# How CubeEgress Secures Outbound Traffic in CubeSandbox: A Four-Stage Defense

> Discover how CubeEgress secures outbound traffic using a four-stage defense. Learn about kernel-level eBPF filtering, TPROXY redirection, and an OpenResty L7 proxy for robust security in CubeSandbox.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: deep-dive
- Published: 2026-07-08

---

**CubeEgress secures outbound traffic through a four-stage defense-in-depth chain that combines kernel-level eBPF filtering, TPROXY redirection, and an OpenResty-based L7 proxy to enforce fine-grained policies while keeping secrets out of the sandbox.**

CubeEgress is the per-host transparent egress gateway in the TencentCloud/CubeSandbox repository that enforces Layer 7 security policies on every outbound request. Unlike simple firewall rules, it implements an inline proxy architecture that intercepts TCP traffic destined for ports 80 and 443, allowing administrators to define precise allow-lists based on host, SNI, path, and HTTP method while ensuring sensitive credentials never reach the sandbox environment.

## The Four-Stage Security Architecture

### Stage 1: Policy Definition via CubeMaster

Users express security policies through the SDK or API, which CubeMaster stores as protobuf `CubeNetworkConfig` messages. These configurations include fields like `allow_out`, `allow_out_v2`, and detailed `rules` arrays that specify allowed CIDRs, DNS entries, and granular L7 match criteria including host, SNI, path, scheme, and action types (audit, inject, deny). The policies reside in the template database before being pushed to individual nodes.

### Stage 2: Kernel-Side Enforcement with eBPF

The `CubeVS` eBPF program extracts only the network-reachable components of each L7 rule—specifically `match.host` and `match.sni`—and populates `allow_out_v2` with an `L7_REQUIRED` flag. When a sandbox initiates a TCP connection to ports 80 or 443 matching a flagged entry, the kernel marks the socket for inspection and redirects it via **TPROXY** rules to the CubeEgress container. Non-HTTP traffic bypasses the proxy entirely, following a fast-path NAT route. This selective capture logic is documented in [`docs/guide/network-policy.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/guide/network-policy.md) and implemented in the eBPF map layout within the source code.

### Stage 3: L7 Proxy Enforcement

CubeEgress runs an **OpenResty** (nginx + Lua) proxy that receives the redirected TCP streams on the host network. The Lua modules—specifically [`CubeEgress/lua/policy.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/policy.lua) and [`CubeEgress/lua/access_phase.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/access_phase.lua)—parse HTTP and HTTPS traffic (extracting SNI for TLS) and apply the complete rule set. When a request violates policy, the proxy returns an immediate HTTP 403 without contacting the upstream service, guaranteeing that no data leaves the cluster. Valid requests proceed only after matching an allow rule.

### Stage 4: Auditing and Credential Protection

Every request processed by CubeEgress generates structured JSON logs at `/data/log/cube-egress/access.jsonl`, capturing timestamps, sandbox IDs, rule hits, and redacted request/response data. The [`CubeEgress/lua/audit.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/audit.lua) module handles this logging. Critically, secrets are never exposed to the sandbox: the proxy performs **credential injection** by adding required headers (e.g., `Authorization`, `x-api-key`) after the sandbox dispatches the request. The per-sandbox CA used for TLS interception is generated by [`CubeEgress/gen-ca.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/gen-ca.sh) and baked into the container image, with signing logic residing in [`CubeEgress/lua/cert_signer.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/cert_signer.lua).

## Key Security Guarantees

CubeEgress provides several hardened security guarantees:

- **Fail-Closed Startup**: Until the policy fully loads and `bootstrap_status` equals `"ready"`, CubeEgress rejects all non-audit traffic with a 403 status, eliminating windows where outbound traffic might bypass controls. This behavior is noted in [`docs/zh/changelog/v0.5.0.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/zh/changelog/v0.5.0.md).

- **Selective Traffic Capture**: Only TCP flows to ports 80 and 443 with the `L7_REQUIRED` flag enter the proxy. All other traffic, including UDP and non-HTTP ports, follows kernel NAT directly, minimizing performance overhead.

- **Policy-Driven Routing**: The eBPF maps `egress_sessions` and `ingress_sessions` track connection state, ensuring only packets belonging to allowed sessions forward through the data plane, preventing spoofed packets from bypassing the proxy. See the architecture diagram in [`docs/zh/architecture/network.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/zh/architecture/network.md).

- **Credential Isolation**: Secrets stored in CubeMaster or external vaults are injected by the proxy at the final hop, satisfying the requirement that sensitive credentials never leave the host environment.

## Configuration Examples

The following examples demonstrate how to define L7 rules that leverage CubeEgress's credential injection and auditing capabilities.

**Python SDK:**

```python

# Define an L7 rule that injects an API key without exposing it to the sandbox

sandbox = client.sandbox.create(
    name="my-sandbox",
    allow_internet_access=True,
    network={
        "rules": [{
            "match": {"host": "api.openai.com"},
            "action": {"inject": {"headers": {"Authorization": "Bearer ${API_KEY}"}}}
        }]
    })

```

**Go SDK:**

```go
// Same rule with audit logging enabled
network := cubenet.NetworkConfig{
    Rules: []cubenet.Rule{{
        Match: cubenet.Match{Host: "api.anthropic.com"},
        Action: cubenet.Action{Inject: &cubenet.Inject{
            Headers: map[string]string{"x-api-key": "${ANTHROPIC_API_KEY}"},
        }, Audit: true},
    }},
}
client.CreateSandbox(ctx, "my-sandbox", network, true)

```

**Verification:**

```bash

# Verify that requests without matching rules are blocked

curl -s https://example.com  # Returns 403 Forbidden - CubeEgress

```

## Core Implementation Files

| Component | File Path | Purpose |
|-----------|-----------|---------|
| Proxy Configuration | [`CubeEgress/nginx.conf`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/nginx.conf) | OpenResty entry-point configuration |
| Policy Engine | [`CubeEgress/lua/policy.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/policy.lua) | Core rule matching (allow/deny/audit/inject) |
| Access Handler | [`CubeEgress/lua/access_phase.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/access_phase.lua) | HTTP/HTTPS request processing logic |
| Audit Logger | [`CubeEgress/lua/audit.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/audit.lua) | JSONL logging to `/data/log/cube-egress/access.jsonl` |
| Certificate Authority | [`CubeEgress/gen-ca.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/gen-ca.sh) | Root CA generation for per-sandbox leaf certificates |
| TLS Signer | [`CubeEgress/lua/cert_signer.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/cert_signer.lua) | Generates per-sandbox TLS certificates |
| TPROXY Setup | [`CubeEgress/scripts/cube-proxy-iptables-init.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/scripts/cube-proxy-iptables-init.sh) | Kernel rules for L7 traffic redirection |

## Summary

- CubeEgress implements a **four-stage chain** spanning policy definition, eBPF kernel filtering, OpenResty L7 proxying, and secure auditing.
- Traffic to ports 80/443 with the **`L7_REQUIRED`** flag is transparently redirected via **TPROXY** to the CubeEgress container.
- The **fail-closed** startup behavior ensures no traffic bypasses policy enforcement during initialization.
- **Credential injection** ensures secrets never reach the sandbox, with headers added by the proxy after the request originates.
- Implementation relies on specific Lua modules in `CubeEgress/lua/` and eBPF programs that maintain session state in `egress_sessions` maps.

## Frequently Asked Questions

### What happens if CubeEgress fails to load its policy?

If the `bootstrap_status` is not `"ready"`, CubeEgress operates in fail-closed mode, returning HTTP 403 Forbidden for all non-audit traffic. This prevents any outbound connections from proceeding until the security policy is fully loaded and validated, as implemented in the OpenResty startup logic.

### How does CubeEgress handle HTTPS traffic without breaking TLS?

CubeEgress generates per-sandbox leaf certificates signed by a dedicated CA created via [`CubeEgress/gen-ca.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/gen-ca.sh). The proxy uses SNI (Server Name Indication) to inspect the target host for routing decisions, terminating TLS when necessary for L7 inspection and then re-encrypting traffic to the upstream. The certificate signing logic resides in [`CubeEgress/lua/cert_signer.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/cert_signer.lua).

### Can non-HTTP traffic bypass the CubeEgress proxy?

Yes. Only TCP traffic destined for ports 80 and 443 that matches an `L7_REQUIRED` flag in the eBPF maps is redirected via TPROXY to the OpenResty proxy. All other traffic—including UDP, plain IP, and TCP on non-standard ports—follows the kernel NAT fast-path and never touches the user-space proxy, as documented in [`docs/guide/network-policy.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/guide/network-policy.md).

### Where are the audit logs stored and what do they contain?

Audit logs are written as structured JSON Lines to `/data/log/cube-egress/access.jsonl`. Each entry includes the timestamp, sandbox ID, matched rule, destination host, and optional redacted request/response details. The [`CubeEgress/lua/audit.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeEgress/lua/audit.lua) module formats these entries, ensuring comprehensive observability without exposing injected credentials.