# How to Configure Per-Sandbox Network Policies with CubeVS

> Learn to configure per-sandbox network policies with CubeVS. Enforce Layer 7 egress rules per sandbox using eBPF maps for robust network security.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: how-to-guide
- Published: 2026-07-12

---

**CubeVS enforces Layer 7 egress rules per sandbox by serializing a `NetworkPolicy` object into the sandbox metadata, which the `network-agent` pushes to the CubeEgress control plane and enforces via eBPF maps in the TAP plugin.**

TencentCloud/CubeSandbox provides granular network isolation through CubeVS (Cube Virtual Sandbox), allowing administrators to define fine-grained outbound traffic rules that override cluster-wide defaults. Configuring these policies requires understanding the data flow from the SDK through the network-agent to the eBPF-based enforcement layer. This guide explains the exact mechanism using source file references from the repository.

## Understanding CubeVS Network Policy Architecture

CubeVS implements per-sandbox network policies through three coordinated components. The **network-agent** receives policy definitions from the Cube-API and maintains an in-memory `policyKnown` flag to track whether a specific sandbox has a custom policy. According to [`network-agent/internal/service/local_service.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/local_service.go), this flag prevents stale policies from being reapplied during sandbox recovery.

The **CubeEgress** control plane handles the actual rule distribution. When the network-agent pushes a policy, it sends a JSON payload to CubeEgress’s administrative endpoint, which then compiles the rules into eBPF programs. The **Cubelet TAP plugin** ([`Cubelet/network/plugin_tap.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/network/plugin_tap.go)) reads these rules from eBPF maps and intercepts every packet exiting the sandbox, applying allow or deny logic at the L7 level.

## Network Policy Structure and Fields

When creating a sandbox, the SDK accepts a `NetworkPolicy` description that maps to the internal representation in [`network-agent/internal/service/config.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/config.go). The policy supports the following fields:

- **`dns`** – Optional list of DNS servers for the sandbox to use.
- **`allow_out`** – CIDR ranges permitted for outbound traffic (comma-separated or list format).
- **`deny_out`** – CIDR ranges explicitly blocked from outbound traffic.
- **`net_type`** – Network classification (e.g., `public` or `private`).

If no policy is supplied, the system returns an empty map, instructing CubeEgress that the sandbox has no L7 policy and should fall back to cluster-wide defaults.

## Configuration Flow from SDK to eBPF

The policy enforcement path follows a strict sequence from user definition to kernel-level filtering:

1. **SDK Serialization** – The Go, Python, or Node.js SDK encodes the `NetworkPolicy` into the sandbox creation request’s metadata field.
2. **API Processing** – Cube-API forwards the metadata to the network-agent running on the target node.
3. **Config Building** – The agent parses the metadata into an internal `Config` struct in [`network-agent/internal/service/config.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/config.go), setting `policyKnown` to true.
4. **Policy Push** – The agent’s [`cubeegress_push.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cubeegress_push.go) issues a `PUT /admin/v1/policies/<sandbox_ip>` request with a JSON body defined in [`network-agent/internal/cubeegress/wire.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/cubeegress/wire.go).
5. **eBPF Installation** – CubeEgress installs the rules into eBPF maps that the TAP driver references.
6. **Runtime Enforcement** – The Cubelet TAP plugin ([`Cubelet/network/plugin_tap.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/network/plugin_tap.go)) checks every egress packet against the loaded policy, dropping connections that match a `deny_out` rule or fail to match an `allow_out` rule.

## Implementing Per-Sandbox Policies with SDKs

### Go SDK

The Go SDK defines the policy structure in [`sdk/go/policy.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/go/policy.go). Pass the serialized policy via the `metadata` map when calling `Sandbox.Create`:

```go
import (
    "github.com/TencentCloud/CubeSandbox/sdk/go"
    "github.com/TencentCloud/CubeSandbox/sdk/go/policy"
)

func main() {
    client := cubesandbox.NewClient("http://cube-api.example.com")
    
    netPol := policy.NetworkPolicy{
        AllowOut: []string{"10.0.0.0/8"},
        DenyOut:  []string{"0.0.0.0/0"},
    }
    
    sandbox, err := client.Sandbox.Create(
        cubesandbox.SandboxCreateRequest{
            Image:    "ubuntu:20.04",
            Metadata: map[string]string{"network-policy": netPol.String()},
        })
    // The sandbox now runs with the specified egress restrictions
}

```

### Python SDK

In [`sdk/python/cubesandbox/_policy.py`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/python/cubesandbox/_policy.py), the `NetworkPolicy` class provides the same functionality:

```python
from cubesandbox import CubeSandbox, NetworkPolicy

cs = CubeSandbox("http://cube-api.example.com")
policy = NetworkPolicy(
    allow_out=["10.0.0.0/8"],
    deny_out=["0.0.0.0/0"]
)

sandbox = cs.sandbox.create(
    image="ubuntu:20.04",
    metadata={"network-policy": policy.to_json()}
)

```

### Node.js SDK

The TypeScript implementation in [`sdk/node/src/policy.ts`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/node/src/policy.ts) exposes a similar interface:

```javascript
import { CubeSandbox } from "@tencentcloud/cubesandbox";
import { NetworkPolicy } from "@tencentcloud/cubesandbox/policy";

const cs = new CubeSandbox("http://cube-api.example.com");
const policy = new NetworkPolicy({
  allowOut: ["10.0.0.0/8"],
  denyOut:  ["0.0.0.0/0"]
});

const sandbox = await cs.sandbox.create({
  image: "ubuntu:20.04",
  metadata: { "network-policy": policy.toJSON() }
});

```

### Legacy Metadata Key

Older clients may use the simplified `"network-policy"` string value. The network-agent treats values like `"deny-all"` as shortcuts and automatically expands them into full policies that block all outbound traffic. This behavior is tested in [`sdk/python/tests/test_sandbox.py`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/python/tests/test_sandbox.py) where `meta = {"network-policy": "deny-all"}` is validated.

## Runtime Enforcement and Cluster Defaults

The Cubelet TAP plugin reads node-level annotations to establish baseline behaviors when no per-sandbox policy exists. As implemented in [`Cubelet/network/plugin_tap.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/network/plugin_tap.go), the plugin checks for constants defined in [`Cubelet/pkg/constants/const.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/pkg/constants/const.go):

- **`MasterAnnotationNetworkPolicyBlockAll`** – When set to `"true"`, blocks all egress by default.
- **`MasterAnnotationNetworkPolicyAllowPublicServices`** – When set to `"true"`, permits traffic to public service endpoints.
- **`MasterAnnotationNetworkPolicyDefault`** – Applies the repository-wide default policy.

Per-sandbox policies always **override** these defaults because the network-agent only pushes a configuration to CubeEgress when `policyKnown` is true. If a sandbox is restored after a node reboot and the recovered state lacks a known policy, the agent logs a warning and leaves the policy untouched, preventing accidental application of stale rules.

## Summary

- **CubeVS** enforces per-sandbox egress rules through the `network-agent`, **CubeEgress**, and **Cubelet TAP** plugin chain.
- The `NetworkPolicy` struct accepts `allow_out`, `deny_out`, `dns`, and `net_type` fields to define traffic rules.
- Policies are serialized into metadata during sandbox creation and pushed to CubeEgress via `PUT /admin/v1/policies/<sandbox_ip>` as defined in [`wire.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/wire.go).
- **Go**, **Python**, and **Node.js** SDKs provide native policy objects that handle JSON serialization automatically.
- Per-sandbox policies override cluster defaults because the `policyKnown` flag ensures only explicitly defined policies are pushed to the eBPF enforcement layer.

## Frequently Asked Questions

### What happens if I don't specify a network policy when creating a sandbox?

If no `NetworkPolicy` is provided, the SDK sends an empty configuration map. CubeEgress interprets this as an absence of L7 policy, causing the sandbox to inherit cluster-wide defaults from Cubelet annotations (such as block-all or allow-public-services modes).

### How do per-sandbox policies interact with cluster-wide network defaults?

Per-sandbox policies take precedence over cluster defaults. The network-agent only pushes a policy to CubeEgress when the `policyKnown` flag is true. If this flag is false, the agent does not transmit any rules, allowing the TAP plugin to apply the default behaviors configured via node annotations.

### What is the JSON format expected by the CubeEgress admin API?

According to [`network-agent/internal/cubeegress/wire.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/cubeegress/wire.go), the payload must include a `policy_id` (set to the sandbox IP) and a `rules` array containing objects with `id` (either `"allow_out"` or `"deny_out"`) and `cidr` arrays listing the IP ranges.

### Can the network-agent recover policies after a node restart?

During sandbox restoration, the network-agent checks its internal state in [`network-agent/internal/service/local_service.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/local_service.go). If the recovered state lacks a known policy (`policyKnown` is false), the agent does not replay any configuration and logs a warning, ensuring that unknown or stale policies are not accidentally enforced.