# How the OpenSandbox Egress Sidecar Enforces Network Policies Using iptables

> Learn how the OpenSandbox egress sidecar enforces network policies with iptables. Discover how it uses CAP_NET_ADMIN to redirect DNS traffic and ensure sandbox security.

- Repository: [Alibaba/OpenSandbox](https://github.com/alibaba/OpenSandbox)
- Tags: internals
- Published: 2026-03-08

---

**The OpenSandbox egress sidecar uses CAP_NET_ADMIN privileges to install iptables REDIRECT rules that force all DNS traffic through a local proxy, while optionally leveraging nftables for IP-level filtering, ensuring the sandbox container cannot bypass network policies.**

OpenSandbox isolates untrusted user code inside Kubernetes Pods using a dual-container architecture: a restricted *sandbox* container runs the workload, while a privileged *egress sidecar* mediates all outbound traffic. According to the OpenSandbox source code, the sidecar enforces zero-trust egress policies by manipulating netfilter rules through iptables, effectively creating a network perimeter that the sandbox cannot reconfigure.

## Architecture Overview: Privilege Separation and Sidecar Injection

The enforcement model relies on strict capability separation between containers. In [`server/src/services/k8s/egress_helper.py`](https://github.com/alibaba/OpenSandbox/blob/main/server/src/services/k8s/egress_helper.py), the function `build_egress_sidecar_container()` constructs the sidecar specification with the **NET_ADMIN** capability added to its security context, while the sandbox container explicitly drops this capability.

This design ensures only the sidecar can modify iptables rules. The sandbox runs as a non-root user without network administration rights, preventing it from circumventing the egress controls. As noted in the OSEP design document ([`oseps/0001-fqdn-based-egress-control.md`](https://github.com/alibaba/OpenSandbox/blob/main/oseps/0001-fqdn-based-egress-control.md)), CAP_NET_ADMIN grants permission to modify network configuration—including iptables—regardless of the user ID, allowing the sidecar to run non-root yet manage packet filtering.

## Injecting Network Policies via Environment Variables

The sidecar receives its enforcement instructions through an environment variable rather than a configuration file. During Pod construction, `build_egress_sidecar_container()` JSON-encodes the `NetworkPolicy` object and injects it as **OPENSANDBOX_EGRESS_RULES**.

```python
from src.services.k8s.egress_helper import build_egress_sidecar_container

sidecar = build_egress_sidecar_container(
    egress_image="opensandbox/egress:v1.0.1",
    network_policy=NetworkPolicy(
        default_action="deny",
        egress=[NetworkRule(action="allow", target="pypi.org")]
    ),
)

# Results in securityContext: {"capabilities": {"add": ["NET_ADMIN"]}}

# and env: [{"name": "OPENSANDBOX_EGRESS_RULES", "value": "..."}]

```

The sidecar reads this variable at startup to determine which destinations are allowed or blocked. This approach keeps the policy lifecycle bound to the Pod specification while ensuring the sidecar has immediate access to rules without requiring Kubernetes API calls.

## DNS Interception Using iptables REDIRECT

At runtime, the sidecar launches a DNS proxy on `127.0.0.1:15353` and uses iptables to intercept all outbound DNS queries. In [`components/egress/pkg/iptables/redirect.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/pkg/iptables/redirect.go), the `SetupRedirect()` function installs NAT rules that transparently redirect UDP and TCP traffic destined for port 53 to the local proxy.

```go
func SetupRedirect(port int) error {
    cmds := [][]string{
        {"iptables", "-t", "nat", "-A", "OUTPUT", "-p", "udp", "--dport", "53",
            "-m", "mark", "--mark", constants.MarkHex, "-j", "RETURN"},
        {"iptables", "-t", "nat", "-A", "OUTPUT", "-p", "tcp", "--dport", "53",
            "-m", "mark", "--mark", constants.MarkHex, "-j", "RETURN"},
        {"iptables", "-t", "nat", "-A", "OUTPUT", "-p", "udp", "--dport", "53",
            "-j", "REDIRECT", "--to-port", strconv.Itoa(port)},
        {"iptables", "-t", "nat", "-A", "OUTPUT", "-p", "tcp", "--dport", "53",
            "-j", "REDIRECT", "--to-port", strconv.Itoa(port)},
    }
    // Execution logic follows...
}

```

This redirection guarantees that every DNS lookup—regardless of the sandbox container's resolver configuration—flows through the sidecar's proxy, enabling FQDN-based policy enforcement.

### Preventing Feedback Loops with SO_MARK

To avoid redirecting the sidecar's own DNS queries back to itself (which would create an infinite loop), the code marks outbound packets originating from the proxy. In [`components/egress/pkg/dnsproxy/proxy_linux.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/pkg/dnsproxy/proxy_linux.go), the proxy sets **SO_MARK** on its socket using a specific mark value defined in `constants.MarkHex`.

The iptables rules check this mark with `-m mark --mark <MARK> -j RETURN`, allowing marked packets to bypass the REDIRECT target and proceed directly to their destination. This mark-based bypass ensures the sidecar can resolve upstream domains to evaluate against the network policy without triggering its own interception logic.

## Layer-2 Enforcement with nftables

While iptables handles DNS redirection, OpenSandbox optionally enforces full IP and port-level filtering using nftables when available. If the sidecar detects the `nft` binary on the host, it loads a compiled nftables policy generated from the same `OPENSANDBOX_EGRESS_RULES` payload.

This Layer-2 filtering drops packets that violate the user-defined allow/deny list before they leave the Pod network namespace. The nftables approach complements iptables by providing fine-grained egress control beyond DNS interception, though iptables remains the baseline requirement for the DNS proxy mechanism.

## Graceful Degradation When iptables Is Unavailable

The sidecar implements defensive programming to handle environments where netfilter manipulation is impossible. In [`components/egress/main.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/main.go), the initialization code checks for `NET_ADMIN` capability and iptables binary availability before calling `SetupRedirect()`.

```go
func main() {
    // ... load policy, start DNS proxy ...
    if err := iptables.SetupRedirect(15353); err != nil {
        log.Fatalf("failed to install iptables redirect: %v", err)
    }
    log.Infof("iptables redirect configured (OUTPUT 53 -> 15353)")
}

```

If the sidecar lacks privileges or the iptables binary is missing, it logs a warning and disables enforcement rather than crashing the sandbox. This graceful degradation ensures workloads remain functional in restricted environments, albeit without egress policy enforcement.

## Summary

- **Privilege isolation**: The egress sidecar holds CAP_NET_ADMIN while the sandbox drops it, ensuring only the sidecar can modify iptables rules in [`server/src/services/k8s/egress_helper.py`](https://github.com/alibaba/OpenSandbox/blob/main/server/src/services/k8s/egress_helper.py).
- **Policy injection**: Network policies pass via the **OPENSANDBOX_EGRESS_RULES** environment variable, deserialized by the sidecar at startup.
- **DNS interception**: `iptables.SetupRedirect()` in [`components/egress/pkg/iptables/redirect.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/pkg/iptables/redirect.go) installs NAT REDIRECT rules forcing all port 53 traffic to a local proxy on port 15353.
- **Loop prevention**: The DNS proxy in [`components/egress/pkg/dnsproxy/proxy_linux.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/pkg/dnsproxy/proxy_linux.go) sets **SO_MARK** to exempt its own traffic from redirection.
- **Deep inspection**: Optional nftables provides Layer-2 packet filtering when available, complementing the iptables DNS interception.
- **Resilience**: The sidecar fails open with warnings if iptables setup fails, preventing sandbox crashes due to missing capabilities.

## Frequently Asked Questions

### Why does the egress sidecar need CAP_NET_ADMIN instead of running as root?

The sidecar requires **CAP_NET_ADMIN** to manipulate iptables rules and network configuration, but does not need full root privileges. As implemented in `alibaba/OpenSandbox`, this capability grants permission to modify packet filtering tables regardless of user ID, allowing the sidecar to run as a non-root user while still enforcing network policies. This follows the principle of least privilege by avoiding unnecessary UID 0 access.

### How does OpenSandbox prevent the sandbox container from bypassing iptables rules?

OpenSandbox prevents bypass by explicitly dropping **NET_ADMIN** (and other network capabilities) from the sandbox container's security context while adding them exclusively to the egress sidecar. Since iptables rules are namespaced to the Pod's network stack but require CAP_NET_ADMIN to modify, the sandbox cannot alter the REDIRECT rules or remove the enforcement hooks installed by the sidecar.

### What happens if the DNS proxy fails to start or iptables rules cannot be installed?

According to the source code in [`components/egress/main.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/main.go), the sidecar performs runtime checks before installing rules. If `iptables.SetupRedirect()` returns an error due to missing capabilities or binaries, the sidecar logs a fatal error and exits, preventing the sandbox from running without policy enforcement. However, the architecture supports graceful degradation scenarios where enforcement is disabled with warnings if explicitly configured for development environments.

### Does OpenSandbox support IPv6 egress filtering, or only IPv4?

The current implementation primarily targets IPv4, as evidenced by the iptables rules in [`components/egress/pkg/iptables/redirect.go`](https://github.com/alibaba/OpenSandbox/blob/main/components/egress/pkg/iptables/redirect.go) which use standard iptables (rather than ip6tables) for DNS redirection. The [`egress_helper.py`](https://github.com/alibaba/OpenSandbox/blob/main/egress_helper.py) file also injects sysctls to disable IPv6 (`net.ipv6.conf.all.disable_ipv6`), suggesting the initial design focuses on IPv4 enforcement while optionally preparing for dual-stack support through nftables in future iterations.