# How Container-to-Container Networking Works Over vmnet in the Apple Container Project

> Discover how Apple's container project implements container-to-container networking over vmnet. Learn about the XPC helper service and runtime helper for NAT or host-only interfaces.

- Repository: [Apple/container](https://github.com/apple/container)
- Tags: internals
- Published: 2026-07-07

---

**Apple's `container` project implements container-to-container networking over the macOS vmnet framework using a three-layer architecture where an XPC helper service creates and serializes vmnet network references, passing them to a runtime helper that constructs NAT or host-only interfaces for container attachment.**

Container-to-container networking over vmnet in the Apple `container` repository leverages the macOS vmnet framework to provide isolated virtual networks for containers. The implementation coordinates between CLI commands, a privileged XPC helper service, and a runtime helper to translate high-level network configurations into kernel-level vmnet objects. This architecture enables seamless container-to-container communication through either NAT or host-only networking modes.

## Three-Layer Architecture

The networking stack consists of three distinct layers that communicate via XPC messages:

- **CLI Layer**: Parses `container network create` commands in [`Sources/ContainerCommands/Network/NetworkCreate.swift`](https://github.com/apple/container/blob/main/Sources/ContainerCommands/Network/NetworkCreate.swift) and constructs a `NetworkConfiguration` object specifying the mode (NAT or host-only), subnets, and the plugin name (`container-network-vmnet`).

- **XPC Helper (`container-network-vmnet`)**: Runs as a privileged XPC service defined in [`Sources/Plugins/NetworkVmnet/NetworkVmnetHelper.swift`](https://github.com/apple/container/blob/main/Sources/Plugins/NetworkVmnet/NetworkVmnetHelper.swift). It implements the `ContainerNetworkServer` API and manages the lifecycle of vmnet networks through the `ReservedVmnetNetwork` class.

- **Runtime Helper (`container-runtime-linux`)**: Receives serialized vmnet references from the XPC helper and constructs concrete network interfaces. The `NonisolatedInterfaceStrategy` class in [`Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift`](https://github.com/apple/container/blob/main/Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift) handles deserialization and interface creation.

## Configuring the vmnet Network

When a user creates a network, the helper translates the `NetworkConfiguration` into a vmnet-compatible structure inside [`Sources/Services/NetworkVmnet/Server/ReservedVmnetNetwork.swift`](https://github.com/apple/container/blob/main/Sources/Services/NetworkVmnet/Server/ReservedVmnetNetwork.swift).

### Mode Selection and DHCP Configuration

The `ReservedVmnetNetwork` class selects the operating mode based on the configuration:

- **NAT Mode**: Uses `.VMNET_SHARED_MODE` to allow containers to access external networks through host NAT.
- **Host-Only Mode**: Uses `.VMNET_HOST_MODE` (set via the `--internal` flag) for isolated container-to-container communication.

The implementation explicitly disables the vmnet DHCP server because the helper manages its own address allocation:

```swift
// In ReservedVmnetNetwork.swift (Lines 16-18)
// Disable DHCP - we manage our own address space
vmnet_network_configuration_set_dhcp_enabled(configuration, false)

```

### Subnet and Interface Configuration

If the user specifies a subnet, the helper configures the vmnet network object using low-level vmnet APIs:

```swift
// In ReservedVmnetNetwork.swift (Lines 30-34)
if let ipv4Subnet = configuration.ipv4Subnet {
    var addr = ipv4Subnet.in_addr
    vmnet_network_configuration_set_ipv4_subnet(vmnetConfig, &addr)
}

// IPv6 configuration (Lines 40-44)
if let ipv6Prefix = configuration.ipv6Prefix {
    var addr = ipv6Prefix.in6_addr
    vmnet_network_configuration_set_ipv6_prefix(vmnetConfig, &addr, prefixLength)
}

```

The network is finally instantiated with `vmnet_network_create` and the resulting `vmnet_network_ref` is stored in the network's state (Lines 54-58).

## Serializing Network References for Container Attachment

To attach a container to an existing network, the XPC helper must transfer the vmnet reference to the runtime process. Since `vmnet_network_ref` objects cannot be passed directly across process boundaries, the helper serializes the reference:

```swift
// In ReservedVmnetNetwork.swift (Lines 92-99)
let serialization = vmnet_network_copy_serialization(networkRef)
// ... add to XPC message ...

```

This serialized data travels through the `ContainerNetworkServer` API to the runtime helper, which reconstructs the reference on the host side using `vmnet_network_create_with_serialization`.

## Runtime Interface Construction

Upon receiving the XPC message, the runtime helper in [`Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift`](https://github.com/apple/container/blob/main/Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift) deserializes the reference and constructs the container's network interface:

```swift
// In NonisolatedInterfaceStrategy.swift (Lines 35-53)
let additionalData: XPCMessage = // received from XPC helper
var status: vmnet_return_t = VMNET_SUCCESS
let networkRef = vmnet_network_create_with_serialization(
    additionalData.underlying, 
    &status
)

let interface = NATNetworkInterface(
    ipv4Address: attachment.ipv4Address,
    ipv4Gateway: attachment.ipv4Gateway,
    reference: networkRef,
    macAddress: attachment.macAddress,
    mtu: attachment.mtu ?? 1280
)

```

The `NATNetworkInterface` stores the container's IPv4 address, optional gateway, MAC address, and the vmnet reference, enabling the container's virtual NIC to communicate over the vmnet bridge.

## Network Modes and IP Allocation

The implementation supports two primary networking modes with distinct address allocation strategies:

- **NAT Mode**: The first container attached to the network receives the gateway address, with subsequent containers receiving the next available IPs in the subnet. This mode enables outbound connectivity through the host's network stack.

- **Host-Only Mode**: Uses the same IP allocation mechanism but restricts traffic to the host and other containers on the same network, providing complete isolation from external networks.

## Platform Requirements and Limitations

Because the vmnet framework requires APIs introduced in macOS 26, the helper code is compiled with `@available(macOS 26, *)` guards. On macOS 15, the vmnet framework only supports isolated host-only networks, which limits functionality as documented in [`docs/technical-overview.md`](https://github.com/apple/container/blob/main/docs/technical-overview.md) (Lines 36-38 and 65-73).

## Practical Usage Examples

Create a NAT network for general container-to-container communication:

```bash
container network create \
    --label env=dev \
    --option mtu=1500 \
    my-net

```

Create an isolated host-only network for secure internal communication:

```bash
container network create \
    --internal \
    --subnet 10.0.0.0/24 \
    host-only-net

```

Inspect the network configuration to verify the assigned subnets and gateway:

```bash
container network inspect my-net

```

## Summary

- **Architecture**: Container-to-container networking over vmnet uses a CLI command layer, an XPC helper service (`container-network-vmnet`), and a runtime helper (`container-runtime-linux`).
- **Configuration**: The `ReservedVmnetNetwork` class in [`Sources/Services/NetworkVmnet/Server/ReservedVmnetNetwork.swift`](https://github.com/apple/container/blob/main/Sources/Services/NetworkVmnet/Server/ReservedVmnetNetwork.swift) configures vmnet modes (NAT or host-only), disables DHCP, and sets IPv4/IPv6 subnets.
- **Serialization**: Network references are serialized using `vmnet_network_copy_serialization` for XPC transport and deserialized with `vmnet_network_create_with_serialization` in the runtime.
- **Interface Creation**: The runtime constructs `NATNetworkInterface` objects in [`Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift`](https://github.com/apple/container/blob/main/Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift) to connect containers to the vmnet bridge.
- **Platform Support**: Full functionality requires macOS 26 or later; macOS 15 supports only isolated host-only networks.

## Frequently Asked Questions

### What is vmnet and why does Apple Container use it?

The vmnet framework is a macOS system API that provides virtual network infrastructure for guest operating systems and containers. Apple Container uses vmnet because it provides kernel-level network isolation and bridging without requiring root privileges for the main container process, delegating privileged network operations to a dedicated XPC helper service.

### What is the difference between NAT and host-only mode in Container networking?

NAT mode (`.VMNET_SHARED_MODE`) allows containers to communicate with each other while also accessing external networks through the host's NAT translation. Host-only mode (`.VMNET_HOST_MODE`) restricts communication to the host machine and other containers on the same network, providing complete isolation from the external network. NAT mode assigns the first container the gateway address, while host-only mode assigns addresses from the same subnet without NAT translation.

### Why does the implementation require macOS 26 or later?

The vmnet framework APIs used for creating shared networks and serializing network references (`vmnet_network_create`, `vmnet_network_copy_serialization`) are only available starting with macOS 26. The code explicitly marks these features with `@available(macOS 26, *)`. On macOS 15, the framework only supports isolated host-only networks, which limits container-to-container networking capabilities.

### How does IP address allocation work without DHCP?

The `container-network-vmnet` XPC helper manages its own IP address pool rather than relying on vmnet's DHCP server. When the network is created, the helper calculates available addresses from the configured subnet. For NAT networks, it reserves the first usable address for the gateway and assigns subsequent addresses to containers as they attach. For host-only networks, it follows the same allocation pattern without configuring a gateway.