How apple/container Uses the Virtualization Framework for Lightweight VM-Based Containers

apple/container leverages Apple's Virtualization framework to run Linux containers inside isolated, lightweight VMs, requiring Apple Silicon M3+ and macOS 15+ for nested virtualization support.

The apple/container project implements a modern container runtime that uses macOS's native Virtualization framework to isolate Linux workloads in dedicated virtual machines. By wrapping containers in lightweight VMs rather than relying solely on process-based isolation, the project provides hardware-level sandboxing while maintaining near-native performance. This architecture requires specific hardware capabilities and involves a multi-stage bootstrap process from CLI configuration to runtime initialization.

Prerequisites: Nested Virtualization Checks

Before spinning up a VM-based container, the system validates host capabilities. In Sources/ContainerCommands/Machine/MachineCapabilities.swift, the requireNestedVirtualizationSupported() function checks whether the hardware supports nested virtualization by querying VZGenericPlatformConfiguration.isNestedVirtualizationSupported.

This check is true only on Apple Silicon M3 or newer running macOS 15+, ensuring the underlying kernel has CONFIG_KVM=y support enabled.

// MachineCapabilities.swift
static func requireNestedVirtualizationSupported() throws {
    guard VZGenericPlatformConfiguration.isNestedVirtualizationSupported else {
        throw ContainerizationError(
            .unsupported,
            message: "nested virtualization is not supported on this host (requires Apple Silicon M3+ and macOS 15+)"
        )
    }
}

The CLI invokes this guard immediately when the user requests virtualization mode, preventing unsupported hardware from attempting to boot a VM container.

CLI Configuration with the --virtualization Flag

The container create command exposed in Sources/ContainerCommands/Machine/MachineCreate.swift accepts a --virtualization flag that triggers the VM-based container path. When parsing arguments, the command first runs the capability check, then persists the configuration to the machine's boot configuration.

// MachineCreate.swift (excerpt)
@Flag(name: .long, help: "Enable nested virtualization …")
public var virtualization: Bool = false

public func run() async throws {
    if virtualization {
        try MachineCapabilities.requireNestedVirtualizationSupported()
    }
    // ... configuration logic ...
    let bootConfig = try defaultConfig.with(
        [
            "virtualization": virtualization ? "true" : nil,
            // ... other config ...
        ].compactMapValues { $0 }
    )
}

This boot configuration is later consumed by the runtime service during the bootstrap phase, determining whether to initialize a VZVirtualMachineManager or fall back to alternative isolation mechanisms.

Bootstrapping the Virtual Machine Runtime

When the container daemon receives a bootstrap request, Sources/Services/RuntimeLinux/Server/RuntimeService.swift orchestrates the VM creation. The core component is the VZVirtualMachineManager, a thin wrapper around the Virtualization framework that owns the VM lifecycle.

The bootstrap sequence follows these steps:

1. Load kernel and filesystem: The manager receives the kernel image (bundle.kernel) and initial filesystem mount (bundle.initialFilesystem.asMount)
2. Initialize LinuxContainer: The LinuxContainer class (provided by the Containerization dependency) configures CPU count, memory allocation, network interfaces, and boot hooks using the manager
3. Start the VM: Calling container.create() internally invokes VZVirtualMachine.start() and launches the guest agent inside the VM

// RuntimeService.swift (bootstrap excerpt)
let vmm = VZVirtualMachineManager(
    kernel: kernel,
    initialFilesystem: bundle.initialFilesystem.asMount,
    rosetta: config.rosetta,
    logger: self.log
)
// ... interface configuration ...
let container = try LinuxContainer(id, rootfs: rootfs, vmm: vmm, logger: self.log) { czConfig in
    try Self.configureContainer(czConfig: &czConfig, config: config,
                                   dynamicEnv: dynamicEnv, log: self.log)
    czConfig.interfaces = interfaces
    // ... additional configuration ...
}
try await container.create()

Networking and Filesystem Optimization

Network interfaces are attached through vmnet (implemented in Sources/Services/RuntimeLinux/Server/NonisolatedInterfaceStrategy.swift), which creates a vmnet_network_t reference. This reference is then attached to the VM through the Virtualization framework's VZNetworkDeviceAttachment APIs handled internally by the container runtime.

For filesystem performance, Sources/ContainerResource/Container/Filesystem.swift configures the rootfs mount using cached mode. The code specifically enables CachedMode = .on to avoid Linux filesystem issues when running under virtualization, significantly improving I/O performance compared to uncached passthrough.

Summary

• Hardware requirements: VM-based containers require Apple Silicon M3+ and macOS 15+ with CONFIG_KVM=y kernel support, verified via VZGenericPlatformConfiguration.isNestedVirtualizationSupported
• CLI integration: The --virtualization flag in MachineCreate.swift triggers nested virtualization checks and stores the configuration for the runtime
• Lifecycle management: VZVirtualMachineManager handles the VM lifecycle, instantiated in RuntimeService.swift and controlled through the LinuxContainer class
• Performance optimizations: Filesystems use cached mode to mitigate Linux-FS performance issues under virtualization, while networking leverages vmnet paired with VZNetworkDeviceAttachment
• Isolation model: Each container runs in a dedicated Linux VM sharing the host kernel architecture but isolated by the Virtualization framework's hardware-level sandbox

Frequently Asked Questions

What hardware is required to run VM-based containers in apple/container?

The Virtualization framework requires Apple Silicon M3 or newer running macOS 15 or later. This hardware combination provides the necessary CONFIG_KVM=y kernel support for nested virtualization. The MachineCapabilities.requireNestedVirtualizationSupported() function explicitly checks VZGenericPlatformConfiguration.isNestedVirtualizationSupported and throws an error on unsupported hardware.

How does the Virtualization framework improve container isolation compared to traditional methods?

Unlike process-based containers that share the host kernel, apple/container uses the Virtualization framework to run each container in a dedicated Linux VM with its own kernel image and root filesystem. This provides hardware-level sandboxing through the VZVirtualMachine APIs, isolating the container from the host operating system and other containers while maintaining compatibility with standard Linux container images.

What is the role of VZVirtualMachineManager in the container lifecycle?

VZVirtualMachineManager acts as the bridge between the container runtime and Apple's Virtualization framework. Instantiated in RuntimeService.swift, it accepts the kernel image and initial filesystem mounts, then coordinates with the LinuxContainer class to configure CPU, memory, and network resources. When container.create() is called, the manager executes VZVirtualMachine.start() to boot the guest Linux system and initialize the container's execution environment.

Why does apple/container use cached mode for the Linux filesystem?

According to comments in Filesystem.swift, cached mode is enabled to avoid performance and compatibility issues with Linux filesystems when running under virtualization. By setting CachedMode = .on, the container runtime ensures that filesystem operations are buffered appropriately for the Virtualization framework, preventing synchronization issues while maintaining acceptable I/O throughput for container workloads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}