How to Use Custom Linux Kernels with Apple Container for Specific Kernel Features

Apple Container lets you override the default kata-containers kernel with any architecture-matching Linux binary by using the container system kernel set command for global defaults or the --kernel flag in machine configurations, enabling specialized features like nested virtualization that require compile-time flags such as CONFIG_KVM=y.

Apple Container runs Linux-based guests inside lightweight VMs on macOS, automatically downloading a kernel that matches the host version. When your workloads require specific compile-time features—such as CONFIG_KVM=y for nested virtualization or custom hardware drivers—you can configure custom Linux kernels with container to replace the default runtime binary.

Architecture Overview

Apple Container manages kernel selection through three primary components that bridge user configuration with the hypervisor.

ContainerSystemConfig

The ContainerSystemConfig structure in Sources/ContainerPersistence/ContainerSystemConfig.swift stores the default kernel information used by the runtime, including binaryPath and url fields. You can override these defaults via the CLI or configuration files to change the system-wide kernel behavior.

MachineConfig

Per-machine settings are stored in Sources/ContainerPersistence/MachineConfig.swift, which parses the kernel field from TOML configuration files. When you create or update a machine, this value is read and passed to the runtime, allowing different machines to run different kernel versions simultaneously.

RuntimeService

The RuntimeService in Sources/Services/RuntimeLinux/Server/RuntimeService.swift initializes the VM by reading RuntimeConfiguration.kernel.path and feeding the specified binary directly to the hypervisor. This ensures the custom kernel is loaded before the guest OS boots.

When to Use a Custom Kernel

You need a custom Linux kernel with container when the default kata-containers build lacks specific capabilities required by your workload:

• Nested virtualization: Running KVM inside the container requires a kernel compiled with CONFIG_KVM=y, which the default kernel disables for security.
• Hardware drivers: Accessing specialized peripherals or security modules requires kernel drivers not included in the default build.
• Kernel testing: Validating patches or specific kernel versions before upstreaming them to production environments.

Prerequisites

Before loading a custom kernel, Apple Container validates two requirements:

1. Architecture match: The binary must target the host CPU architecture (arm64 for Apple Silicon, amd64 for Intel). The CLI defaults to arm64 on Apple Silicon Macs.
2. Feature availability: While the runtime does not validate CONFIG_ flags at boot, the kernel must be compiled with the features your workload expects (e.g., CONFIG_KVM=y for nested virtualization on Apple Silicon M3+ and macOS 15+).

Configuring Custom Linux Kernels

You can configure kernels at the system level (affecting all new machines) or per-machine (overriding the system default).

Set a System-Wide Default Kernel

Use container system kernel set to install a custom binary as the default for all newly created machines:

# Install a custom kernel from a local binary

container system kernel set \
    --binary ./vmlinux \
    --force

The --binary flag accepts a raw kernel file or a path within a tarball when combined with --tar. This updates the binaryPath in ContainerSystemConfig.

Configure Per-Machine Kernel Overrides

To use a specific kernel for an individual machine without changing the system default:

# Create a new machine with a custom kernel

container machine create \
    --name dev \
    --virtualization \
    --kernel ./vmlinux-kvm \
    alpine:latest

# Or update an existing machine

container machine set \
    -n dev \
    virtualization=true \
    kernel=/opt/kernels/vmlinux-kvm

The kernel value is stored in the machine's TOML configuration at ~/.container/machines/<name>/boot-config.json.

Verify the Kernel Inside the Container

After starting the container, verify the running kernel matches your custom binary:

container run --name test --virtualization \
    --kernel ./vmlinux-kvm \
    ubuntu:latest \
    sh -c "uname -r && dmesg | grep kvm"

The uname -r output should display the version string of your custom kernel, confirming that RuntimeService loaded the correct binary.

Revert to the Default Kernel

To clear a machine-specific override and return to the system default:

container machine set -n dev kernel=

Restart the machine to apply the change.

Practical Examples

Enabling Nested Virtualization

To run nested KVM workloads, supply a kernel compiled with CONFIG_KVM=y:

# Assuming vmlinux-kvm is built with CONFIG_KVM=y

container machine create \
    -n nested \
    --virtualization \
    --kernel ./vmlinux-kvm \
    ubuntu:latest

Switching Kernels on Existing Machines

You can change kernels without destroying the machine's persistent storage:

container machine set -n nested kernel=./vmlinux-new
container machine stop -n nested
container machine start -n nested

Installing from a Tarball

Download and extract a kernel before setting it as the system default:

curl -L https://example.com/custom-kernel.tar.zst -o kernel.tar.zst
tar --extract --file=kernel.tar.zst --strip-components=1 --wildcards '*/vmlinux'
container system kernel set \
    --binary ./vmlinux \
    --force

Summary

• Custom Linux kernels with container enable specialized features like nested virtualization that the default kata-containers kernel excludes.
• System-wide defaults are controlled via ContainerSystemConfig and set using container system kernel set --binary <path>.
• Per-machine overrides are stored in MachineConfig (parsed from TOML in Sources/ContainerPersistence/MachineConfig.swift) and specified via --kernel or kernel= in the CLI.
• Runtime loading is handled by RuntimeService in Sources/Services/RuntimeLinux/Server/RuntimeService.swift, which passes the binary path to the hypervisor.
• Clear machine-specific overrides by setting kernel= (empty value) to revert to system defaults.

Frequently Asked Questions

How do I verify that my custom kernel is actually running inside the container?

Run uname -r inside the container to check the version string matches your custom build. You can also check for specific features with dmesg | grep <feature> or examine /boot/config-$(uname -r) if your kernel build includes the config file.

What is the difference between the system kernel and machine kernel in Apple Container?

The system kernel (managed in ContainerSystemConfig.swift) serves as the default for all newly created machines, while the machine kernel (stored in MachineConfig.swift) overrides this on a per-machine basis. If a machine has no kernel specified, it falls back to the system default.

Can I use any Linux kernel binary, or does it need specific patches for Apple Container?

You can use any Linux kernel binary that matches the host architecture (arm64 or amd64). The kernel does not require Apple-specific patches, but it must include the features your workload requires (e.g., CONFIG_KVM for nested virtualization).

Does Apple Container support nested virtualization on all Macs?

Nested virtualization requires Apple Silicon M3 or later running macOS 15 or newer. Additionally, you must provide a custom kernel compiled with CONFIG_KVM=y, as the default kernel shipped with Apple Container disables this flag.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}