Container Kernel Management and Custom Kernel Installation: A Complete Guide

The Apple Container runtime automatically downloads a default Linux kernel on first use, stores configuration in config.toml via the KernelConfig struct, and enables global or per-machine kernel overrides through the container system kernel set and container machine create --kernel commands.

The apple/container repository provides a lightweight container runtime for macOS that leverages Linux virtual machines for workload isolation. Container kernel management is handled through a declarative configuration system defined in Sources/ContainerPersistence/ContainerSystemConfig.swift, enabling automatic downloads of pre-built kernels alongside support for custom kernel binaries.

Understanding the Kernel Configuration Schema

The kernel configuration is centralized in the [kernel] section of config.toml and exposed through the ContainerSystemConfig persistence layer.

The config.toml Structure

The top-level [kernel] section defines the binary location and remote source archive. As documented in docs/container-system-config.md (lines 53-60), this TOML section controls which kernel the runtime downloads and where it extracts the binary within the archive.

The KernelConfig Implementation

In Sources/ContainerPersistence/ContainerSystemConfig.swift (lines 67-84), the KernelConfig struct maps the TOML configuration to Swift types with two key properties:

• binaryPath: The path inside the downloaded archive pointing to the kernel binary (default: opt/kata/share/kata-containers/vmlinux-6.18.15-186)
• url: The remote archive URL containing the kernel (default: https://github.com/kata-containers/kata-containers/releases/download/3.28.0/kata-static-3.28.0-arm64.tar.zst)

Installing and Managing the Default Kernel

The runtime supports both automatic lazy loading and explicit manual installation workflows.

Automatic Download on First Start

When executing container system start, the runtime checks the host's container data directory for the default kernel matching your architecture. If the binary is absent, the startup routine in Sources/ContainerCommands/System/SystemStart.swift (lines 180-185) emits a diagnostic hint directing you to run container system kernel set --recommended. You can also pass --enable-kernel-install to trigger the download prompt immediately.

Manual Installation with container system kernel set

To explicitly set or replace the global default kernel, use the container system kernel set command. This workflow extracts the kernel from either the configured remote URL or a local tarball specified by the user.

Install the recommended default kernel:

container system kernel set --recommended

Force installation from a custom tarball with a specific binary path:

container system kernel set \
    --force \
    --tar /tmp/kata-static-custom.tar.zst \
    --binary vmlinux.custom

The implementation validates successful installation by verifying the guest kernel version matches expectations, as exercised in Tests/IntegrationTests/System/TestCLIKernelSetSerial.swift.

Running Containers with Custom Kernels

For development or specialized workloads, you can override the global kernel default for individual container machines without affecting system-wide settings.

The --kernel Flag

As documented in docs/command-reference.md (lines 31-32), the container machine create command accepts a -k, --kernel <path> argument. This flag instructs the runtime to launch the VM using the specified kernel binary rather than the global default.

Create a machine with a custom kernel:

container machine create \
    --virtualization \
    --kernel /path/to/vmlinux-kvm \
    --name kvm-dev \
    alpine:latest

Nested Virtualization Requirements

Using custom kernels with container machines requires nested virtualization support, available exclusively on Apple Silicon M3 or later devices running macOS 15 or newer. The docs/container-machine.md (lines 75-82) details these prerequisites and demonstrates the complete workflow for supplying custom kernels in development environments.

Verification and Testing

After installing or switching kernels, verify the guest environment reports the correct kernel version:

# Check the running kernel version inside the VM

container machine run -n kvm-dev -- uname -r

The integration test suite in Tests/IntegrationTests/System/TestCLIKernelSetSerial.swift validates this behavior by asserting that the guest uname -r output matches the expected kernel version after swapping.

Error Handling and Troubleshooting

If the default kernel for your host architecture is missing at startup, Sources/ContainerCommands/System/SystemStart.swift (lines 180-185) detects the condition and prints actionable guidance to run container system kernel set --recommended. The kernel binary persists in the host's container data directory and is referenced by the runtime whenever launching a VM.

Summary

• Configuration-driven architecture: Kernel settings reside in the [kernel] TOML section and map to the KernelConfig struct in Sources/ContainerPersistence/ContainerSystemConfig.swift (lines 67-84).
• Automatic provisioning: The runtime downloads the default Kata Containers kernel (v6.18.15-186) from the configured URL on first use or on demand.
• Global kernel replacement: Use container system kernel set with --force, --tar, and --binary flags to install a custom kernel as the system-wide default.
• Per-machine overrides: Pass --kernel <path> to container machine create to use a specific kernel for a single container machine without modifying global configuration.
• Hardware constraints: Custom kernels for container machines require nested virtualization (Apple Silicon M3+ and macOS 15+).

Frequently Asked Questions

Where is the kernel configuration stored in the apple/container repository?

The kernel configuration is stored in the [kernel] section of config.toml and mapped to the KernelConfig struct in Sources/ContainerPersistence/ContainerSystemConfig.swift (lines 67-84). This structure defines the binaryPath within the archive and the remote url for downloading the kernel binary.

How do I install a custom kernel globally for all container machines?

Run container system kernel set --force --tar /path/to/archive.tar.zst --binary vmlinux-name to extract and install a custom kernel from a local tarball. This updates the global default stored in the container data directory, affecting all subsequent container machine creations unless overridden with the --kernel flag.

Can I use different kernels for different container machines?

Yes. Pass the --kernel /path/to/vmlinux flag to container machine create to specify a custom kernel for that specific machine only. This requires nested virtualization support (Apple Silicon M3+ and macOS 15+) and overrides the global default without modifying it.

What happens if the kernel is missing when I start the container system?

If the default kernel is missing, Sources/ContainerCommands/System/SystemStart.swift (lines 180-185) detects the absence and prints a hint instructing you to run container system kernel set --recommended to download and install the appropriate kernel for your architecture.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}