Debugging Container Startup Failures and Analyzing Boot Logs in apple/container

Use container logs --boot <name> to view the vminitd.log file inside the container's bundle, which captures kernel messages and init daemon output from the VM boot sequence to diagnose why a container failed to start.

When a container fails to start in the apple/container framework, the lightweight virtual machine (VM) often encounters issues during its initialization phase. The runtime captures all kernel and init-process output to a dedicated boot log, providing the primary diagnostic source for startup failures. Understanding how to access and interpret this log is essential for troubleshooting configuration errors, missing kernel features, or init script failures.

Understanding the Boot Log Architecture

The boot log system is integrated directly into the runtime service layer, capturing VM initialization output before the container's main processes begin.

Runtime Configuration in RuntimeService.swift

In Sources/Services/RuntimeLinux/Server/RuntimeService.swift, the runtime configures a BootLog instance that writes to the bundle's designated boot log path (lines 73-74). This establishes the logging pipeline that streams output from the vminitd init process to persistent storage.

Bundle Path Definition in MachineBundle.swift

The concrete file location is defined in Sources/Services/MachineAPIService/Client/MachineBundle.swift (lines 28-52), which specifies vminitd.log as the default boot log filename within the container's bundle directory. This file stores the complete boot sequence output, from kernel initialization through init daemon completion.

Viewing Boot Logs with the CLI

The container logs command provides direct access to boot-time output using the --boot flag.

To display the boot log for a specific container:

container logs --boot my-web-server

To follow the boot log in real-time during container startup:

container logs --boot -f my-web-server

To search for specific initialization errors within the boot log:

container logs --boot my-web-server | grep "failed to mount"

For standard container output after successful boot, omit the --boot flag:

container logs my-web-server

Troubleshooting Startup Failures

When a container enters a stopped state immediately after creation, follow this systematic approach to identify the root cause.

Check Container State

First, verify the container's current state using container ps or container list. A container that shows as stopped or fails to transition from created to running typically indicates a VM boot failure rather than an application-level error.

Inspect Boot Log Contents

Examine the vminitd.log file for specific error patterns:

• Kernel messages formatted as [ 0.xxx] ... indicating hardware detection or filesystem mounting issues
• Init daemon messages prefixed with info vminitd : ... confirming boot sequence progress
• Critical errors such as "failed to mount rootfs" or "cannot find /etc/hosts"

Validate Boot Configuration

Inspect the boot-config.json file located in the container's bundle directory. Access this configuration using:

container inspect my-web-server --format json | jq .bootConfig

Alternatively, locate the bundle directly:

cat $(container info --bundle-path my-web-server)/boot-config.json

Common configuration issues include mis-typed network settings, invalid kernel paths, or incorrect --init-image references.

Enable Debug Mode

For verbose XPC messaging and detailed runtime diagnostics, enable debug logging:

container logs --debug --boot my-web-server

Or set the environment variable when running the container:

CONTAINER_DEBUG=1 container run --name my-web-server registry.example.com/fido/web-test:latest

Common Failure Patterns

Kernel and Hardware Detection Issues

VM startup requires specific host capabilities. According to the implementation in Sources/Services/MachineAPIService/Server/MachinesService.swift, the host must be an Apple M3 or newer with virtualization-enabled kernels. Boot logs showing early kernel panics or hardware detection failures often indicate missing nested-virtualization support or incompatible kernel configurations.

Init Script Failures

Custom initialization scripts referenced in the boot configuration may fail before the container reaches a running state. Search the boot log for your custom init identifiers to locate script-specific errors.

Configuration Errors

Malformed boot-config.json entries—such as invalid network subnets, missing kernel images, or inaccessible init paths—surface as explicit errors in the boot log. Correlating the timestamp of the configuration error with the container creation time helps isolate the offending setting.

Summary

• The boot log (vminitd.log) captures all VM initialization output from the vminitd init process, stored within the container's bundle directory as defined in MachineBundle.swift
• Use container logs --boot <name> to access boot logs, and add -f to follow real-time output during startup
• Check RuntimeService.swift for the underlying BootLog configuration that wires the logging system
• Inspect boot-config.json via container inspect to identify configuration errors causing startup failures
• Ensure host hardware meets Apple M3+ requirements with virtualization support to prevent low-level boot failures

Frequently Asked Questions

Where is the boot log stored?

The boot log is stored as vminitd.log inside the container's bundle directory. The specific path is defined in Sources/Services/MachineAPIService/Client/MachineBundle.swift, where the bundle structure manages the concrete file location for each container's VM boot output.

Why does my container immediately stop after creation?

If container ps shows a stopped state immediately after container create or container run, the VM likely failed to boot. Examine the boot log using container logs --boot <name> to identify kernel panics, missing init processes, or configuration errors in the boot sequence.

Can I follow the boot log in real-time?

Yes. Append the -f or --follow flag to the boot log command: container logs --boot -f <name>. This streams the vminitd output as it is written, allowing you to monitor the initialization process during container startup.

How do I debug XPC communication errors during boot?

Enable verbose debugging by setting CONTAINER_DEBUG=1 or using the --debug flag with the container logs command. This exposes detailed XPC messaging between the host and the VM, as implemented in the runtime service layer, helping diagnose communication failures during the boot sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}