# How to Use Extra Kernel Cmdline Parameters for VM Customization in CubeSandbox

> Customize your VM with extra kernel cmdline parameters in CubeSandbox. Learn how to inject JSON arguments into the KVM micro-VM command line for advanced control.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: how-to-guide
- Published: 2026-07-15

---

**CubeSandbox supports custom kernel boot parameters through the `cube.vm.kernel.cmdline.append` annotation, which validates and injects JSON-formatted arguments into the KVM micro-VM's command line before guest memory initialization.**

CubeSandbox builds a KVM micro-VM for each sandbox runtime, assembling the guest kernel command line dynamically within the hypervisor (VMM). You can customize this boot configuration—enabling kernel flags like `net.ifnames=0` or debug consoles—without rebuilding the VM image by leveraging the annotation-based parameter injection system implemented in the CubeShim component.

## How Kernel Command-Line Customization Works

The injection follows a strict four-phase pipeline that ensures internal stability while allowing flexible customization.

### Base Command-Line Assembly

The foundation is built in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs) within the `generate_cmdline()` function (lines 38-51). This method creates a `linux_loader::cmdline::Cmdline` object and populates it with static payload settings, followed by **internal** parameters gathered from the device manager. These internal arguments include essential settings like `quiet`, `highres=off`, and `clocksource=kvm-clock` to ensure proper KVM guest operation.

### Parsing the Annotation

CubeShim defines the annotation key `cube.vm.kernel.cmdline.append` in [`CubeShim/shim/src/sandbox/config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/sandbox/config.rs) (lines 24-27). When `Config::new()` executes (lines 176-189), it parses this annotation using `Utils::anno_to_obj::<Vec<String>>`, expecting a JSON array of strings. Valid examples include `["net.ifnames=0","pci=noaer"]` or `["console=ttyS0,115200n8"]`. The parsed values are stored in `Config.extra_kernel_params`.

### Conflict Detection

Before VM launch, `CubeShim/shim/src/sandbox/sb.rs::build_vm_config()` (lines 51-58) validates user-defined parameters against the existing command line via `vc.check_cmdline_conflicts`. If any extra parameter conflicts with an internal requirement, sandbox creation fails immediately with a descriptive error, preventing boot-time kernel panics.

### Final Injection into Guest Memory

After passing validation, each custom parameter is appended via `vc.add_cmdline(param)` (lines 62-64 in [`sb.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/sb.rs)). The finalized command line is then written into the guest's boot memory by `Vm::load_kernel` in the VMM ([`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs), lines 1022-1023) through the `linux_loader::loader::load_cmdline` mechanism.

## Configuring Extra Kernel Parameters

You can supply custom boot arguments through YAML manifests or programmatically via the Go SDK.

### YAML Template Configuration

Add the annotation to your Sandbox metadata:

```yaml
apiVersion: cubesandbox.io/v1
kind: Sandbox
metadata:
  name: my-sandbox
  annotations:
    # Append kernel boot parameters

    cube.vm.kernel.cmdline.append: |
      ["net.ifnames=0", "pci=noaer", "console=ttyS0,115200n8"]
spec:
  # … other sandbox spec …

```

### Go SDK Implementation

The `Cubelet` SDK exports the annotation constant for programmatic access:

```go
import (
    cubesandbox "github.com/TencentCloud/CubeSandbox/Cubelet/pkg/client"
    "github.com/TencentCloud/CubeSandbox/Cubelet/pkg/constants"
)

sandbox := &cubesandbox.Sandbox{
    Metadata: &cubesandbox.Metadata{
        Name: "my-sandbox",
        Annotations: map[string]string{
            constants.AnnotationVMKernelCmdlineAppend: `["net.ifnames=0","pci=noaer"]`,
        },
    },
    // … spec …
}
client.CreateSandbox(context.Background(), sandbox)

```

## Verifying the Effective Command Line

After the sandbox starts, inspect the effective boot parameters through CubeShim's request logs:

```bash

# Find the sandbox ID first

SID=$(cubecli get sandbox my-sandbox -o jsonpath='{.metadata.uid}')

# Grep the kernel boot line from the shim request log

grep "\"InstanceId\":\"$SID\"" /data/log/CubeShim/cube-shim-req.log | \
  jq -r '.LogContent' | grep "console="

```

This confirms that your **extra kernel cmdline parameters** were successfully appended to the guest kernel.

## Key Source Files and Functions

Understanding these implementation details helps troubleshoot injection failures:

- **[`CubeShim/shim/src/sandbox/config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/sandbox/config.rs)** – Declares `ANNO_VM_KERNEL_CMDLINE_APPEND` and parses the annotation into `extra_kernel_params` using `Utils::anno_to_obj::<Vec<String>>`.
- **[`CubeShim/shim/src/sandbox/sb.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/sandbox/sb.rs)** – Contains `build_vm_config()` which validates conflicts via `vc.check_cmdline_conflicts` and injects parameters using `vc.add_cmdline()`.
- **[`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs)** – Houses `generate_cmdline()` for base assembly and `Vm::load_kernel` for final memory writing via `linux_loader`.
- **[`hypervisor/vmm/src/device_manager.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/device_manager.rs)** – Provides internal command-line fragments (console, clocksource) that are merged before custom parameters.
- **[`Cubelet/pkg/constants/const.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/pkg/constants/const.go)** – Exports `AnnotationVMKernelCmdlineAppend` for SDK consumers.

## Summary

- CubeSandbox uses the **`cube.vm.kernel.cmdline.append`** annotation to accept extra kernel cmdline parameters as a JSON string array.
- The **CubeShim** component validates parameters against internal requirements in `sb.rs::build_vm_config()` before injection.
- Valid parameters are appended after internal settings and loaded into guest memory via `linux_loader` in `vm.rs::load_kernel`.
- You can configure parameters via **YAML annotations** or the **Go SDK constant** `AnnotationVMKernelCmdlineAppend`.
- Use `cubecli` and CubeShim logs to verify the final effective command line inside the running micro-VM.

## Frequently Asked Questions

### What format should the `cube.vm.kernel.cmdline.append` value use?

The annotation requires a **JSON array of strings**, such as `["param1=value1","param2"]`. Invalid JSON or non-string array formats will fail parsing during `Config::new()` in [`CubeShim/shim/src/sandbox/config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/sandbox/config.rs), preventing sandbox creation.

### What happens if my custom parameter conflicts with internal ones?

CubeSandbox executes strict conflict detection in `sb.rs::build_vm_config()`. If your parameter duplicates or contradicts an internal setting (like `clocksource`), the sandbox creation fails immediately with a clear error message before the VM starts.

### Can I override existing kernel parameters or only append new ones?

The current implementation only **appends** parameters. If you need to override an internal setting, you must ensure the internal value is not set by the device manager, or rely on the kernel's "last value wins" behavior for specific boot flags. The conflict checker prevents dangerous overrides that could destabilize the KVM micro-VM.

### Where is the final command line physically stored in the VM?

The finalized command line is written into the guest's physical memory by `Vm::load_kernel` in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs) (lines 1022-1023) using `linux_loader::loader::load_cmdline`, making it available to the guest kernel at boot time.