Nydus Runtime Environments: Integrating with Containerd and Kata Containers

Nydus supports two distinct runtime integration modes: a global Containerd snapshotter for node-wide deployment and a runtime-level snapshotter specifically designed for Kata Containers VM-based workloads.

The dragonflyoss/nydus repository provides flexible deployment options that allow operators to use Nydus with standard OCI runtimes like runC or with VM-isolated runtimes like Kata Containers. The integration architecture differs significantly between these environments, requiring specific configuration in Containerd's plugin system or runtime handler definitions.

Containerd Global Snapshotter Integration

The simplest deployment method configures Nydus as a system-wide proxy plugin that intercepts all snapshot operations on the node. This mode is ideal when every workload on a Kubernetes node should use Nydus image acceleration.

Configuration Steps

1. Install the Nydus snapshotter binary (containerd-nydus-grpc) and the Nydus daemon (nydusd).
2. Add a proxy plugin entry to /etc/containerd/config.toml that points to the Nydus gRPC socket:

[proxy_plugins]
  [proxy_plugins.nydus]
    type = "snapshot"
    address = "/run/containerd-nydus/containerd-nydus-grpc.sock"

3. Configure Containerd to use Nydus as the default snapshotter for CRI workloads:

[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "nydus"
  disable_snapshot_annotations = false

After restarting Containerd, any pod that pulls a Nydus-formatted image will automatically use the nydusd daemon for filesystem operations. This configuration is documented in docs/containerd-env-setup.md under "Option 1: Configure as Containerd Global Snapshotter".

Containerd Runtime-Level Snapshotter for Kata Containers

For VM-based workloads using Kata Containers, Nydus provides a runtime-level integration that isolates the snapshotter to specific runtime handlers. This allows operators to run Nydus-accelerated workloads inside VMs while keeping standard runC containers on the default overlayfs snapshotter.

Configuration Requirements

This mode requires a patched Containerd that supports runtime-level snapshotter selection. The dragonflyoss/nydus documentation references a specific upstream patch:

https://github.com/nydusaccelerator/containerd/commit/0959cdb0b190e35c058a0e5bc2e256e59b95b584

Configure Containerd to use the default snapshotter for most workloads, but override it for a specific runtime handler:

[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "overlayfs"   # default for standard containers

  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc-nydus]
    snapshotter = "nydus"     # only for Kata/Nydus workloads

Sandbox Annotations

To trigger the Nydus snapshotter, the pod sandbox must include a specific runtime handler annotation:

metadata:
  name: nydus-sandbox
linux:
  security_context:
    namespace_options:
      network: 2
annotations:
  "io.containerd.cri.runtime-handler": "runc-nydus"

Only pods carrying this annotation will use the Nydus snapshotter; all others continue using overlayfs.

Nydus-Overlayfs Helper

Kata Containers passes additional mount options that must be stripped before mounting. The nydus-overlayfs helper handles this translation:

Located in contrib/nydus-overlayfs/cmd/main.go, the helper defines constants for Kata-specific options:

const (
    extraOptionKey      = "extraoption"
    kataVolumeOptionKey = "io.katacontainers.volume="
)

When invoked by the Kata shim with a mount line containing io.katacontainers.volume=..., the helper removes this option before calling syscall.Mount. This allows Nydus to function within the Kata VM boundary while maintaining compatibility with standard overlayfs expectations.

Key Components and Source Files

• docs/containerd-env-setup.md – Primary documentation for both global and runtime-level snapshotter configuration, including Option 1 (global) and Option 2 (Kata/runtime-level).
• contrib/nydus-overlayfs/cmd/main.go – Mount helper for Kata Containers that strips io.katacontainers.volume= options before mounting (lines 15-22 and 49-56).
• misc/performance/snapshotter_config.toml – Example configuration showing enable_kata_volume = false flag for Kata volume handling (line 79).
• containerd-nydus-grpc – The snapshotter binary that implements Containerd's snapshotter interface via gRPC.
• nydusd – The userspace daemon that serves Nydus (rafs) filesystems to container processes.

Summary

• Global snapshotter mode configures Nydus as a Containerd proxy plugin, making it the default for all node workloads via proxy_plugins.nydus configuration.
• Runtime-level snapshotter mode isolates Nydus to specific runtime handlers (like Kata Containers), allowing selective use through the io.containerd.cri.runtime-handler annotation while preserving overlayfs for standard containers.
• Kata integration requires the nydus-overlayfs helper to handle VM-specific mount options by stripping io.katacontainers.volume= before mounting.
• Both modes rely on the containerd-nydus-grpc snapshotter and nydusd daemon, but differ in Containerd configuration scope and runtime handler selection.

Frequently Asked Questions

What is the difference between the global and runtime-level Nydus snapshotter?

The global snapshotter configures Nydus as a system-wide proxy plugin in Containerd, affecting all containers on the node. The runtime-level snapshotter restricts Nydus to specific runtime handlers defined in Containerd's configuration, allowing only containers with matching io.containerd.cri.runtime-handler annotations to use Nydus while others remain on the default overlayfs snapshotter.

How do I configure Nydus for Kata Containers specifically?

To use Nydus with Kata Containers, you must enable the runtime-level snapshotter mode. First, apply the required Containerd patch that supports runtime-level snapshotter selection. Then configure a specific runtime handler in /etc/containerd/config.toml with snapshotter = "nydus", and ensure your pod sandbox spec includes the annotation "io.containerd.cri.runtime-handler": "runc-nydus" (or your handler name).

What is the purpose of the nydus-overlayfs helper?

The nydus-overlayfs helper is a mount utility located at contrib/nydus-overlayfs/cmd/main.go that processes mount options passed by Kata Containers. It strips the Kata-specific io.katacontainers.volume= option (defined as kataVolumeOptionKey in the source) before invoking the standard Linux mount syscall, allowing Nydus to function correctly within Kata's VM-based container boundary.

Can I use Nydus with runC and Kata on the same node?

Yes. By using the runtime-level snapshotter configuration, you can designate Nydus for specific runtime handlers (such as Kata) while keeping runC containers on the default overlayfs snapshotter. Configure the global default snapshotter as overlayfs in Containerd, then override it with snapshotter = "nydus" only for the specific runtime handlers that require Nydus acceleration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}