How Nydus Integrates with Kubernetes: CRI Configuration and Containerd Setup Guide

Nydus integrates with Kubernetes through the CRI (Container Runtime Interface) by registering the nydus-snapshotter as a proxy plugin in containerd, enabling on-demand image loading via the nydusd daemon.

The dragonflyoss/nydus repository provides a container image acceleration solution that integrates with Kubernetes clusters without requiring kubelet modifications. By leveraging the standard CRI implementation in containerd, Nydus enables lazy loading of container images through a specialized snapshotter architecture that works transparently with existing Kubernetes workloads.

Understanding the Nydus Kubernetes Architecture

Nydus integrates with Kubernetes through a three-layer architecture that bridges the gap between the Kubernetes control plane and the Nydus image format. This design allows pods to consume Nydus-converted images while maintaining full compatibility with standard Kubernetes operations.

The CRI and Containerd Layer

The containerd runtime acts as the CRI server that kubelet communicates with. According to the source code in docs/containerd-env-setup.md, containerd registers the nydus-snapshotter as a proxy plugin through the proxy_plugins.nydus configuration. This registration points to a Unix domain socket at /run/containerd-nydus/containerd-nydus-grpc.sock, allowing containerd to delegate snapshot operations to the Nydus implementation.

The Nydus-Snapshotter Layer

The nydus-snapshotter implements the io.containerd.snapshot.v1 API. When Kubernetes creates a pod configured to use the nydus snapshotter, the snapshotter launches a nydusd daemon process. As implemented in the dragonflyoss/nydus source code, this daemon serves Nydus image layers on-demand, lazily pulling chunks from remote registries or Dragonfly P2P networks and presenting a virtual filesystem to the container.

The snapshotter configuration in misc/performance/snapshotter_config.toml includes the enable_kubeconfig_keychain flag, which enables synchronization of image pull secrets from the Kubernetes API, ensuring authenticated registry access works seamlessly with Kubernetes service accounts.

The Kubernetes Pod Specification Layer

At the Kubernetes level, integration requires specific annotations in the pod sandbox configuration. The pod must include the annotation io.containerd.cri.runtime-handler: runc-nydus to signal containerd to use the runtime-level snapshotter configuration that points to Nydus.

Configuring Containerd for Nydus Integration

To enable Nydus in a Kubernetes cluster, you must modify the containerd configuration file at /etc/containerd/config.toml to register the proxy plugin and set the default snapshotter.

First, add the proxy plugin definition to register the nydus-snapshotter:

[proxy_plugins]
  [proxy_plugins.nydus]
    type = "snapshot"
    address = "/run/containerd-nydus/containerd-nydus-grpc.sock"

Next, configure the CRI plugin to use the nydus snapshotter for all container operations:

[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "nydus"
  disable_snapshot_annotations = false
  discard_unpacked_layers = false

After updating the configuration, restart containerd to apply the changes. The nydus-snapshotter binary (containerd-nydus-grpc) must be running and listening on the specified socket path before containerd attempts to use the snapshotter.

Deploying Pods with Nydus Images

Once containerd is configured, you can deploy Kubernetes pods that use Nydus-accelerated images. The deployment requires a sandbox configuration with specific annotations and a container spec referencing a Nydus-converted image.

Create the sandbox configuration file nydus-sandbox.yaml:

metadata:
  name: nydus-sandbox
  namespace: default
linux:
  security_context:
    namespace_options:
      network: 2
annotations:
  "io.containerd.cri.runtime-handler": "runc-nydus"

Create the container specification nydus-container.yaml:

metadata:
  name: nydus-container
image:
  image: localhost:5000/ubuntu-nydus:latest
command:
  - /bin/sleep
args:
  - "600"

Deploy the pod using crictl or Kubernetes APIs:

# Pull the Nydus image

sudo crictl pull localhost:5000/ubuntu-nydus:latest

# Create and start the pod

POD=$(sudo crictl runp nydus-sandbox.yaml)
CONTAINER=$(sudo crictl create $POD nydus-container.yaml nydus-sandbox.yaml)
sudo crictl start $CONTAINER

# Verify the running container

sudo crictl ps

Key Configuration Files and Source Paths

The dragonflyoss/nydus repository contains several critical files that define the Kubernetes integration:

• docs/containerd-env-setup.md – Documents the complete containerd configuration, proxy plugin setup, and CRI annotation requirements for Kubernetes deployment.
• misc/performance/snapshotter_config.toml – Contains the enable_kubeconfig_keychain configuration that enables Kubernetes secret synchronization for authenticated registry access.
• contrib/nydusify/pkg/snapshotter/external/backend/walker.go – Implements the backend walker for fetching layers from remote registries, used by nydusd in Kubernetes nodes.
• smoke/tests/tool/snapshotter.go – Test harness that exercises the snapshotter via CRI sockets, demonstrating the communication flow between containerd and the nydus-snapshotter.

Summary

Nydus integrates with Kubernetes through a CRI-compliant architecture that requires minimal cluster configuration:

• Containerd proxy plugin – Registers the nydus-snapshotter at /run/containerd-nydus/containerd-nydus-grpc.sock to handle snapshot operations.
• CRI configuration – Sets snapshotter = "nydus" in the containerd CRI plugin configuration to enable Nydus for all workloads.
• Pod annotations – Uses io.containerd.cri.runtime-handler: runc-nydus in the sandbox configuration to activate the Nydus snapshotter for specific pods.
• Lazy loading – The nydusd daemon serves image layers on-demand, reducing startup time and network bandwidth for Kubernetes containers.

Frequently Asked Questions

How does Nydus integrate with Kubernetes without modifying the kubelet?

Nydus integrates with Kubernetes through the standard CRI (Container Runtime Interface) implementation in containerd. The kubelet communicates with containerd using the standard CRI API, and containerd routes snapshot operations to the nydus-snapshotter through a proxy plugin configuration. This architecture requires no changes to the kubelet code or Kubernetes control plane components, as the integration happens entirely within the container runtime layer.

What is the role of the nydus-snapshotter in Kubernetes clusters?

The nydus-snapshotter implements the io.containerd.snapshot.v1 API and acts as the bridge between containerd and the Nydus image format. When Kubernetes creates a pod using the Nydus snapshotter, the snapshotter launches a nydusd daemon process that serves the container image layers. This daemon performs lazy loading by fetching only the required data chunks from remote registries or Dragonfly P2P networks, presenting a virtual filesystem to the container while dramatically reducing startup latency.

How do I configure containerd to use Nydus for specific pods only?

To use Nydus for specific pods rather than cluster-wide, omit the global snapshotter = "nydus" setting in the containerd CRI configuration. Instead, configure only the proxy plugin registration in /etc/containerd/config.toml to make the nydus-snapshotter available. Then, annotate specific pod sandboxes with io.containerd.cri.runtime-handler: runc-nydus to activate Nydus for those workloads. Pods without this annotation will use the default overlayfs snapshotter, allowing mixed-runtime clusters.

What annotation is required to enable Nydus in a Kubernetes pod?

To enable Nydus for a Kubernetes pod, you must add the annotation io.containerd.cri.runtime-handler: runc-nydus to the pod sandbox configuration. This annotation signals containerd to use the runtime handler that points to the nydus-snapshotter. Additionally, ensure the containerd CRI configuration has disable_snapshot_annotations = false to allow snapshotter selection through annotations. Without this annotation, containerd will use the default snapshotter instead of Nydus.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}