Nydus Storage Backends: Complete Guide to Registry, OSS, and S3 Support

Nydus supports five primary storage backends—Registry, OSS, S3, local filesystem, and HTTP proxy—abstracted through the BlobFactory interface in storage/src/factory.rs.

The dragonflyoss/nydus repository implements a content-addressable storage layer that decouples image artifact location from retrieval logic. Understanding the available Nydus storage backends ensures you can configure optimal blob fetching for cloud-native deployments, whether pulling from public container registries or private object stores.

Core Cloud Storage Backends

Nydus provides first-class implementations for the three major remote storage protocols. Each backend resides in storage/src/backend/ and exposes a consistent interface through the factory pattern.

Alibaba Cloud OSS

The OSS backend connects to Alibaba Cloud Object Storage Service. Implementation lives in storage/src/backend/oss.rs, with configuration defined by the OssConfig struct in api/src/config.rs (lines 484–507).

Key configuration parameters include:

• endpoint – Regional endpoint (e.g., oss-cn-hangzhou.aliyuncs.com)
• bucket_name – Target storage bucket
• access_key_id and access_key_secret – Authentication credentials

Amazon S3 and Compatible Stores

The S3 backend supports AWS S3 and API-compatible alternatives (MinIO, Ceph, etc.). Source code resides in storage/src/backend/s3.rs, configured via S3Config in api/src/config.rs (lines 521–545).

Required fields mirror standard S3 SDK configuration:

• region – AWS region identifier
• endpoint – Custom endpoint for non-AWS deployments
• bucket_name – Storage container
• access_key_id and access_key_secret – IAM credentials

OCI-Compatible Container Registries

The Registry backend enables direct blob fetching from any OCI-compliant container registry (Docker Hub, Harbor, GHCR, etc.). Implementation is in storage/src/backend/registry.rs, using RegistryConfig from api/src/config.rs (lines 888–910).

Configuration supports:

• host – Registry domain (e.g., registry-1.docker.io)
• repo – Repository path (e.g., library/ubuntu)
• auth – Base64-encoded user:pass string for authentication

Optional and Local Storage Backends

Beyond cloud providers, Nydus includes additional backends gated by Cargo features for specialized deployment scenarios.

Local Filesystem (localfs)

Enabled via the backend-localfs feature. Implementation in storage/src/backend/localfs.rs allows mounting blobs from local directories. Useful for air-gapped environments or pre-seeded cache scenarios.

Local Disk (localdisk)

Enabled via the backend-localdisk feature. Found in storage/src/backend/localdisk.rs. Provides raw block device access for high-performance local storage without filesystem overhead.

HTTP Proxy (http-proxy)

Enabled via the backend-http-proxy feature. Implementation in storage/src/backend/http_proxy.rs. Routes blob requests through HTTP/HTTPS proxies for restricted network environments.

Configuring Nydus Storage Backends

Nydus selects backends at runtime through the BlobFactory interface defined in storage/src/factory.rs. The factory's supported_backends method (lines 16–31) enumerates available types, while new_backend (lines 34–71) instantiates the concrete implementation based on the backend_type field in BackendConfigV2.

JSON Configuration Example

Configure your backend via the backend section of a Nydus configuration file:

{
  "id": "production-blob",
  "backend": {
    "backend_type": "s3",
    "s3": {
      "scheme": "https",
      "endpoint": "s3.amazonaws.com",
      "region": "us-east-1",
      "bucket_name": "nydus-image-blobs",
      "access_key_id": "AKIAIOSFODNN7EXAMPLE",
      "access_key_secret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
    }
  },
  "cache": {
    "type": "blobcache"
  }
}

Change backend_type to "oss" or "registry" and populate the corresponding configuration block. Only the section matching the selected type is processed by the factory.

Programmatic Configuration (Rust)

For dynamic configuration, use the BlobFactory directly:

use nydus_storage::factory::BlobFactory;
use nydus_api::{BackendConfigV2, S3Config};

// Construct S3 configuration
let s3_config = S3Config {
    scheme: "https".into(),
    endpoint: "s3.amazonaws.com".into(),
    region: "us-west-2".into(),
    bucket_name: "my-nydus-bucket".into(),
    object_prefix: "images/".into(),
    access_key_id: std::env::var("AWS_ACCESS_KEY_ID").unwrap().into(),
    access_key_secret: std::env::var("AWS_SECRET_ACCESS_KEY").unwrap().into(),
    ..Default::default()
};

// Wrap in BackendConfigV2
let backend_cfg = BackendConfigV2 {
    backend_type: "s3".to_string(),
    s3: Some(s3_config),
    ..Default::default()
};

// Instantiate backend via factory
let backend = BlobFactory::new_backend(&backend_cfg, "blob-001")
    .expect("Failed to create S3 backend");

Replace S3Config with OssConfig or RegistryConfig and adjust backend_type accordingly to target different storage systems.

Summary

• Nydus storage backends abstract blob retrieval through the BlobFactory pattern in storage/src/factory.rs.
• Core remote backends include OSS (storage/src/backend/oss.rs), S3 (storage/src/backend/s3.rs), and Registry (storage/src/backend/registry.rs).
• Optional local backends (localfs, localdisk, http-proxy) require specific Cargo features and suit air-gapped or proxy-restricted environments.
• Runtime selection occurs via the backend_type field in BackendConfigV2, with configuration structs defined in api/src/config.rs.

Frequently Asked Questions

What storage backends does Nydus support for container images?

Nydus supports Registry (OCI-compliant container registries), OSS (Alibaba Cloud Object Storage), and S3 (Amazon S3 and compatible stores) as its primary remote backends. Additionally, it supports localfs, localdisk, and http-proxy backends for specialized local or proxied deployments.

How do I configure Nydus to use Amazon S3 instead of a container registry?

Set the backend_type field to "s3" in your BackendConfigV2 configuration and provide the S3Config struct with your bucket name, region, endpoint, and credentials. The factory method BlobFactory::new_backend in storage/src/factory.rs will instantiate the S3 backend from storage/src/backend/s3.rs accordingly.

Can Nydus run entirely without cloud storage using only local disks?

Yes. Enable the backend-localfs or backend-localdisk Cargo features to compile the local storage backends. Configure backend_type as "localfs" or "localdisk" and point to your local directory or block device path. This configuration is ideal for air-gapped environments or high-performance local caching scenarios.

What is the difference between the Registry backend and S3/OSS backends in Nydus?

The Registry backend (storage/src/backend/registry.rs) speaks the OCI Distribution protocol to fetch layers from container registries like Docker Hub or Harbor. The S3 and OSS backends (storage/src/backend/s3.rs and storage/src/backend/oss.rs) use native object storage APIs with authentication schemes specific to those cloud providers. While all three store blob data, the Registry backend handles manifest parsing and authentication flows required by the OCI specification, whereas S3/OSS treat blobs as raw objects in buckets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}