How KCloud-Platform-IoT Handles Object Storage with MinIO and Amazon S3

KCloud-Platform-IoT abstracts MinIO and Amazon S3 operations behind a generic AbstractStorage<T> template, using separate SDK implementations selected at runtime via the StoragePolicy enum.

KCloud-Platform-IoT provides a unified object storage layer that supports both MinIO and Amazon S3 through a clean strategy pattern implementation. The platform eliminates provider-specific boilerplate by defining common operations in an abstract template class while delegating SDK-specific logic to dedicated adapter classes.

The AbstractStorage Template

At the core of the KCloud-Platform-IoT storage system is the AbstractStorage<T> class located in the laokou-common-oss module. This generic template defines the lifecycle contract for any object storage provider, allowing the rest of the platform to interact with storage agnostically.

The abstract class declares four key extension points that concrete implementations must provide:

• protected abstract T getObj() – Constructs and returns the native client (MinioClient or S3Client).
• protected abstract void checkBucket(T client) – Verifies bucket existence using provider-specific APIs.
• protected abstract void upload(T client) – Handles the actual file upload operations.
• protected abstract String getUrl(T client) – Generates time-limited pre-signed download URLs.

All storage-specific logic remains encapsulated in concrete subclasses while the framework manages resource lifecycle and error handling uniformly.

MinIO Integration

MinIOStorage Implementation

The MinIO adapter resides in laokou-common/laokou-common-oss/src/main/java/org/laokou/common/oss/template/MinIOStorage.java. It extends AbstractStorage<MinioClient> and utilizes the MinIO Java SDK (io.minio.*) to communicate with S3-compatible storage clusters.

public final class MinIOStorage extends AbstractStorage<MinioClient> {
    private final MinIO minIO;

    public MinIOStorage(FileInfo fileInfo, BaseOss baseOss) {
        super(fileInfo);
        this.minIO = (MinIO) baseOss;
    }

    @Override
    protected MinioClient getObj() {
        return MinioClient.builder()
            .endpoint(minIO.getEndpoint())
            .credentials(minIO.getAccessKey(), minIO.getSecretKey())
            .region(minIO.getRegion())
            .build();
    }
}

The implementation constructs the client using configuration parameters stored in the MinIO model object (endpoint, access key, secret key, and region). Bucket validation leverages minioClient.bucketExists(), uploads utilize PutObjectArgs, and pre-signed URLs are generated via GetPresignedObjectUrlArgs with a default 5-day expiration.

Amazon S3 Integration

AmazonS3Storage Implementation

For AWS and S3-compatible services, the platform provides AmazonS3Storage in laokou-common/laokou-common-oss/src/main/java/org/laokou/common/oss/template/AmazonS3Storage.java. This adapter uses AWS SDK for Java v2 (software.amazon.awssdk.services.s3.*) and extends AbstractStorage<S3Client>.

public final class AmazonS3Storage extends AbstractStorage<S3Client> {
    private final AmazonS3 amazonS3;

    public AmazonS3Storage(FileInfo fileInfo, BaseOss baseOss) {
        super(fileInfo);
        this.amazonS3 = (AmazonS3) baseOss;
    }

    @Override
    protected S3Client getObj() {
        return S3Client.builder()
            .region(Region.of(amazonS3.getRegion()))
            .credentialsProvider(
                StaticCredentialsProvider.create(
                    AwsBasicCredentials.create(amazonS3.getAccessKey(),
                                              amazonS3.getSecretKey())))
            .endpointOverride(URI.create(amazonS3.getEndpoint()))
            .serviceConfiguration(
                S3Configuration.builder()
                    .pathStyleAccessEnabled(amazonS3.getPathStyleAccessEnabled() == 1)
                    .build())
            .build();
    }
}

Beyond standard credentials and endpoint configuration, this implementation supports path-style access—critical for on-premise S3-compatible deployments. The adapter verifies buckets using HeadBucketRequest, uploads via PutObjectRequest, and generates download URLs through S3Presigner with GetObjectPresignRequest, maintaining the same 5-day expiry policy as MinIO.

Runtime Provider Selection

The system determines which storage implementation to instantiate using the StoragePolicy enum defined in laokou-common/laokou-common-oss/src/main/java/org/laokou/common/oss/model/enums/StoragePolicy.java.

public enum StoragePolicy {
    MINIO("minio", "MinIO") {
        @Override
        public AbstractStorage<?> newInstance(FileInfo fileInfo, BaseOss baseOss) {
            return new MinIOStorage(fileInfo, baseOss);
        }
    },
    AMAZON_S3("amazon_s3", "亚马逊S3") {
        @Override
        public AbstractStorage<?> newInstance(FileInfo fileInfo, BaseOss baseOss) {
            return new AmazonS3Storage(fileInfo, baseOss);
        }
    };
}

The OssConvertor class handles deserialization of the JSON param field from the database into either MinIO or AmazonS3 configuration objects. When a storage operation is requested, the platform reads the type column from the sys_oss table, resolves the corresponding StoragePolicy enum value, and invokes newInstance() to create the appropriate adapter.

Administrative Configuration

Administrators configure storage providers through the React-based UI defined in ui/src/pages/Sys/Oss/config.tsx. The interface presents distinct configuration panels for each provider:

• MinIO: Requires endpoint, access key, secret key, region, and bucket name.
• Amazon S3: Requires endpoint, access key, secret key, region, bucket name, and path-style access toggle.

The selected provider type (minio or amazon_s3) persists in the database and drives the StoragePolicy lookup during runtime execution.

End-to-End File Upload Example

The following service-layer pattern demonstrates the complete flow for storing files using KCloud-Platform-IoT's storage abstraction:

public String storeFile(MultipartFile multipart, Long ossId) throws IOException {
    // Load OSS configuration from database
    SysOss oss = ossRepository.findById(ossId);
    BaseOss base = OssConvertor.toBaseOss(oss.getId(), oss.getName(), oss.getParam());

    // Prepare file metadata wrapper
    FileInfo fileInfo = new FileInfo();
    fileInfo.setName(UUID.randomUUID() + "_" + multipart.getOriginalFilename());
    fileInfo.setInputStream(multipart.getInputStream());
    fileInfo.setSize(multipart.getSize());
    fileInfo.setContentType(multipart.getContentType());

    // Resolve concrete implementation via policy enum
    StoragePolicy policy = StoragePolicy.of(oss.getType());
    AbstractStorage<?> storage = policy.newInstance(fileInfo, base);

    // Execute storage operations
    storage.createBucket();
    storage.upload(null);
    return storage.getUrl(null);
}

This approach ensures that switching from MinIO to Amazon S3 (or vice versa) requires zero code changes in the business logic—only a database configuration update.

Summary

• AbstractStorage provides a generic template defining the contract for all object storage providers in KCloud-Platform-IoT.
• MinIOStorage implements the contract using the MinIO Java SDK, supporting on-premise S3-compatible clusters.
• AmazonS3Storage implements the contract using AWS SDK v2, with support for path-style access and standard AWS endpoints.
• StoragePolicy enum acts as a factory, instantiating the correct adapter based on the type field stored in the sys_oss table.
• OssConvertor deserializes JSON configuration parameters into typed MinIO or AmazonS3 objects.
• The architecture enables provider-agnostic file operations while preserving access to provider-specific features like S3 path-style access.

Frequently Asked Questions

How does KCloud-Platform-IoT decide whether to use MinIO or Amazon S3?

The platform inspects the type column in the sys_oss database table, which contains the string identifier minio or amazon_s3. The StoragePolicy.of() method maps this string to the corresponding enum value, and the newInstance() method creates either a MinIOStorage or AmazonS3Storage object. This selection occurs at runtime for every storage operation.

Can KCloud-Platform-IoT support other S3-compatible storage providers beyond MinIO and AWS?

Yes. The AmazonS3Storage class supports any S3-compatible service through its configurable endpoint and pathStyleAccessEnabled parameters. For providers requiring significantly different SDKs, you would extend AbstractStorage<T> with a new implementation and add a corresponding entry to the StoragePolicy enum.

What is the default expiration time for pre-signed URLs in KCloud-Platform-IoT?

Both MinIOStorage and AmazonS3Storage generate pre-signed URLs with a 5-day expiration by default. This duration is hardcoded in the getUrl() implementations of each concrete class, utilizing GetPresignedObjectUrlArgs for MinIO and GetObjectPresignRequest for S3 with a 5-day duration configuration.

Where is the storage configuration stored in KCloud-Platform-IoT?

Storage configurations reside in the sys_oss table, where each row represents a distinct storage endpoint. The param column contains a JSON object with provider-specific settings (endpoint, credentials, region), which the OssConvertor class deserializes into MinIO or AmazonS3 model objects based on the type column value.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}