How to Configure Object Storage for Asynchronous Image Tasks in Sub2API

Sub2API supports asynchronous image generation by offloading large image results to an S3-compatible object storage service when a complete configuration is provided, keeping Redis memory usage low.

Sub2API is an open-source API gateway that handles resource-intensive image generation tasks asynchronously to prevent Redis memory exhaustion. When you configure object storage for asynchronous image tasks in Sub2API, the system uploads finished images to an S3-compatible bucket rather than storing base64 data in memory. This guide explains the configuration options, runtime behavior, and troubleshooting steps based on the actual source code implementation.

Enabling the Feature

The asynchronous image storage feature is disabled by default and activates only when a complete S3-compatible configuration is supplied. You can enable it through the Admin UI or configuration files, with the UI taking precedence over file-based settings.

Admin UI Configuration

The recommended method is navigating to Admin → Backup → Async image object storage and toggling Enabled. The interface reuses your existing backup S3 configuration (endpoint, region, credentials) while allowing you to specify a dedicated bucket and prefix for images. Saving the form instantly rebuilds the storage client without requiring a container restart, as handled in backend/internal/server/routes/admin.go lines 624-626.

Configuration File and Environment Variables

When no settings exist in the UI database, Sub2API falls back to the image_storage block in config.yaml. This block also respects environment variable overrides using the IMAGE_STORAGE_* prefix. However, versions prior to v0.1.161 ignored environment variables for storage credentials; if running older releases, place the image_storage block directly in /app/data/config.yaml (copy from deploy/config.example.yaml).

Critical requirement: If image_storage.enabled is true but any required credentials (bucket, access key, secret key) are missing, Sub2API logs the warning "image_storage.enabled is true but object storage is not fully configured; async image tasks are disabled" and the async endpoints return 404 to prevent Redis memory exhaustion.

Required Configuration Fields

The ImageStorageSettings struct in backend/internal/service/image_storage_settings.go (lines 30-45) defines the exact fields required for the async pipeline:

• Enabled – Turns the async image feature on or off.
• ReuseBackupS3 – When true, reuses backup S3 credentials already configured for database backups; only Bucket and Prefix are required.
• Bucket – Target bucket for image objects (optional if reusing backup).
• Prefix – Object key prefix (e.g., images/).
• PublicBaseURL – Base URL for public links; if empty, Sub2API generates a presigned URL.
• PresignExpiry – TTL in hours for presigned URLs.
• MaxDownloadBytes – Upper limit for downloading external images (e.g., re-hosted URLs).
• Endpoint, Region, AccessKeyID, SecretAccessKey, ForcePathStyle – Required only when not reusing backup credentials.

YAML Configuration Example

Use the following structure in your config.yaml to configure a Cloudflare R2 or MinIO endpoint:

# config.yaml – image storage block

image_storage:
  enabled: true
  endpoint: "https://<account_id>.r2.cloudflarestorage.com"
  region: "auto"
  bucket: "my-images"
  access_key_id: "YOUR_ACCESS_KEY"
  secret_access_key: "YOUR_SECRET_KEY"
  prefix: "images/"
  force_path_style: false          # Set true for MinIO or path-style buckets

  public_base_url: ""              # Empty → presigned URL

  presign_expiry_hours: 24
  max_download_bytes: 33554432     # 32 MiB limit for upstream image URLs

Environment variables follow the pattern IMAGE_STORAGE_ENDPOINT, IMAGE_STORAGE_BUCKET, etc., and merge with UI-saved values at runtime.

Runtime Flow

The async image pipeline follows this execution path based on the implementation in backend/internal/service/wire.go and related files:

1. Resolver creation – ProvideImageStorageSettingService (lines 638-645 in wire.go) builds an ImageStorageSettingService that resolves the current uploader based on stored settings.
2. Resolution – On the first request, the resolver checks cfg.Enabled and cfg.IsConfigured(). If both are true, it constructs an ImageResultUploader via the factory and marks the feature as enabled (lines 9-18 in image_storage_settings.go).
3. Task submission – A client calls POST /v1/images/generations/async or /edits/async. Sub2API creates a task record in Redis and immediately returns 202 Accepted with a polling URL.
4. Task processing – When upstream generation finishes, each image uploads to the configured bucket using the ImageStorage implementation. The Redis result replaces the base64 payload with a compact reference containing the URL.
5. Polling – Clients poll GET /v1/images/tasks/{task_id}. Success returns the public URL of the uploaded image; failure marks the task as failed.

If the uploader cannot store the image (e.g., bucket unreachable), the task fails and surfaces the error to the client.

Troubleshooting Common Issues

• 404 "async image tasks are not enabled" – Indicates incomplete storage configuration. Check startup logs for specific missing keys in backend/internal/service/image_storage_settings.go (lines 12-15).
• Missing environment variables – Sub2API versions before v0.1.161 do not read IMAGE_STORAGE_* variables. Upgrade or use config.yaml exclusively.
• Authentication failures – Verify ForcePathStyle is set to true for MinIO deployments and false for virtual-hosted style buckets like AWS S3 or Cloudflare R2.

Summary

• Sub2API offloads large image results to S3-compatible object storage only when fully configured to prevent Redis memory exhaustion.
• Enable the feature via Admin → Backup → Async image object storage or the image_storage block in config.yaml with complete credentials.
• The system validates configuration at startup; incomplete settings trigger a warning in backend/internal/service/image_storage_settings.go and disable async endpoints with 404 responses.
• Supported configuration fields include ReuseBackupS3, Bucket, Prefix, and presigned URL options defined in the ImageStorageSettings struct.
• Async tasks return 202 Accepted immediately, process images in the background, and store results in the configured bucket while returning compact URLs to clients.

Frequently Asked Questions

What happens if object storage is not fully configured?

Sub2API logs a warning stating "image_storage.enabled is true but object storage is not fully configured; async image tasks are disabled" and returns 404 Not Found on async endpoints. This safety mechanism prevents base64 image data from exhausting Redis memory.

Can I use the same S3 credentials for backups and async images?

Yes. Set ReuseBackupS3 to true in your configuration. When enabled, Sub2API reuses the existing backup S3 credentials (endpoint, region, access keys) and requires only the Bucket and Prefix fields for the image storage settings.

Why do async image endpoints return 404 after enabling the feature?

The 404 response indicates that image_storage.enabled is true but required fields (bucket, access key, secret key) are missing or invalid. Check the startup logs for the specific missing configuration keys and verify your config.yaml or Admin UI settings in backend/internal/service/image_storage_settings.go.

Which Sub2API version supports environment variables for image storage?

Environment variable overrides (IMAGE_STORAGE_*) work correctly in Sub2API version v0.1.161 and later. Earlier versions ignore these variables; upgrade your deployment or define settings exclusively in /app/data/config.yaml to ensure the configuration loads properly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}