NekoImageGallery Storage Backends: How to Configure S3-Compatible Storage

NekoImageGallery supports three storage backends—local filesystem, S3-compatible object storage, and disabled mode—selected via the StorageMode enum and configured through environment variables prefixed with APP_STORAGE__.

NekoImageGallery abstracts all file operations behind a unified storage service, allowing seamless switching between local development and production cloud deployments. The repository hv0905/nekoimagegallery implements this abstraction in app/Services/storage/base.py, with concrete implementations for filesystem and S3-compatible storage. Understanding these storage backends enables you to scale from single-node deployments to distributed cloud-native architectures without modifying application code.

Supported Storage Backends

The storage layer is initialized in app/Services/storage/__init__.py based on the StorageMode defined in app/config.py. All implementations inherit from BaseStorage, ensuring a consistent async API across every backend.

Local Filesystem Storage

StorageMode.LOCAL instantiates the LocalStorage class defined in app/Services/storage/local_storage.py. This backend stores files on the host filesystem, defaulting to the ./static directory. It is ideal for development environments or small-scale deployments where persistent volumes are mounted directly to the container.

S3-Compatible Object Storage

StorageMode.S3 activates the S3Storage class in app/Services/storage/s3_compatible_storage.py. This backend uses the opendal library to communicate with any S3-compatible service including AWS S3, MinIO, Ceph, and DigitalOcean Spaces. It provides async operations for upload, download, presigned URL generation, and batch file listing, making it suitable for production and cloud-native setups.

Disabled Storage Mode

StorageMode.DISABLED creates a DisabledStorage instance that raises NotImplementedError for all storage operations. Use this mode when running the API without local persistence, such as when all images are referenced by external URLs only.

Configuring S3-Compatible Storage

Switching to an S3-compatible backend requires setting the storage method and providing credential and endpoint details through the S3StorageSettings model.

Configuration Model and Settings

The S3StorageSettings class in app/config.py defines every configurable parameter for S3 connections:

class S3StorageSettings(BaseModel):
    path: str = "./static"
    bucket: str | None = None
    region: str | None = None
    endpoint_url: str | None = None
    access_key_id: str | None = None
    secret_access_key: str | None = None
    session_token: str | None = None
    user_endpoint_url: str | None = None

Environment variables use the prefix APP_STORAGE__S3__ to populate these fields. The default example in config/default.env demonstrates the expected format.

Required Environment Variables

Set the following variables to activate and configure the S3 backend:

• APP_STORAGE__METHOD — Must be set to s3 to select the S3 backend.
• APP_STORAGE__S3__BUCKET — Name of the target bucket.
• APP_STORAGE__S3__PATH — Prefix inside the bucket (default: ./static).
• APP_STORAGE__S3__REGION — Region identifier required by some providers.
• APP_STORAGE__S3__ENDPOINT_URL — URL of the S3-compatible service (e.g., https://s3.amazonaws.com or http://minio:9000).
• APP_STORAGE__S3__ACCESS_KEY_ID — Authentication access key.
• APP_STORAGE__S3__SECRET_ACCESS_KEY — Authentication secret key.
• APP_STORAGE__S3__SESSION_TOKEN — (Optional) Temporary session token.
• APP_STORAGE__S3__USER_ENDPOINT_URL — (Optional) Public-facing URL for presigned links (e.g., CDN domain).

Example .env configuration:

APP_STORAGE__METHOD=s3
APP_STORAGE__S3__BUCKET=my-gallery-bucket
APP_STORAGE__S3__PATH=static
APP_STORAGE__S3__REGION=us-east-1
APP_STORAGE__S3__ENDPOINT_URL=https://s3.amazonaws.com
APP_STORAGE__S3__ACCESS_KEY_ID=AKIA...
APP_STORAGE__S3__SECRET_ACCESS_KEY=xxxxxxxx

# optional

APP_STORAGE__S3__USER_ENDPOINT_URL=https://cdn.my-gallery.com/static

Implementation Details

When the application starts, StorageService reads the global configuration and constructs an S3Storage instance. The constructor builds an opendal.AsyncOperator with the provided credentials, as implemented in app/Services/storage/s3_compatible_storage.py:

self.op = (AsyncOperator("s3",
                         root=str(self.static_dir),
                         bucket=self.bucket,
                         region=self.region,
                         endpoint=self.endpoint,
                         access_key_id=config.storage.s3.access_key_id,
                         secret_access_key=config.storage.s3.secret_access_key)
          .layer(RetryLayer(...))
          .layer(MimeGuessLayer()))

The root parameter maps to the configured path, while bucket, region, endpoint, and credentials are forwarded directly to the operator. All subsequent async calls (upload, fetch, presign_url) delegate to this operator, ensuring robust S3-compatible behavior with automatic retries and MIME type detection.

Presigned URLs and Custom Endpoints

The presign_url method generates temporary signed URLs via opendal.presign_read. If user_endpoint_url is configured, the rewrite_s3_presign_url function replaces the internal S3 endpoint with your custom domain (e.g., a CDN), ensuring clients receive optimized public URLs while the application communicates with the internal endpoint.

Practical Configuration Examples

Docker Compose for S3 Production

Deploy NekoImageGallery with S3 backend using Docker Compose:

services:
  neko:
    image: edgeneko/neko-image-gallery:latest
    environment:
      - APP_STORAGE__METHOD=s3
      - APP_STORAGE__S3__BUCKET=my-gallery-bucket
      - APP_STORAGE__S3__PATH=static
      - APP_STORAGE__S3__REGION=us-east-1
      - APP_STORAGE__S3__ENDPOINT_URL=https://s3.amazonaws.com
      - APP_STORAGE__S3__ACCESS_KEY_ID=${S3_ACCESS_KEY}
      - APP_STORAGE__S3__SECRET_ACCESS_KEY=${S3_SECRET_KEY}
      - APP_STORAGE__S3__USER_ENDPOINT_URL=https://cdn.my-gallery.com/static

Supply secrets via Docker secrets or your orchestrator's secret management to avoid exposing credentials in compose files.

Local Development Setup

For local development with filesystem storage, create a .env file:

APP_STORAGE__METHOD=local
APP_STORAGE__LOCAL__PATH=./static

No code changes are required; the StorageService automatically instantiates LocalStorage, and the same API calls work identically across both backends.

Summary

• NekoImageGallery provides three storage backends—local, S3-compatible, and disabled—controlled by the StorageMode enum in app/config.py.
• S3Storage uses the opendal library to support any S3-compatible provider via the S3StorageSettings configuration model.
• Configuration is environment-driven using variables prefixed with APP_STORAGE__S3__, including bucket, endpoint, region, and credentials.
• The user_endpoint_url option enables CDN integration by rewriting presigned URLs to custom domains.
• The unified BaseStorage interface ensures that switching between local filesystem and S3 requires no application code changes.

Frequently Asked Questions

What storage backends does NekoImageGallery support?

NekoImageGallery supports local filesystem storage (StorageMode.LOCAL), S3-compatible object storage (StorageMode.S3), and a disabled mode (StorageMode.DISABLED). The local backend stores files in ./static by default, while the S3 backend uses opendal to connect to AWS S3, MinIO, Ceph, and similar services.

How do I configure S3-compatible storage like MinIO?

Set APP_STORAGE__METHOD=s3 and provide APP_STORAGE__S3__ENDPOINT_URL pointing to your MinIO instance (e.g., http://minio:9000), along with APP_STORAGE__S3__ACCESS_KEY_ID, APP_STORAGE__S3__SECRET_ACCESS_KEY, and APP_STORAGE__S3__BUCKET. The S3Storage class in app/Services/storage/s3_compatible_storage.py automatically handles the connection using these parameters.

Can I use a custom CDN domain with S3 storage?

Yes. Set the APP_STORAGE__S3__USER_ENDPOINT_URL environment variable to your CDN domain (e.g., https://cdn.example.com). When generating presigned URLs, the application rewrites the internal S3 endpoint to this public URL, allowing clients to fetch content through your CDN while the server communicates directly with the storage backend.

What happens if I set the storage method to disabled?

When APP_STORAGE__METHOD is set to disabled, the application instantiates DisabledStorage, which raises NotImplementedError for any storage operation. This mode is useful for API-only deployments where images are managed externally and only referenced by URL in the database.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}