# How to Configure Asset Storage in OpenMAIC: PostgreSQL vs S3 Setup Guide

> Configure OpenMAIC asset storage with PostgreSQL or S3. Learn how to set up your environment variable for seamless binary asset management in your OpenMAIC deployment.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-09

---

**OpenMAIC determines where binary assets are stored at runtime based on the `ASSET_S3_BUCKET` environment variable, defaulting to PostgreSQL for local deployments or routing to Amazon S3 when a valid bucket name is configured.**

Asset storage configuration in OpenMAIC relies on a pluggable **asset byte store** architecture that keeps the persistence layer agnostic to underlying storage mechanics. The system supports two distinct backends— PostgreSQL for simplicity or Amazon S3 for scalability—selected automatically without code changes based on environment variables.

## Runtime Storage Selection Logic

The storage backend is resolved in [`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts) through a conditional check on `process.env.ASSET_S3_BUCKET`. This design allows the same deployment artifact to run in development against a local database or in production against object storage.

### PostgreSQL Default Storage

When `ASSET_S3_BUCKET` is unset or empty, OpenMAIC stores asset bytes directly in the application database using **`PgAssetByteStore`**. The `lazyAssetByteStore` function returns a **`PgForwardedByteStore`** instance that wraps PostgreSQL connections and exposes transaction-pinned methods (`writeWith`, `readWith`, `deleteWith`). This ensures that asset operations participate in database transactions, maintaining ACID compliance with metadata records.

### Amazon S3 Scalable Storage

When `ASSET_S3_BUCKET` contains a valid bucket name, the system dynamically imports `@openmaic/storage/asset/s3-bytes` via **`loadS3AssetByteStore`** and instantiates **`S3AssetByteStore`**. This implementation sets the flag `writesOutsideRegistryDatabase: true` because S3 writes cannot participate in PostgreSQL transactions, requiring the persistence layer to handle potential inconsistencies explicitly.

## Core Configuration in asset-byte-store.ts

The [`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts) file contains three critical functions that govern asset storage configuration.

### Bucket Name Validation

The **`configuredS3Bucket(value)`** function sanitizes and validates the environment variable before use. It trims whitespace, enforces AWS naming constraints (3–63 characters, allowed character set, reserved prefix/suffix restrictions), and throws descriptive errors for malformed bucket names. This validation prevents runtime failures from invalid configuration strings.

### Lazy Initialization Pattern

The **`lazyAssetByteStore(bucketValue, queryable)`** factory returns an `AssetByteStore` that defers actual construction until the first asset request. This laziness ensures that a misconfigured S3 bucket does not crash the server at startup—only asset-related endpoints fail, leaving other application functions operational.

```typescript
import { lazyAssetByteStore } from '@/lib/persistence/asset-byte-store';
import { getPgQueryable } from '@/lib/persistence/pg';

const queryable = await getPgQueryable();
export const assetByteStore = lazyAssetByteStore(
  process.env.ASSET_S3_BUCKET, 
  queryable
);

// First invocation triggers backend selection
const assetId = await assetByteStore.write(fileHash, fileBuffer);

```

### Transaction Boundary Handling

PostgreSQL storage supports transactional asset operations through `writeWith`, `readWith`, and `deleteWith` methods. S3 storage lacks these methods since object storage operations cannot roll back with database transactions, requiring application-level compensation logic in [`lib/persistence/server-provider.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/server-provider.ts).

## Client Egress Configuration

Beyond storage location, OpenMAIC controls how assets reach clients via the **`ASSET_BYTE_EGRESS`** environment variable. The routing logic resides in [`lib/persistence/server-provider.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/server-provider.ts) and the behavior is validated in [`tests/persistence/route.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/persistence/route.test.ts).

When set to **`redirect`**, the server generates pre-signed S3 URLs and returns HTTP 302 responses, offloading bandwidth to AWS. Any other value (or when unset) triggers direct streaming, where OpenMAIC reads bytes from the configured store and pipes them through the application server.

## Production Configuration Examples

Configure your deployment environment variables to select the appropriate storage backend:

```bash

# PostgreSQL-only deployment (development/small scale)

ASSET_S3_BUCKET=

# S3-backed production deployment

ASSET_S3_BUCKET=acme-openmaic-assets-prod
ASSET_BYTE_EGRESS=redirect  # Optional: use signed URLs for downloads

AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=yyy

```

Implementation in application bootstrap:

```typescript
// lib/persistence/asset-init.ts
import { lazyAssetByteStore } from './asset-byte-store';
import { getQueryable } from './pg';

export const store = lazyAssetByteStore(
  process.env.ASSET_S3_BUCKET,
  getQueryable()
);

```

## Summary

- **Environment-driven selection**: Set `ASSET_S3_BUCKET` to choose between PostgreSQL (`PgAssetByteStore`) and S3 (`S3AssetByteStore`) implementations in [`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts).
- **Lazy initialization**: The `lazyAssetByteStore` factory defers backend instantiation until first use, preventing startup crashes from invalid S3 configuration.
- **Transaction semantics**: PostgreSQL storage supports transaction-pinned methods (`writeWith`, `readWith`), while S3 storage sets `writesOutsideRegistryDatabase: true` indicating non-transactional behavior.
- **Client delivery modes**: Configure `ASSET_BYTE_EGRESS=redirect` for signed S3 URLs or omit for direct streaming through the application server.

## Frequently Asked Questions

### How does OpenMAIC validate S3 bucket names?

The `configuredS3Bucket` function in [`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts) enforces AWS naming rules including length constraints (3–63 characters), allowed characters (lowercase letters, numbers, hyphens, periods), and prohibitions against reserved prefixes like `xn--` or suffixes like `-s3alias`. Invalid configurations throw immediate errors with descriptive messages.

### What happens if I configure an invalid S3 bucket but don't use asset features?

Due to the lazy initialization pattern in `lazyAssetByteStore`, the application starts successfully even with malformed `ASSET_S3_BUCKET` values. The error only surfaces when code first attempts to read or write an asset, isolating the failure to asset-specific requests while keeping the rest of the system operational.

### Can I migrate assets from PostgreSQL to S3 after deployment?

The source code analysis does not reveal built-in migration utilities. Since `PgAssetByteStore` and `S3AssetByteStore` implement the same interface but store bytes differently, migration would require custom scripts to read from the PostgreSQL store via `readWith` methods and write to S3 via the S3 store, followed by updating the `ASSET_S3_BUCKET` environment variable.

### Does OpenMAIC support storage backends other than PostgreSQL and S3?

According to the current implementation in [`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts), only two backends are implemented: `PgAssetByteStore` for PostgreSQL and `S3AssetByteStore` for Amazon S3. The architecture uses a pluggable interface (`AssetByteStore`), but no additional adapters (such as Azure Blob or GCS) are present in the analyzed codebase.