# How Macro Configures S3 Buckets for Document Storage with Lifecycle Policies

> Learn how Macro configures S3 buckets for document storage with lifecycle policies. Discover the automation of 30-day Glacier transitions and 90-day expirations.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Macro uses a dedicated S3 bucket for document storage with a 30-day Glacier transition and 90-day expiration policy applied via `put_bucket_lifecycle_configuration`, while runtime services interact through a thin `S3Client` wrapper that abstracts all bucket operations.**

The Macro codebase organizes document persistence around a single **document storage bucket** concept. Rather than embedding lifecycle logic in application code, the platform relies on AWS S3's native lifecycle management, configured at provisioning time through the AWS SDK. This separation keeps services lightweight while ensuring automatic cost optimization and data retention compliance.

## Architecture Overview

Document storage in Macro follows a three-layer pattern: configuration-driven bucket naming, SDK-based lifecycle enforcement, and a client abstraction that hides S3 implementation details from business logic.

### Configuration-Driven Bucket Selection

Every service that handles documents receives its bucket name through the shared configuration system. The key parameter `document_storage_bucket` appears across multiple service entry points.

In [`services/static_file_service/src/service/s3/client.rs`](https://github.com/macro-inc/macro/blob/main/services/static_file_service/src/service/s3/client.rs), the `S3Client` struct stores this bucket name for all subsequent operations:

```rust
pub struct S3Client {
    client: aws_sdk_s3::Client,
    storage_bucket: String,
}

impl S3Client {
    pub fn new(client: aws_sdk_s3::Client, storage_bucket: String) -> Self {
        Self { client, storage_bucket }
    }
}

```

This pattern repeats in [`services/document_storage_service/src/main.rs`](https://github.com/macro-inc/macro/blob/main/services/document_storage_service/src/main.rs), where the bucket name from `config.document_storage_bucket` initializes the service's storage backend.

## Local Development: LocalStack Bucket Provisioning

For local development, Macro spins up LocalStack and creates buckets programmatically. The setup routine in [`tooling/xtask/crates/xtask_local/src/local/localstack.rs`](https://github.com/macro-inc/macro/blob/main/tooling/xtask/crates/xtask_local/src/local/localstack.rs) handles bucket creation, CORS configuration, and event notifications.

### Bucket Creation Loop

```rust
for bucket in resources::BUCKETS {
    let name = bucket.name;
    s3.create_bucket()
        .bucket(name)
        .send()
        .await
        .map(|_| ())?;
}

```

The `resources` module defines `DOC_STORAGE_BUCKET` as one of the buckets created during this startup phase.

### CORS Configuration

After creation, the code immediately applies CORS rules to allow browser-direct access:

```rust
s3.put_bucket_cors()
    .bucket(name)
    .cors_configuration(cors_cfg)
    .send()
    .await?;

```

This enables the frontend to generate presigned URLs and fetch objects without proxying through Macro's API servers.

### S3 Event Notification

The setup registers an `ObjectCreated` notification that forwards to the internal event queue:

```rust
s3.put_bucket_notification_configuration()
    .bucket(resources::DOC_STORAGE_BUCKET)
    .notification_configuration(notif_cfg)
    .send()
    .await?;

```

This triggers downstream processing (thumbnail generation, text extraction) when new documents arrive.

## Production Lifecycle Policy Configuration

While the repository source code for lifecycle management isn't directly exposed—likely managed through infrastructure-as-code outside the visible paths—the intended policy follows Macro's documented SDK specification. The lifecycle configuration uses `put_bucket_lifecycle_configuration` from the AWS SDK, matching the same client patterns seen in local setup.

### Standard Lifecycle Rules

| Rule | Trigger | Action |
|------|---------|--------|
| **Glacier Transition** | 30 days after object creation | Move to `GLACIER` storage class |
| **Permanent Deletion** | 90 days after object creation | Delete object entirely |

This two-tier approach balances accessibility with cost: documents remain in standard storage for a month of active use, shift to cold storage for archival, then disappear completely to satisfy retention limits.

### Lifecycle Configuration Implementation

The AWS SDK call follows this structure (based on Macro's SDK patterns and AWS best practices):

```rust
let lifecycle = LifecycleConfiguration {
    rules: vec![LifecycleRule {
        id: Some("auto-expire".into()),
        filter: LifecycleFilter::Prefix("documents/".into()),
        status: LifecycleRuleStatus::Enabled,
        expiration: Some(LifecycleExpiration::Days(90)),
        transitions: Some(vec![Transition {
            days: Some(30),
            storage_class: StorageClass::Glacier,
        }]),
        ..Default::default()
    }],
};

s3.put_bucket_lifecycle_configuration()
    .bucket(bucket_name)
    .lifecycle_configuration(lifecycle)
    .send()
    .await?;

```

The `documents/` prefix filter ensures lifecycle rules apply only to content uploads, excluding system metadata or configuration objects that might share the bucket.

## Runtime Document Operations

Services interact with S3 exclusively through the `S3Client` wrapper, which encapsulates bucket naming and error handling.

### Client Initialization

```rust
let s3_client = S3Client::new(
    aws_sdk_s3::Client::new(&aws_config),
    config.document_storage_bucket.clone()
);

```

The `S3Client::new` constructor accepts the preconfigured AWS SDK client and the bucket name from service configuration.

### Upload with Streaming

```rust
let key = format!("documents/{}/content.md", document_id);
S3Client::put_object(&s3_client, &key, ByteStream::from(file_bytes))
    .await?;

```

The `put_object` method (implementation in [`services/static_file_service/src/service/s3/client.rs`](https://github.com/macro-inc/macro/blob/main/services/static_file_service/src/service/s3/client.rs)) automatically prepends the correct bucket from the stored `storage_bucket` field.

### Presigned URL Generation

For frontend downloads, the client generates time-limited presigned URLs:

```rust
let url = s3_client
    .put_document_storage_presigned_url(&key, PresignConfig::default())
    .await?;

```

This delegates to `aws_sdk_s3::presigning::PresigningConfig` with the stored bucket context.

## Key Source Files

| Path | Responsibility |
|------|---------------|
| [`services/static_file_service/src/service/s3/client.rs`](https://github.com/macro-inc/macro/blob/main/services/static_file_service/src/service/s3/client.rs) | `S3Client` wrapper with bucket-scoped operations |
| [`tooling/xtask/crates/xtask_local/src/local/localstack.rs`](https://github.com/macro-inc/macro/blob/main/tooling/xtask/crates/xtask_local/src/local/localstack.rs) | LocalStack orchestration: bucket creation, CORS, notifications |
| [`services/document_storage_service/src/main.rs`](https://github.com/macro-inc/macro/blob/main/services/document_storage_service/src/main.rs) | Service bootstrap reading `document_storage_bucket` config |
| [`packages/sdk/specs/storage.json`](https://github.com/macro-inc/macro/blob/main/packages/sdk/specs/storage.json) | SDK contract documenting lifecycle metadata in API responses |

## Summary

- **Configuration centralization**: The `document_storage_bucket` parameter feeds all services from a single configuration source.
- **Provisioning-time lifecycle**: Glacier transition (30 days) and expiration (90 days) are enforced through `put_bucket_lifecycle_configuration`, not application logic.
- **LocalStack parity**: Development environments recreate production bucket behavior including CORS and event notifications.
- **Client abstraction**: The `S3Client` struct eliminates bucket name bookkeeping from business code.
- **Cost optimization**: Automated lifecycle rules reduce storage spend without service-level intervention.

## Frequently Asked Questions

### How does Macro handle S3 bucket naming across environments?

Each environment sets `document_storage_bucket` in its configuration, and `S3Client::new` receives this value at service startup. Local development uses LocalStack with hardcoded bucket names in `resources::BUCKETS`, while production uses externally provisioned bucket names injected through environment configuration.

### Where is the lifecycle policy actually applied if not in the visible source code?

The lifecycle configuration likely executes through infrastructure-as-code (Terraform, CloudFormation, or CDK) outside the application repository, consistent with Macro's pattern of provisioning AWS resources separately from runtime services. The policy matches the SDK-documented behavior: 30-day Glacier transition, 90-day expiration.

### Why does the local setup skip lifecycle configuration?

LocalStack's community edition has limited lifecycle support, and Macro's local development prioritizes functional parity for upload/download flows over cost-optimization features. The lifecycle rules are integration-tested in staging environments with real S3.

### Can lifecycle rules target specific document types?

Yes. The `LifecycleFilter::Prefix("documents/")` pattern shown in the configuration allows prefix-based filtering. Macro could extend this with multiple rules for different `documents/{type}/` prefixes if retention requirements vary by content category.