# Static File Service and CDN Integration in the Macro Platform

> Discover the Macro static file service and its CDN integration. Learn how it uses Amazon S3, CloudFront, and Lambda for global asset distribution and on-the-fly image optimization.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: how-to-guide
- Published: 2026-08-18

---

**The Macro static file service combines Amazon S3, CloudFront, and Lambda to provide a globally distributed CDN for binary assets with on-the-fly image optimization.**

This repository (`macro-inc/macro`) implements a fully-managed storage layer for binary assets that exposes files through a fast, globally-distributed CDN. The architecture leverages AWS infrastructure to store files securely, serve them efficiently at the edge, and transform images dynamically without manual intervention.

## Core Architecture Components

The static file service integrates multiple AWS services into a cohesive delivery pipeline. Each component is defined in TypeScript using Pulumi infrastructure-as-code, ensuring reproducible deployments across development, staging, and production environments.

### S3 Bucket and DynamoDB Metadata Store

At the foundation lies the **`static-file-storage-<stack>`** S3 bucket, configured with versioning enabled and lifecycle rules for managing old versions. The bucket definition appears in [[`infra/stacks/static-file-service/static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts)](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts) (lines 21‑49), where CORS policies allow cross-origin requests from any domain.

Metadata tracking occurs through a DynamoDB table created in the same stack (lines 95‑121). This table stores critical file attributes including `owner_id`, MIME type, and the S3 key mapping. The table name passes through the `dynamoDbTableName` parameter, utilizing a primary key schema of `file_id` with a Global Secondary Index (GSI) on `owner_id` for efficient owner-based queries.

### ECS Fargate API Layer

The Axum-based HTTP API runs on **ECS Fargate** behind an Application Load Balancer. The service definition in [[`static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/static-file-service.ts)](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts) (lines 42‑62) specifies container builds using `docker/Dockerfile` with the `STATIC_FILE_SERVICE` build argument. This architecture handles presigned URL generation and metadata management while delegating actual file delivery to the CDN layer.

Security groups segregate the ALB from the service containers, with explicit ingress and egress rules defined in the `initializeSecurityGroups` function. IAM roles aggregate granular policies for S3 access, DynamoDB operations, ECR pulls, and Secrets Manager retrieval.

### CloudFront CDN and Origin Access Controls

The global distribution layer resides in [[`infra/stacks/static-file-service/distribution.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/distribution.ts)](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/distribution.ts). Unlike legacy implementations using bucket policies, this architecture employs **Origin Access Controls (OAC)** for both the S3 bucket and Lambda origins (lines 48‑231).

The CloudFront distribution implements a fail-over **origin group** named `image-optimization-group`, where S3 serves as the primary origin and the image optimizer Lambda functions as the secondary. Custom error handling routes 403 responses from S3 to the Lambda origin automatically. A response-headers policy adds security headers at the edge, while cache behaviors specify `minTtl: 0` and `defaultTtl: 31536000` for optimal performance.

Critical security enforcement occurs through bucket policies (lines 90‑107) that validate the `AWS:SourceArn` condition matches the distribution ARN explicitly. This ensures objects remain inaccessible outside the CDN pathway.

### Image Optimization Lambda

Dynamic image transformation occurs through a dedicated Lambda function defined in [[`static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/static-file-service.ts)](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts) (lines 151‑210). When triggered, the function fetches the original image from S3, performs resizing or format conversion, writes the transformed version back to a deterministic key pattern (`file/<uuid>/<format>`), and returns the optimized asset.

Route 53 provides friendly DNS resolution at `static-file-service.<stack>.macro.com`, creating a seamless endpoint for client applications (lines 82‑94).

## CDN Integration Flow

The complete data flow follows four distinct phases that abstract storage complexity from client applications.

1. **Presigned Upload Generation** — Clients request upload capabilities via `POST /api/file`. The ECS service writes a DynamoDB entry using the helper logic in [[`crates/s3_key/src/static_file_key.rs`](https://github.com/macro-inc/macro/blob/main/crates/s3_key/src/static_file_key.rs)](https://github.com/macro-inc/macro/blob/main/crates/s3_key/src/static_file_key.rs), generates an S3 presigned URL, and returns a `PutFileResponse` containing the `upload_url` and `file_id`.

2. **Direct S3 Upload** — The client uploads binary data directly to the presigned URL, bypassing the application servers to handle large payloads efficiently. URLs expire after 15 minutes by default.

3. **Edge Caching and Delivery** — File retrieval occurs via `https://<cdn-domain>/file/<file_id>`. CloudFront serves cached objects directly when available, pulling from the S3 origin only on cache misses.

4. **Dynamic Image Transformation** — Requests containing query parameters like `?size=300` trigger the **viewer-request function** (`image-url-rewrite`) which rewrites the URI to `/size=<value>` (lines 22‑33 in [`distribution.ts`](https://github.com/macro-inc/macro/blob/main/distribution.ts)). CloudFront attempts S3 first; upon receiving a 403 (indicating the transformed version does not exist), it fails over to the image optimizer Lambda. The Lambda generates the resized asset, stores it in S3, and returns the optimized image. Subsequent requests hit the edge cache directly without invoking Lambda again.

## Security Implementation

Security relies on modern AWS paradigms rather than public bucket configurations. **Origin Access Controls (OAC)** replace legacy origin access identities, establishing cryptographically secure relationships between CloudFront and S3. The bucket policy explicitly denies access unless the `AWS:SourceArn` matches the specific CloudFront distribution ARN, preventing direct S3 URL access.

Container security includes dedicated security groups for the ALB and ECS tasks, with the log configuration forwarding to Datadog via a `log_router` sidecar container. SQS event notifications fire when new objects arrive, enabling asynchronous downstream processing such as search indexing or document analysis.

## Local Development with NGINX

The repository provides local parity through an NGINX configuration that mimics CloudFront behavior. The file [[`infra/local/nginx/static-file-cdn.conf`](https://github.com/macro-inc/macro/blob/main/infra/local/nginx/static-file-cdn.conf)](https://github.com/macro-inc/macro/blob/main/infra/local/nginx/static-file-cdn.conf) proxies `/file/*` routes to LocalStack's S3 endpoint while applying identical CORS and security headers to the production CDN.

```bash

# Start the local development stack

docker compose up -d

# Access files through the local CDN simulator

curl http://localhost/file/123e4567-e89b-12d3-a456-426614174000

```

This configuration enables rapid iteration without AWS round-trips, routing requests to either the local Axum service or LocalStack depending on the path pattern.

## Client Integration Examples

### Uploading Files via Rust SDK

The `macro_sdk` crate abstracts the HTTP API defined in [[`packages/sdk/specs/static-files.json`](https://github.com/macro-inc/macro/blob/main/packages/sdk/specs/static-files.json)](https://github.com/macro-inc/macro/blob/main/packages/sdk/specs/static-files.json) (lines 13‑60).

```rust
use macro_sdk::client::MacroClient;
use macro_sdk::models::static_files::{PutFileRequest, PutFileResponse};

let client = MacroClient::new("https://api.macro.com")?;
let req = PutFileRequest {
    file_name: "avatar.png".into(),
    content_type: Some("image/png".into()),
    extension_data: None,
};

let resp: PutFileResponse = client
    .post("/api/file")
    .json(&req)
    .send()
    .await?
    .json()
    .await?;

println!("Upload to: {}", resp.upload_url);

```

### Direct S3 Upload

```bash
curl -X PUT \
  -H "Content-Type: image/png" \
  --upload-file ./avatar.png \
  "$UPLOAD_URL"

```

### Retrieving Optimized Images

```bash

# Original file

curl https://static-file-service.dev.macro.com/file/123e4567-e89b-12d3-a456-426614174000 -o avatar.png

# Resized to 300px width (triggers Lambda optimization on first request)

curl "https://static-file-service.dev.macro.com/file/123e4567-e89b-12d3-a456-426614174000?size=300" -o avatar_small.png

```

## Summary

- **Hybrid Storage Architecture**: The service combines S3 for durable storage, DynamoDB for metadata indexing, and ECS Fargate for API logic, creating a separation of concerns that optimizes cost and performance.
- **Origin Access Controls**: Modern OAC implementation ensures S3 objects remain private while allowing CloudFront edge locations to serve content securely.
- **Intelligent Caching**: The origin group pattern with CloudFront cache behaviors minimizes Lambda invocations by persisting transformed images back to S3 and serving subsequent requests from the edge.
- **Developer Parity**: Local NGINX configuration in [`infra/local/nginx/static-file-cdn.conf`](https://github.com/macro-inc/macro/blob/main/infra/local/nginx/static-file-cdn.conf) provides identical routing logic for local development, ensuring consistent behavior across environments.

## Frequently Asked Questions

### How does the image optimization handle concurrent requests for the same image size?

The architecture leverages CloudFront's cache behaviors and the origin group's fail-over mechanism. When multiple concurrent requests arrive for a non-existent transformed image, CloudFront forwards one request to the Lambda optimizer while queueing others. The Lambda writes the transformed image to a deterministic S3 key (`file/<uuid>/<format>`) before returning. Subsequent requests hit the S3 origin directly once the object propagates, avoiding redundant Lambda executions.

### What prevents direct access to files via S3 URLs outside the CDN?

The bucket policy in [`distribution.ts`](https://github.com/macro-inc/macro/blob/main/distribution.ts) (lines 90‑107) implements a strict `AWS:SourceArn` condition that validates requests originate from the specific CloudFront distribution ARN. Combined with **Origin Access Controls (OAC)**, this ensures the S3 bucket rejects any direct HTTP requests lacking the cryptographic signatures provided by CloudFront's edge infrastructure.

### How does the local NGINX configuration mirror production behavior?

The [`static-file-cdn.conf`](https://github.com/macro-inc/macro/blob/main/static-file-cdn.conf) file replicates CloudFront's path routing logic by proxying `/file/*` requests to LocalStack's S3 implementation while injecting identical CORS headers and security policies. Other paths route to the local Axum service, creating a faithful simulation of the production request flow without requiring AWS credentials or network latency.

### Where is the S3 key generation logic implemented?

Deterministic key construction resides in [[`crates/s3_key/src/static_file_key.rs`](https://github.com/macro-inc/macro/blob/main/crates/s3_key/src/static_file_key.rs)](https://github.com/macro-inc/macro/blob/main/crates/s3_key/src/static_file_key.rs). This shared library ensures consistent key formats between the upload presigning logic in the ECS service and the transformation logic in the Lambda optimizer, preventing path mismatches that could break file retrieval.