# Image Optimizer Service S3 Integration in Macro: CloudFront Lambda Origin Pattern > Discover how Macro integrates S3 with CloudFront Lambda for on-the-fly image optimization. Learn about the origin group pattern and automatic caching of transformed images. - Repository: [Macro/macro](https://github.com/macro-inc/macro) - Tags: internals - Published: 2026-08-18 --- **Macro implements on-the-fly image optimization using a CloudFront origin group where S3 serves as the primary origin and a custom Lambda function acts as a secondary origin, automatically caching transformed images back to S3 for subsequent requests.** The Macro repository utilizes a serverless architecture to deliver optimized images without pre-rendering millions of variants. By combining Amazon CloudFront, S3, and Lambda, the system generates image transformations—such as format conversion and resizing—on demand while storing results durably in S3. This pattern minimizes compute costs by ensuring that only the first request for a specific transformation invokes the Lambda function, with all subsequent requests served directly from the S3 bucket. ## CloudFront Origin Group Fail-Over Architecture The core of Macro's image optimizer relies on a **CloudFront origin group** configured in [`infra/stacks/static-file-service/distribution.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/distribution.ts). The group establishes S3 as the primary origin and the Lambda Function URL as the secondary, creating an automatic fail-over pattern. When a client requests a transformed image using a URL pattern like `/file/{uuid}/format=webp,width=300`, CloudFront first attempts to fetch the object from S3. Since the transformed variant does not yet exist, S3 returns a **403 Forbidden** response. CloudFront interprets this as a cache miss and automatically routes the request to the secondary origin—the Image Optimizer Lambda. This origin group configuration ensures: - **Zero cold-start latency** for cached images (S3 serves directly) - **Automatic processing** for uncached variants (Lambda invoked only when needed) - **Deterministic cache keys** based on the full URL path including transformation parameters ## IAM Role and S3 Access Policies The Lambda function operates under a dedicated IAM role defined in [`infra/stacks/static-file-service/static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts). The role `-image-optimizer-role` trusts the Lambda service principal and attaches the AWS-managed `AWSLambdaBasicExecutionRole` for basic CloudWatch logging. An inline policy grants least-privilege access to the S3 bucket: ```typescript new aws.iam.RolePolicy(`${SERVICE_NAME}-image-optimizer-s3-policy`, { role: imageOptimizerRole.id, policy: staticFilesBucket.arn.apply(bucketArn => JSON.stringify({ Version: "2012-10-17", Statement: [ { Effect: "Allow", Action: "s3:GetObject", Resource: `${bucketArn}/*` }, { Effect: "Allow", Action: "s3:PutObject", Resource: `${bucketArn}/file/*/*` }, ], }) ), }); ``` The permissions are scoped specifically: - **`s3:GetObject`** on the entire bucket allows reading original source images - **`s3:PutObject`** restricted to the `file/*/*` prefix ensures transformed images are written only to designated cache locations ## Lambda Function Configuration The Image Optimizer Lambda is defined as a **provided.al2023** runtime function, compiled as a custom binary and packaged at `target/lambda/image_optimizer/bootstrap.zip`. This Rust-based implementation (implied by the bootstrap naming convention) receives the target bucket name via the `BUCKET` environment variable. ```typescript const imageOptimizerLambda = new aws.lambda.Function( `${SERVICE_NAME}-image-optimizer`, { name: `${SERVICE_NAME}-image-optimizer-${stack}`, role: imageOptimizerRole.arn, handler: "bootstrap", runtime: "provided.al2023", code: new pulumi.asset.FileArchive(`${REPO_ROOT}/target/lambda/image_optimizer/bootstrap.zip`), timeout: 30, memorySize: 1536, environment: { variables: { BUCKET: STATIC_FILE_BUCKET } }, tags: this.tags, }, { parent: this } ); ``` A **FunctionUrl** with `AWS_IAM` authorization type exposes the Lambda to CloudFront without requiring an API Gateway: ```typescript const imageOptimizerUrl = new aws.lambda.FunctionUrl( `${SERVICE_NAME}-image-optimizer-url`, { functionName: imageOptimizerLambda.name, authorizationType: "AWS_IAM", }, { parent: this } ); ``` ## CloudFront Distribution Wiring The `StaticFileCloudFront` construct in [`infra/stacks/static-file-service/static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts) binds all components together. This construct receives both the S3 bucket reference and the `imageOptimizerUrl`, configuring the distribution with the origin group logic. The distribution instantiation passes the Lambda's function name and URL: ```typescript const distribution = new StaticFileCloudFront( `static-files-${stack}`, { bucket: staticFilesBucket, imageOptimizerUrl: imageOptimizerUrl.functionUrl, imageOptimizerFunctionName: imageOptimizerLambda.name, // …custom domain, not-found page, and other settings… }, { dependsOn: notFound } ); ``` Inside the `StaticFileCloudFront` construct (defined in [`distribution.ts`](https://github.com/macro-inc/macro/blob/main/distribution.ts)), cache behaviors are configured to route traffic based on path patterns, ensuring that requests matching the image optimizer signature trigger the origin group logic. ## End-to-End Request Flow The complete lifecycle of an image optimization request follows this sequence: 1. **Client Request**: Browser requests `https://static-files.macro.com/file/123e4567-e89b-12d3-a456-426614174000/format=webp,width=300` 2. **S3 Check**: CloudFront forwards to S3 primary origin; object does not exist, returning **403** 3. **Lambda Invocation**: CloudFront fails over to the Lambda origin via the Function URL 4. **Transformation**: Lambda downloads the original image from S3 using `GetObject`, performs the transformation (WebP conversion + resize), and streams the result 5. **Cache Write**: Lambda writes the transformed image to S3 at the exact key `file/123e4567-e89b-12d3-a456-426614174000/format=webp,width=300` using `PutObject` 6. **Client Response**: Lambda returns the image bytes to CloudFront, which delivers to the client and caches at the edge 7. **Subsequent Requests**: Future requests for the same URL hit the now-existing S3 object directly, bypassing Lambda entirely The runtime code in [`services/static_file_service/src/api/event/s3_create.rs`](https://github.com/macro-inc/macro/blob/main/services/static_file_service/src/api/event/s3_create.rs) specifically filters out optimizer-specific keys (`file/{uuid}/format=webp,width=300`) from S3 event processing, preventing the system from treating generated thumbnails as new uploads requiring further processing. ## Summary - **Macro's image optimizer service** uses a CloudFront origin group with S3 as primary and Lambda as secondary to minimize compute costs. - **IAM roles** in [`static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/static-file-service.ts) grant the Lambda scoped permissions to read originals and write transformed images to specific S3 prefixes. - **Lambda configuration** uses the `provided.al2023` runtime with a compiled binary, exposed via a Function URL with IAM authorization. - **Deterministic caching** ensures transformed images are stored at predictable S3 keys, allowing subsequent requests to bypass the Lambda entirely. - **Runtime filters** prevent the system from processing generated thumbnails as new source files. ## Frequently Asked Questions ### How does Macro handle the first request for an image transformation? When a client requests a transformed image that does not exist in S3, CloudFront receives a 403 response from the primary S3 origin and automatically fails over to the Image Optimizer Lambda. The Lambda downloads the original, applies the requested transformations (format, dimensions), writes the result to the same S3 path that was originally requested, and returns the image to the client. Subsequent requests hit the cached object directly in S3. ### What permissions does the Image Optimizer Lambda need for S3 integration? According to [`infra/stacks/static-file-service/static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/static-file-service/static-file-service.ts), the Lambda requires `s3:GetObject` permission on the entire bucket to read source images, and `s3:PutObject` permission restricted to the `file/*/*` prefix to store transformed outputs. The role also attaches `AWSLambdaBasicExecutionRole` for CloudWatch logging. ### Why does Macro use a Lambda Function URL instead of API Gateway? The implementation uses a **FunctionUrl** with `AWS_IAM` authorization (defined in [`static-file-service.ts`](https://github.com/macro-inc/macro/blob/main/static-file-service.ts) lines 123-130) because it provides a direct HTTPS endpoint for the Lambda without the cost or complexity of API Gateway. CloudFront integrates directly with this URL as a custom origin, leveraging the origin group pattern to handle the S3-Lambda fail-over logic natively. ### How does the system prevent infinite processing loops for generated images? The runtime code in [`services/static_file_service/src/api/event/s3_create.rs`](https://github.com/macro-inc/macro/blob/main/services/static_file_service/src/api/event/s3_create.rs) specifically filters S3 create events to exclude keys matching the optimizer pattern `file/{uuid}/format=webp,width=300`. This ensures that when the Lambda writes a transformed image back to S3, the system does not treat it as a new upload requiring further optimization.