AWS SDK v3 S3 Presigned URLs, Multipart Uploads, and Waiters: Implementation Guide

The AWS SDK for JavaScript v3 provides modular, tree-shakable packages that enable you to generate presigned URLs, stream large files via multipart uploads, and poll resource states using waiters, as documented in the aws/agent-toolkit-for-aws repository.

The @aws-sdk/client-s3 package and its companion libraries offer a composable architecture for handling complex S3 operations without bundling unnecessary code. According to the source code in skills/core-skills/aws-sdk-js-v3-usage/references/s3.md, these utilities implement distinct patterns for temporary access grants, efficient large file transfers, and asynchronous resource readiness checks.

Presigned URLs for GET and PUT Operations

Presigned URLs provide temporary, secure access to S3 objects without exposing AWS credentials to end users.

Generating Signed URLs with getSignedUrl

The @aws-sdk/s3-request-presigner package exports the getSignedUrl helper, which creates time-limited URLs valid for a configurable duration (defaulting to 15 minutes). You construct an S3Client, instantiate a command (GetObjectCommand or PutObjectCommand), and pass both to the signing utility.

import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const s3 = new S3Client({ region: "<REGION>" });

const getUrl = await getSignedUrl(
  s3,
  new GetObjectCommand({ Bucket: "<BUCKET>", Key: "<KEY>" }),
  { expiresIn: 3600 } // 1-hour expiry
);

const putUrl = await getSignedUrl(
  s3,
  new PutObjectCommand({ Bucket: "<BUCKET>", Key: "<KEY>" }),
  { expiresIn: 3600 }
);

Configuring Expiration and Headers

Optional parameters allow you to sign additional headers such as Content-Type by specifying signableHeaders or unhoistableHeaders for x-amz-* headers. This ensures that clients using the presigned URL must match the exact headers used during signing, adding an extra layer of validation.

Browser-Based Uploads with Presigned POST

For direct browser uploads, presigned POST requests allow HTML forms to upload files directly to S3 without intermediary servers handling the bytes.

Creating HTML Form Uploads with createPresignedPost

The @aws-sdk/s3-presigned-post package provides createPresignedPost, which returns a URL and a set of signed form fields. The browser substitutes the ${filename} placeholder in the key template and submits via standard FormData.

import { createPresignedPost } from "@aws-sdk/s3-presigned-post";

const { url, fields } = await createPresignedPost(s3, {
  Bucket: "<BUCKET>",
  Key: "uploads/${filename}", // Browser replaces ${filename}
  Expires: 600, // 10 minutes
  Conditions: [["content-length-range", 0, 10_485_760]],
  Fields: { acl: "bucket-owner-full-control" }
});

// Browser implementation:
const form = new FormData();
Object.entries(fields).forEach(([k, v]) => form.append(k, v));
form.append("file", fileInput.files[0]);

await fetch(url, { method: "POST", body: form });

This method supports constraints like content-length ranges and ACL settings, enforcing security policies at the S3 level.

Multipart Uploads for Large Files

Objects larger than 5 MiB or streams of unknown size require multipart upload handling to maximize throughput and reliability.

Streaming Uploads with @aws-sdk/lib-storage

The Upload class from @aws-sdk/lib-storage abstracts the multipart upload lifecycle, automatically handling part sizing, parallel transfers, and error recovery. You provide an S3Client, upload parameters (Bucket, Key, Body), and optional tuning values.

import { Upload } from "@aws-sdk/lib-storage";
import { S3Client } from "@aws-sdk/client-s3";
import fs from "fs";

const client = new S3Client({ region: "<REGION>" });
const fileStream = fs.createReadStream("large-file.bin");

const upload = new Upload({
  client,
  params: { Bucket: "<BUCKET>", Key: "<KEY>", Body: fileStream },
  queueSize: 4, // Parallel uploads
  partSize: 5 * 1024 * 1024, // 5 MiB minimum
  leavePartsOnError: false
});

upload.on("httpUploadProgress", (progress) => console.log(progress));
await upload.done(); // Resolves when assembly is complete

Tuning Parallel Uploads and Progress Tracking

The Upload class accepts three critical configuration options:

• queueSize – Controls how many parts upload simultaneously (default 4).
• partSize – Defines minimum part size, must be ≥ 5 MiB.
• leavePartsOnError – Determines whether to retain unfinished parts for inspection upon failure.

The httpUploadProgress event emits progress details, enabling real-time UI updates for user-facing upload interfaces.

Waiters for Resource State Management

Waiters automate the polling logic required to verify that S3 resources reach desired states before proceeding with dependent operations.

Polling with waitUntilBucketExists and waitUntilObjectExists

The @aws-sdk/client-s3 package exports waitUntil* functions that implement exponential back-off and retry logic. You configure these with a client, required maxWaitTime (in seconds), and optional minDelay/maxDelay parameters.

import { S3Client, waitUntilBucketExists, waitUntilObjectExists } from "@aws-sdk/client-s3";

const s3 = new S3Client({ region: "<REGION>" });

await waitUntilBucketExists(
  { client: s3, maxWaitTime: 60 }, // 60 seconds max
  { Bucket: "<BUCKET>" }
);

await waitUntilObjectExists(
  { client: s3, maxWaitTime: 120 },
  { Bucket: "<BUCKET>", Key: "<KEY>" }
);

These functions abstract the underlying HeadBucket and HeadObject calls, aggregating errors and handling rate limits automatically.

Summary

• Presigned URLs – Use @aws-sdk/s3-request-presigner with getSignedUrl to generate temporary GET/PUT URLs with configurable expiration and header signing.
• Presigned POST – Leverage @aws-sdk/s3-presigned-post and createPresignedPost to enable direct browser uploads with policy constraints.
• Multipart Uploads – Implement @aws-sdk/lib-storage Upload class for automatic parallelization, progress events, and error handling of large files.
• Waiters – Utilize waitUntilBucketExists and waitUntilObjectExists from @aws-sdk/client-s3 to poll resource states with built-in retry logic.

Frequently Asked Questions

How do I generate a presigned URL that expires after a specific time?

Pass the expiresIn parameter (in seconds) to the getSignedUrl function from @aws-sdk/s3-request-presigner. For example, { expiresIn: 3600 } creates a URL valid for one hour. The function returns a signed URL string that clients can use without AWS credentials until expiration.

What is the difference between presigned URLs and presigned POST for S3 uploads?

Presigned URLs (GET/PUT) work for simple HTTP requests where you control the client code, while presigned POST generates a URL and signed form fields for HTML form submissions. Use createPresignedPost when building browser-based uploads that submit directly to S3 via FormData, as it supports file-specific substitutions like ${filename} and content-length constraints.

When should I use @aws-sdk/lib-storage instead of standard PutObjectCommand?

Use the Upload class from @aws-sdk/lib-storage when transferring files larger than 5 MiB, streaming data of unknown size, or requiring automatic multipart parallelization. The standard PutObjectCommand loads the entire payload into memory, while Upload handles part splitting, parallel queueSize management, and progress tracking via httpUploadProgress events.

How do waiters handle timeouts and retry logic in AWS SDK v3?

Waiters like waitUntilObjectExists accept a maxWaitTime parameter (required) and optional minDelay/maxDelay values to control polling frequency. The SDK implements exponential back-off between attempts, automatically retrying throttled requests until the resource meets the condition or the timeout expires, eliminating manual polling loops in your application code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}