How to Handle S3 Presigned URLs and Multipart Uploads with AWS SDK v3

Use @aws-sdk/s3-request-presigner for temporary GET/PUT URLs, @aws-sdk/s3-presigned-post for browser-based uploads, and @aws-sdk/lib-storage for streaming multipart uploads with automatic parallelization.

The AWS SDK for JavaScript v3 provides a modular, tree-shakable architecture for handling S3 presigned URLs and multipart uploads efficiently. According to the aws/agent-toolkit-for-aws repository, specifically the reference documentation in skills/core-skills/aws-sdk-js-v3-usage/references/s3.md, you can implement secure temporary access and large file transfers using specialized packages that separate concerns like signing, uploading, and polling.

Generating S3 Presigned URLs for GET and PUT Operations

The @aws-sdk/s3-request-presigner package exports the getSignedUrl helper function that creates time-limited URLs for specific S3 operations. These URLs allow clients to access or upload objects without exposing AWS credentials.

Creating Temporary Download Links

To generate a presigned URL for retrieving objects, combine the S3Client with a GetObjectCommand. The expiresIn parameter configures the URL validity period in seconds (defaulting to 900 seconds/15 minutes if omitted).

import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const s3 = new S3Client({ region: "<REGION>" });

const getUrl = await getSignedUrl(
  s3,
  new GetObjectCommand({ Bucket: "<BUCKET>", Key: "<KEY>" }),
  { expiresIn: 3600 }  // 1-hour expiry
);

As implemented in the source reference, you can also constrain the URL by specifying signableHeaders or unhoistableHeaders for x-amz-* headers that must match the client's request.

Uploading Objects with Presigned PUT URLs

For direct client uploads, use PutObjectCommand with the same getSignedUrl function. The generated URL allows HTTP PUT requests to a specific key location.

import { PutObjectCommand } from "@aws-sdk/client-s3";

const putUrl = await getSignedUrl(
  s3,
  new PutObjectCommand({ Bucket: "<BUCKET>", Key: "<KEY>" }),
  { expiresIn: 3600 }
);

Enabling Browser Uploads with Presigned POST

When building HTML form uploads, use the @aws-sdk/s3-presigned-post package and its createPresignedPost function. This method returns a URL and a set of signed fields that browsers can POST directly to S3, supporting dynamic filenames via template substitution.

import { createPresignedPost } from "@aws-sdk/s3-presigned-post";

const { url, fields } = await createPresignedPost(s3, {
  Bucket: "<BUCKET>",
  Key: "uploads/${filename}",  // ${filename} replaced by browser
  Expires: 600,               // 10 minutes
  Conditions: [["content-length-range", 0, 10_485_760]],  // 10 MB limit
  Fields: { acl: "bucket-owner-full-control" }
});

// Browser implementation
const form = new FormData();
Object.entries(fields).forEach(([k, v]) => form.append(k, v));
form.append("file", fileInput.files[0]);

await fetch(url, { method: "POST", body: form });

The Conditions array accepts AWS policy conditions such as content-length-range and Content-Type, while the Fields object specifies additional metadata like ACL settings.

Managing Large Files with Multipart Uploads

For objects larger than 5 MiB or when streaming data of unknown size, the @aws-sdk/lib-storage package provides the Upload class. This utility handles the multipart upload lifecycle automatically, including part sizing, parallelization, and error recovery.

Configuring Upload Parallelization

The Upload constructor accepts tuning parameters that control performance and reliability:

• queueSize: Number of parts to upload concurrently (default: 4)
• partSize: Minimum bytes per part (must be ≥ 5 MiB)
• leavePartsOnError: Boolean indicating whether to preserve unfinished parts for inspection

import { Upload } from "@aws-sdk/lib-storage";
import { S3Client } from "@aws-sdk/client-s3";
import fs from "fs";

const client = new S3Client({ region: "<REGION>" });
const fileStream = fs.createReadStream("large-file.bin");

const upload = new Upload({
  client,
  params: { Bucket: "<BUCKET>", Key: "<KEY>", Body: fileStream },
  queueSize: 4,
  partSize: 5 * 1024 * 1024,  // 5 MiB
  leavePartsOnError: false
});

upload.on("httpUploadProgress", (progress) => {
  console.log(`Uploaded ${progress.loaded} of ${progress.total} bytes`);
});

await upload.done();  // Resolves when multipart assembly completes

The httpUploadProgress event emits progress objects containing loaded and total bytes, enabling real-time UI updates during S3 multipart uploads.

Ensuring Resource Availability with S3 Waiters

The AWS SDK v3 exports waitUntil* functions from @aws-sdk/client-s3 that poll resources until they reach desired states. These waiters abstract retry logic, exponential back-off, and timeout handling.

Available waiters include:

• waitUntilBucketExists: Polls until bucket creation propagates
• waitUntilObjectExists: Confirms object availability after upload
• waitUntilBucketNotExists: Verifies deletion completion

import { waitUntilBucketExists, waitUntilObjectExists } from "@aws-sdk/client-s3";

const s3 = new S3Client({ region: "<REGION>" });

await waitUntilBucketExists(
  { client: s3, maxWaitTime: 60 },  // seconds
  { Bucket: "<BUCKET>" }
);

await waitUntilObjectExists(
  { client: s3, maxWaitTime: 120 },
  { Bucket: "<BUCKET>", Key: "<KEY>" }
);

The maxWaitTime parameter (required) specifies the total polling duration in seconds, while optional minDelay and maxDelay parameters control the polling interval.

Summary

• Presigned GET/PUT URLs: Use @aws-sdk/s3-request-presigner with getSignedUrl() and command objects (GetObjectCommand, PutObjectCommand) to generate temporary access URLs with configurable expiration.
• Browser Uploads: Implement @aws-sdk/s3-presigned-post and createPresignedPost() to generate form-compatible URLs and signed fields for direct browser-to-S3 uploads.
• Multipart Uploads: Handle large files using @aws-sdk/lib-storage and the Upload class, which manages parallel part uploads, progress events, and automatic completion.
• Waiters: Leverage waitUntilBucketExists and waitUntilObjectExists from @aws-sdk/client-s3 to poll for resource readiness with built-in retry logic.

Frequently Asked Questions

How long can S3 presigned URLs remain valid?

S3 presigned URLs can remain valid for up to 7 days when using the AWS Signature Version 4 (SigV4) signing process. When generating URLs with getSignedUrl, specify the expiresIn parameter in seconds; the default is 15 minutes (900 seconds). For longer durations, ensure the IAM credentials used to sign the request remain valid throughout the entire expiration period.

When should I use multipart uploads instead of single-part uploads?

Use multipart uploads when uploading files larger than 100 MB, or when streaming data of unknown size. According to the aws/agent-toolkit-for-aws implementation, the @aws-sdk/lib-storage Upload class automatically initiates multipart uploads for objects exceeding 5 MiB, parallelizing transfers via the queueSize parameter to maximize throughput.

How do I track upload progress with AWS SDK v3?

The Upload class from @aws-sdk/lib-storage emits httpUploadProgress events containing loaded and total byte counts. Attach an event listener using upload.on("httpUploadProgress", callback) to receive progress updates during S3 multipart uploads, enabling real-time progress bars or monitoring dashboards.

What is the difference between presigned POST and presigned PUT?

Presigned PUT generates a URL for a specific HTTP PUT request to a predetermined key, ideal for API clients that know the destination path. Presigned POST (from @aws-sdk/s3-presigned-post) creates a URL and signed form fields that allow browsers to upload files with dynamic filenames via HTML forms, supporting policy conditions like content-type restrictions and file size limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}