Hugo Deploy Command: How It Deploys to AWS S3, GCS, and Azure Blob Storage

Hugo's deploy command uses the Go Cloud Development Kit (gocloud.dev/blob) to provide a unified, cloud-agnostic deployment engine that syncs static sites to Amazon S3, Google Cloud Storage, and Azure Blob Storage through URL-based configuration and delta-diffing.

The hugo deploy command, implemented in the gohugoio/hugo repository, eliminates the need for external CI/CD tools by providing native support for deploying static sites to major cloud object storage providers. It handles authentication through standard cloud SDK environment variables, calculates precise file diffs using MD5 hashes, and supports concurrent uploads with optional CDN cache invalidation.

Configuration and URL Schemes for Cloud Storage

Deployment targets are defined in your site's configuration file (config.toml, config.yaml, or config.json) under the deployment key. Each target requires a destination URL following the Go Cloud blob URL scheme, which determines the provider and connection parameters.

Amazon S3 Configuration

For AWS S3 deployments, use the s3:// scheme with optional query parameters for region and credentials:

[[deployment.targets]]
  name = "prod-s3"
  url = "s3://my-prod-bucket?region=us-west-2"
  cloudFrontDistributionID = "E1A2B3C4D5E6F7"
  stripIndexHTML = true

The region parameter is required unless implicit in your AWS credential chain. Authentication relies on the standard AWS credential provider chain (environment variables, IAM roles, or ~/.aws/credentials).

Google Cloud Storage Configuration

Google Cloud Storage uses the gs:// scheme with optional prefix parameters for subdirectory deployment:

[[deployment.targets]]
  name = "prod-gcs"
  url = "gs://my-prod-gcs?prefix=site/"
  googleCloudCDNOrigin = "my-gcp-project/origin"
  stripIndexHTML = true

GCP authentication uses Application Default Credentials or the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a service account key.

Azure Blob Storage Configuration

Azure Blob storage uses the azblob:// scheme referencing the container name:

[[deployment.targets]]
  name = "prod-azure"
  url = "azblob://my-azure-container"
  stripIndexHTML = true

Azure authentication requires the AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY environment variables, or uses Azure CLI credentials when available.

The Deployment Architecture

When hugo deploy executes, the command layer in commands/deploy.go instantiates a Deployer struct that orchestrates the entire synchronization process through the Go Cloud SDK abstraction layer.

Initialization and Deployer Creation

The entry point deploy.New() in deploy/deploy.go initializes the deployment pipeline:

// commands/deploy.go → runDeploy()
d, err := deploy.New(cfg, logger, cfg.BaseFs.Source)

This function extracts the deployment configuration section, selects the target (defaulting to the first defined target unless overridden by --target), and returns a *Deployer populated with:

• localFs: The source filesystem pointing to the built site in publishDir
• bucket: A lazily-initialized *blob.Bucket handle
• target: The parsed deployconfig.Target containing include/exclude globs and CDN settings
• cfg: Global deployment settings including workers, force, and dryRun flags

Opening Remote Buckets with Go Cloud SDK

The cloud provider drivers are imported in deploy/deploy.go for S3 and GCS, while Azure is conditionally imported in deploy/deploy_azure.go to manage dependencies:

// deploy/deploy.go
_ "gocloud.dev/blob/fileblob"
_ "gocloud.dev/blob/gcsblob"
_ "gocloud.dev/blob/s3blob"

// deploy/deploy_azure.go
_ "gocloud.dev/blob/azureblob"

The bucket connection is established lazily via (*Deployer).openBucket:

func (d *Deployer) openBucket(ctx context.Context) (*blob.Bucket, error) {
    if d.bucket != nil { return d.bucket, nil }
    d.logger.Printf("Deploying to target %q (%s)\n", d.target.Name, d.target.URL)
    return blob.OpenBucket(ctx, d.target.URL)
}

blob.OpenBucket parses the URL scheme, selects the appropriate driver (s3blob, gcsblob, or azureblob), and returns a uniform interface implementing NewWriter, NewReader, Delete, List, and Attributes.

Syncing Local and Remote Files

The deployment engine performs a bidirectional walk to determine the exact set of changes required, avoiding unnecessary uploads through MD5-based delta detection.

Walking the Local Build Directory

(*Deployer).walkLocal traverses the local publishDir in parallel using Hugo's para package. For each file, it:

• Applies include/exclude globs defined in the target configuration (Target.IncludeGlob, Target.ExcludeGlob)
• Matches the first applicable Matcher for gzip compression and cache-control headers
• Constructs a localFile object that optionally gzips content and computes an MD5 hash for comparison

// Pseudo-code representation of the local file processing
localFile := newLocalFile(path, content)
localFile.MD5 = calculateMD5(content)
localFile.ContentType = determineMediaType(path)

Enumerating Remote Objects

(*Deployer).walkRemote lists all objects in the remote bucket using bucket.List. It applies the same include/exclude filters and extracts MD5 hashes either from object metadata (using the metaMD5Hash key) or by reading the object content. This ensures accurate diffing without requiring additional HTTP round-trips.

Diffing and Change Detection

(*Deployer).findDiffs compares the local and remote file maps:

• Upload triggers when a file is missing remotely, the size differs, the MD5 hash mismatches, or the force flag is enabled
• Delete marks remote objects for removal when they no longer exist in the local build

The result is two slices: uploads []*fileToUpload and deletes []string, which feed into the execution pipeline.

Executing the Deployment

Once the change set is calculated, Hugo executes uploads and deletes with configurable concurrency and optional upload ordering for dependency management.

Upload Ordering and Prioritization

The deployment.order configuration accepts a list of regular expressions that control upload sequencing. Files matching earlier patterns upload before later ones, ensuring that HTML files deploy before assets, or that service workers update last:

[deployment]
  order = ["\\.html$", "\\.css$", "\\.js$"]

applyOrdering groups uploads so that earlier groups complete entirely before subsequent groups begin, critical for progressive web apps and CDN cache consistency.

Concurrent Uploads and Deletes

Uploads and deletes execute in parallel using a semaphore channel where nParallel defaults to the workers configuration value (default: 10):

sem := make(chan struct{}, nParallel)

Each upload calls doSingleUpload, which sets object metadata including Cache-Control, Content-Encoding, and persists the MD5 hash for future delta comparisons:

func (d *Deployer) doSingleUpload(ctx context.Context, bucket *blob.Bucket, upload *fileToUpload) error {
    opts := &blob.WriterOptions{
        CacheControl:    upload.Local.CacheControl(),
        ContentEncoding: upload.Local.ContentEncoding(),
        ContentType:     upload.Local.ContentType(),
        Metadata:        map[string]string{metaMD5Hash: hex.EncodeToString(upload.Local.MD5())},
    }
    w, err := bucket.NewWriter(ctx, upload.Local.SlashPath, opts)
    // ... copy data and close writer
}

Deletes execute as simple bucket.Delete(ctx, key) operations.

CDN Invalidation

After successful deployment, if deployment.invalidateCDN is true and the target specifies cloudFrontDistributionID (AWS) or googleCloudCDNOrigin (GCP), Hugo calls provider-specific invalidation helpers to purge edge caches:

• CloudFront: InvalidateCloudFront creates an invalidation batch for the distribution ID
• Google Cloud CDN: InvalidateGoogleCloudCDN purges the specified origin

Safety Features and Dry Runs

The deployment command includes safeguards for production use. When deployment.confirm is true, Hugo prompts for confirmation before executing changes. The --dryRun flag simulates the deployment without writing to cloud storage, logging intended actions:

hugo deploy --dryRun

Output example:

Found 124 local files.
Found 119 remote files.
Identified 10 file(s) to upload, totaling 5.2 MiB, and 3 file(s) to delete.
[DRY RUN] Would upload: assets/main.css
[DRY RUN] Would delete: old/unused.png

Summary

• Hugo's deploy command uses gocloud.dev/blob to abstract AWS S3, GCS, and Azure Blob Storage through URL schemes (s3://, gs://, azblob://).
• Configuration resides in the deployment section of config.toml, supporting multiple named targets with provider-specific options like cloudFrontDistributionID.
• Delta detection relies on MD5 hashes stored in object metadata (metaMD5Hash) to skip unchanged files, with the force flag available to override.
• Execution walks local files and remote buckets in parallel, diffs the results, and performs concurrent uploads/deletes controlled by the workers setting (default 10).
• Advanced features include upload ordering via regex patterns, automatic gzip compression for matched files, and post-deploy CDN invalidation for CloudFront and Google Cloud CDN.

Frequently Asked Questions

What URL format does Hugo use for S3 deployment?

Hugo uses the Go Cloud blob URL format s3://<bucket>?region=<region> for Amazon S3, as parsed by blob.OpenBucket in deploy/deploy.go. The region parameter is typically required unless implicit in your AWS credential configuration. For example: s3://my-bucket?region=us-west-2.

How does Hugo determine which files to upload?

Hugo compares MD5 hashes between local files and remote objects stored in the metamd5hash metadata key. In deploy/deploy.go, the findDiffs function identifies uploads when hashes mismatch, sizes differ, or files are missing remotely. This delta detection prevents re-uploading unchanged assets.

Can I deploy to multiple cloud providers simultaneously?

No, the hugo deploy command processes one target per invocation. However, you can define multiple targets in your configuration and select them individually using the --target flag. For example: hugo deploy --target prod-s3 followed by hugo deploy --target prod-gcs.

Does Hugo support CDN cache invalidation?

Yes, Hugo supports automatic CDN invalidation for AWS CloudFront and Google Cloud CDN. When deployment.invalidateCDN is true and you specify cloudFrontDistributionID for S3 targets or googleCloudCDNOrigin for GCS targets, Hugo calls the respective invalidation APIs after successful deployment, purging edge caches to serve updated content immediately.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}