How to Limit Concurrent Image Processing in next-export-optimize-images with `processingConcurrency`

Set the processingConcurrency option in your export-images.config.js file to control how many images are processed simultaneously, with a default value of 10 concurrent tasks.

The next-export-optimize-images library parallelizes image optimization and external asset downloading to accelerate static site builds. You can regulate CPU and I/O pressure by configuring the processingConcurrency parameter, which governs the maximum number of simultaneous operations across both local image processing and remote image fetching.

How processingConcurrency Works in the Source Code

The concurrency limit is enforced in two critical CLI stages that process promise batches in chunks. If you omit the setting, the system falls back to 10 parallel jobs.

Main Image Optimization Loop (src/cli/index.ts)

The primary optimization pipeline limits simultaneous calls to getOptimizeResult, which handles Sharp processing and cache management. In src/cli/index.ts, the implementation batches promises according to your configured limit:

const concurrency = config.processingConcurrency ?? 10
for (let i = 0; i < promises.length; i += concurrency) {
  const chunk = promises.slice(i, i + concurrency)
  await Promise.all(chunk)
}

This ensures that only processingConcurrency images are processed at once, preventing memory exhaustion during large builds.

External Image Downloads (src/cli/external-images/index.ts)

When fetching remote assets, the same concurrency cap applies. The downloader in src/cli/external-images/index.ts uses identical batching logic to limit concurrent fetch streams:

const concurrency = processingConcurrency ?? 10
for (let i = 0; i < promises.length; i += concurrency) {
  const chunk = promises.slice(i, i + concurrency)
  await Promise.all(chunk)
}

Configuration Schema (src/utils/getConfig.ts)

The configuration validator and loader in src/utils/getConfig.ts defines the processingConcurrency property type and supplies the default value of 10 when the user does not specify an override.

Configuring processingConcurrency in Your Project

Create or modify export-images.config.js at your project root to inject the concurrency limit into the runtime configuration.

Basic Configuration

To reduce load on CI runners or low-specification machines, set a lower integer value:

// export-images.config.js
module.exports = {
  quality: 80,
  processingConcurrency: 5, // limits both processing and downloads to 5 concurrent jobs
}

Using the Default Behavior

If you do not define processingConcurrency, the library automatically uses the default of 10 parallel jobs:

// export-images.config.js
module.exports = {
  quality: 80,
  // processingConcurrency defaults to 10
}

Runtime Verification

After configuration, run the export command to observe the throttled execution:

$ npx next-export-optimize-images

The CLI processes images in batches corresponding to your processingConcurrency setting, reducing peak memory usage.

Summary

• processingConcurrency controls parallelization in both the main optimization loop and external image downloads.
• The default value is 10 concurrent tasks, defined in src/utils/getConfig.ts.
• Implementation resides in src/cli/index.ts (local images) and src/cli/external-images/index.ts (remote assets).
• Lower values decrease CPU and I/O pressure for resource-constrained environments; higher values may accelerate builds on powerful hardware.
• Configure the option in export-images.config.js to override the default behavior.

Frequently Asked Questions

What is the default processingConcurrency value?

The default value is 10 concurrent tasks. This is hard-coded in src/utils/getConfig.ts and applied in src/cli/index.ts and src/cli/external-images/index.ts when no user configuration is provided.

Does processingConcurrency affect external image downloads?

Yes. The same concurrency limit applies to both local image optimization and external image fetching. The external image downloader in src/cli/external-images/index.ts reads the processingConcurrency value to batch fetch operations, ensuring remote assets download at the same throttled rate as local processing.

How do I know if I should lower the concurrency limit?

Reduce processingConcurrency if you encounter memory exhaustion errors, CPU throttling, or network timeouts during builds. This is particularly important on CI/CD runners with restricted resources or when processing high-resolution image batches that consume significant Sharp processing memory.

Can I set different concurrency limits for processing versus downloading?

Currently, the library exposes only a single processingConcurrency setting that governs both stages. Both src/cli/index.ts and src/cli/external-images/index.ts reference the same configuration value. To implement different limits, you would need to modify the source code or fork the repository according to the implementation in the dc7290/next-export-optimize-images codebase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}