How to Configure VarMQ's Minimum Idle Worker Ratio

Use WithMinIdleWorkerRatio(percentage) to set a per-worker idle ratio (1–100%) or call DefaultMinIdleWorkerRatio() to establish a global default, ensuring VarMQ maintains a calculated fraction of goroutines in an idle state for immediate job pickup.

VarMQ is a high-performance job queue library for Go found in the goptics/varmq repository. Configuring the minimum idle worker ratio lets you control how many worker goroutines remain idle relative to your total concurrency, balancing resource consumption against cold-start latency when new jobs arrive.

Understanding the Minimum Idle Worker Ratio

The minimum idle worker ratio determines what percentage of your total concurrency must remain idle (not actively processing jobs). These idle workers stay warm in the pool, ready to grab new jobs immediately without the overhead of spawning new goroutines.

According to the source code in config.go, the ratio is stored in the configs struct as minIdleWorkerRatio (a uint8 representing a percentage). If you do not explicitly configure this value, the system defaults to maintaining at least one idle worker regardless of concurrency size, achieved by internally clamping a zero value to one inside numMinIdleWorkers().

Configuration Methods

You can configure the idle worker ratio either per-worker or globally across your application.

Per-Worker Configuration

Pass WithMinIdleWorkerRatio() as an option when instantiating a new worker. This accepts a uint8 between 0 and 100, though the value is automatically clamped to the range 1–100 by the internal clampPercentage() function in config.go.

import "github.com/goptics/varmq"

// Maintain 20% of concurrency as idle workers
worker := varmq.NewWorker[int](
    varmq.WithConcurrency(10),            // Total workers = 10
    varmq.WithMinIdleWorkerRatio(20),     // Keep at least 2 idle (10*20% = 2)
)

In this example, the pool will ensure at least two goroutines remain idle at all times. If the value provided were 0, clampPercentage() would force it to 1, guaranteeing a minimum of one idle worker.

Global Default Configuration

Set a global default using DefaultMinIdleWorkerRatio(). This affects all workers created after the call unless they specify their own WithMinIdleWorkerRatio() option.

// Set global default to 30%
varmq.DefaultMinIdleWorkerRatio(30)

// This worker inherits the 30% ratio (5*30% ≈ 1.5 → rounded to 2 idle workers)
w1 := varmq.NewWorker[string](varmq.WithConcurrency(5))

The global setter modifies the package-level default configuration that loadConfigs() uses when merging options during worker initialization.

Implementation Details

Value Clamping (1–100%)

VarMQ enforces valid percentages through the clampPercentage() function in config.go (lines 45–55). This helper ensures:

• Values below 1 are raised to 1 (so 0% becomes 1% effectively)
• Values above 100 are reduced to 100%

This guarantees that numMinIdleWorkers() always calculates a meaningful minimum of at least one idle worker.

Idle Worker Calculation

The actual number of idle workers to maintain is computed by numMinIdleWorkers() in worker.go (lines 58–64). The implementation uses integer arithmetic:

// Pseudocode based on worker.go implementation
targetIdle := int(max((concurrency * percentage) / 100, 1))

For a concurrency of 10 and a ratio of 25%, this yields max(2.5, 1) → 2 idle workers. The function ensures the result is never less than 1, even at low concurrency levels.

Enforcement and Expiry

During normal operation, the worker pool never shrinks below the target calculated by numMinIdleWorkers(). However, if you configure WithIdleWorkerExpiryDuration(), the background goRemoveIdleWorkers() routine (found in worker.go, lines 66–86) periodically removes idle workers that exceed the target count and have been idle longer than the specified duration.

// Allow excess idle workers to be removed after 30 seconds of inactivity
w := varmq.NewWorker[int](
    varmq.WithConcurrency(8),
    varmq.WithMinIdleWorkerRatio(25),
    varmq.WithIdleWorkerExpiryDuration(30 * time.Second),
)

Without an expiry duration, the pool strictly maintains the minimum idle count indefinitely.

Practical Examples

Runtime Inspection

You can inspect the calculated target at runtime by accessing the configuration field and calculation method:

ratio := w.Configs.minIdleWorkerRatio    // Configured percentage, e.g., 20
target := w.numMinIdleWorkers()          // Calculated count, e.g., 2

fmt.Printf("Target idle workers: %d (ratio %d%%)\n", target, ratio)

Note that Configs (lowercase in the struct definition but accessible in the example context) holds the clamped value, and numMinIdleWorkers() performs the live calculation based on current concurrency.

Combining with Expiry

For production workloads with variable traffic, combining the minimum ratio with idle expiry prevents resource waste during quiet periods while maintaining responsiveness during spikes:

// High concurrency with 10% minimum idle, expiring after 1 minute
batchWorker := varmq.NewWorker[Task](
    varmq.WithConcurrency(100),
    varmq.WithMinIdleWorkerRatio(10),          // Keep ~10 idle workers
    varmq.WithIdleWorkerExpiryDuration(time.Minute),
)

Summary

• Configure the minimum idle worker ratio using WithMinIdleWorkerRatio(percentage) per worker or DefaultMinIdleWorkerRatio() globally.
• Values are automatically clamped to 1–100% by clampPercentage() in config.go, ensuring at least one idle worker always exists.
• The actual idle count is calculated by numMinIdleWorkers() in worker.go as max((concurrency*percentage)/100, 1).
• Pair the ratio with WithIdleWorkerExpiryDuration() to let the pool shrink during low-traffic periods while maintaining the minimum threshold.

Frequently Asked Questions

What is the default minimum idle worker ratio in VarMQ?

If you do not call WithMinIdleWorkerRatio(), the internal default is 0%, but the system treats this as 1% due to clamping in numMinIdleWorkers(). This guarantees that at least one worker goroutine remains idle in the pool at all times, preventing cold-start latency for the first job.

How does VarMQ calculate the exact number of idle workers from the percentage?

The calculation happens in worker.go inside numMinIdleWorkers(), which computes int(max((concurrency*percentage)/100, 1)). For example, with 50 concurrency and a 10% ratio, the target is 5 idle workers. The function always returns at least 1, even if your math results in a fraction below 1.

What happens when I enable idle worker expiry alongside the minimum ratio?

The goRemoveIdleWorkers() background ticker (in worker.go) removes idle workers only if they exceed the count returned by numMinIdleWorkers() and have been idle longer than idleWorkerExpiryDuration. Workers at or below the calculated minimum are preserved regardless of how long they have been idle, ensuring you always maintain your configured capacity for immediate job handling.

Is there a maximum value I can set for the minimum idle worker ratio?

Yes. The clampPercentage() function in config.go enforces a hard ceiling of 100%. If you pass a value greater than 100 to WithMinIdleWorkerRatio(), it is automatically reduced to 100%, meaning all workers in the concurrency pool would theoretically remain idle (though in practice, as jobs arrive, active workers will process them while the pool attempts to spawn replacements to maintain the idle count).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}