How VarMQ Uses Linked Lists for Its Worker Pool: A Deep Dive into the goptics/varmq Architecture

VarMQ implements its worker pool using a generic doubly-linked list in internal/linkedlist to achieve O(1) insertion, removal, and traversal while recycling nodes via sync.Pool for minimal memory overhead.

The goptics/varmq repository provides a high-performance job queue for Go that leverages an innovative linked list-based worker pool architecture. Understanding how VarMQ uses linked lists for its worker pool reveals the design decisions behind its low-latency goroutine management and efficient memory reuse patterns.

Core Data Structures

The Generic Doubly-Linked List (linkedlist.List[T])

Located in internal/linkedlist/linkedlist.go, this structure features a sentinel root node that eliminates edge-case handling for empty lists. It provides O(1) operations for PushBack, PushNode, PopBack, Remove, and NodeSlice. A sync.RWMutex guards structural changes, enabling concurrent readers (via NodeSlice) while ensuring exclusive write access for modifications.

The Worker Node (pool.Node[T])

Defined in internal/pool/node.go, each node encapsulates a buffered channel (ch) that worker goroutines read from. It tracks a lastUsed timestamp to support idle-worker eviction, allowing the pool to identify and terminate stale goroutines efficiently.

The Pool Manager (pool.Pool[T])

Found in internal/pool/pool.go, this embeds linkedlist.List[Node[T]] and maintains a sync.Pool named Cache. This cache hands out fresh linkedlist.Node[Node[T]] objects, enabling node reuse without repeated heap allocations and reducing GC pressure during high-throughput scenarios.

The Worker Orchestrator (worker[T, JobType])

In worker.go, this component coordinates job processing by pulling idle nodes from the pool using PopBack, spawning goroutines that call node.Value.Serve(), and recycling nodes back into the linked list via PushNode when tasks complete.

Worker Node Lifecycle in the Linked List

1. Pool Initialization – When newWorker creates a pool via pool.New[JobType], it constructs a linkedlist.List[Node[JobType]] and initializes a sync.Pool that returns fresh linkedlist.Node objects containing newly created pool.Node instances.

2. Spawning a Goroutine – Upon job arrival, sendToNextChannel attempts to pull the last idle node using PopBack. If none exist, initPoolNode retrieves a node from the cache:

   node := w.pool.Cache.Get().(*linkedlist.Node[pool.Node[JobType]])
   go node.Value.Serve(func(j JobType) { … })

3. Processing a Job – The goroutine reads from node.Value.ch, executes the user-provided workerFunc, marks the job finished, and triggers freePoolNode.

4. Returning to Pool – freePoolNode evaluates pool limits and idle worker counts. If the node remains active, it returns to the linked list via w.pool.PushNode(node); otherwise, it recycles the container:

   if w.queues.Len() >= w.NumConcurrency() ||
      enabledIdleWorkersRemover ||
      w.pool.Len() < w.numMinIdleWorkers() {
       w.pool.PushNode(node)
   } else {
       node.Value.Stop()
       w.pool.Cache.Put(node)
   }

5. Idle-Worker Eviction – The background ticker goRemoveIdleWorkers periodically scans the linked list via NodeSlice() and removes nodes whose lastUsed timestamp exceeds the configured expiry. The Remove(node) method operates in O(1) because each node maintains direct prev/next pointers:

   nodes := w.pool.NodeSlice()
   for _, node := range nodes[targetIdleWorkers:] {
       if node.Value.GetLastUsed().Add(interval).Before(time.Now()) {
           w.pool.Remove(node)
           node.Value.Stop()
           w.pool.Cache.Put(node)
       }
   }

Why VarMQ Chose a Linked List Over Slices

VarMQ selected a doubly-linked list architecture for four critical performance characteristics:

• O(1) push/pop at both ends – Workers are added to the back via PushNode and removed from the back via PopBack without shifting elements. This is essential for a dynamic pool that constantly grows and shrinks based on load.

• O(1) arbitrary removal – The idle-worker reaper can delete any node it discovers using Remove(node) without traversing the entire structure. Each node already knows its neighbors, unlike slices which require O(n) traversal for deletion.

• Memory reuse – The list stores pointers to linkedlist.Node, while the underlying sync.Pool recycles the actual node structs. This separation minimizes allocations and keeps the memory footprint low during bursty workloads.

• Thread-safe traversal – linkedlist.List uses a sync.RWMutex to guard structural changes, allowing concurrent readers (e.g., NodeSlice calls during eviction) while writers hold exclusive locks for PushNode or Remove operations.

Practical Implementation Example

The following example demonstrates creating a worker and observing the linked-list pool behavior:

package main

import (
	"fmt"
	"github.com/goptics/varmq"
	"time"
)

// Simple job type
type MyJob struct{ varmq.BaseJob[int] }

func main() {
	// Create a worker that prints the payload
	w := varmq.New[int](func(j *MyJob) {
		fmt.Println("Job:", j.Payload())
	}, varmq.WithConcurrency(4), varmq.WithIdleWorkerExpiry(30*time.Second))

	// Start the worker
	_ = w.Start()

	// Enqueue a few jobs
	for i := 0; i < 10; i++ {
		w.Enqueue(&MyJob{Payload: i})
	}

	// Wait for completion
	w.WaitUntilFinished()
}

Behind the scenes, w.pool holds a linkedlist.List[pool.Node[*MyJob]]. Each new goroutine is wrapped in a linkedlist.Node that gets PushNode/PopBack operations. When the queue drains, idle nodes sit in the linked list until either the pool size limit is exceeded or the idle-worker ticker evicts them.

Summary

• VarMQ's worker pool relies on a generic doubly-linked list in internal/linkedlist/linkedlist.go to manage idle workers with O(1) operations.
• Each worker exists as a node in the linked list, enabling constant-time acquisition via PopBack and release via PushNode.
• A sync.Pool integrated into internal/pool/pool.go recycles node containers, minimizing heap allocations during high concurrency.
• An idle-worker eviction routine traverses the list using NodeSlice() and safely removes stale nodes via Remove(), preserving intended concurrency while preventing resource leaks.

Frequently Asked Questions

What makes linked lists better than slices for worker pools in VarMQ?

Linked lists provide O(1) insertion and removal at both ends, whereas slices require shifting elements. In worker.go, PopBack and PushNode operate in constant time regardless of pool size, while arbitrary deletion via Remove avoids the O(n) traversal cost inherent in slices when removing idle workers from the middle of the collection.

How does VarMQ prevent memory leaks in its worker pool?

The pool combines a sync.Pool for recycling linkedlist.Node objects with a background eviction routine in goRemoveIdleWorkers. Nodes exceeding the idle timeout are removed from the list via Remove() and returned to the cache, preventing unbounded growth while reusing memory for new workers instead of allocating fresh structs.

Is the VarMQ linked list implementation thread-safe?

Yes. The linkedlist.List in internal/linkedlist/linkedlist.go guards all structural operations with a sync.RWMutex, allowing concurrent read access via NodeSlice while ensuring exclusive write access for PushNode, PopBack, and Remove operations. This design supports concurrent job processing without race conditions.

What happens when all idle workers are busy in VarMQ?

When sendToNextChannel finds no idle nodes via PopBack, it calls initPoolNode to retrieve a fresh node from w.pool.Cache, initializes it with a new goroutine, and begins processing immediately. This ensures the pool scales dynamically up to the configured concurrency limits, creating new workers only when the linked list of idle nodes is exhausted.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}