How Jenkins Cloud Provisioning Plugins Work with Elastic Agents: A Deep Dive into the Source Code

Jenkins cloud provisioning plugins create elastic agents on-demand by implementing the hudson.slaves.Cloud abstract class, which the NodeProvisioner periodically invokes when load statistics indicate excess workload exceeds configurable margins.

In the jenkinsci/jenkins repository, elastic agents are ephemeral build nodes that materialize dynamically to handle queue spikes and dissolve when idle. Understanding how Jenkins cloud provisioning plugins work with elastic agents requires examining the coordination between the Cloud abstraction, the NodeProvisioner engine, and the load-driven strategies that evaluate provisioning necessity.

The Core Architecture of Jenkins Cloud Provisioning

The provisioning system centers on three primary components that coordinate between the build queue and infrastructure APIs.

The Cloud Abstract Class

At core/src/main/java/hudson/slaves/Cloud.java, the abstract Cloud class defines the contract that every provisioning plugin must fulfill. Plugin authors extend this class to integrate with specific infrastructures like AWS, Azure, or Kubernetes.

The class mandates two critical methods:

• canProvision(CloudState state): Returns true if the cloud can create nodes for the requested label expression.
• provision(CloudState state, int excessWorkload): Returns a collection of NodeProvisioner.PlannedNode objects representing asynchronous node launches.

The NodeProvisioner Engine

Located at core/src/main/java/hudson/slaves/NodeProvisioner.java, the NodeProvisioner class functions as the central orchestrator. It periodically analyzes load statistics per label and delegates creation requests to registered clouds. The update() method triggers either on a schedule via NodeProvisionerInvoker (a PeriodicWork implementation) or through explicit hints when new builds enter the queue.

How Load-Driven Provisioning Decisions Work

Jenkins evaluates provisioning needs through statistical analysis rather than simple queue depth checks.

Excess Workload Calculation

The standard strategy (StandardStrategyImpl.apply()) computes excess workload using the formula:

float excessWorkload = qlen - plannedCapacity - connectingCapacity;
if (excessWorkload > 1 - margin) {
    // Trigger provisioning
}

This calculation subtracts already planned capacity and connecting executors from the current queue length. When the result exceeds the configurable margin (typically near zero), Jenkins determines that additional elastic agents are necessary.

The Provisioning Invocation Flow

When excess workload triggers:

1. Jenkins iterates through Jenkins.get().clouds
2. For each Cloud implementation, it invokes canProvision(cloudState)
3. If affirmative, it calls provision(cloudState, workloadToProvision)
4. The method returns Collection<PlannedNode> representing pending agent launches

Implementing the Cloud Provision Contract

Plugin developers must implement the Cloud contract to enable elastic agent creation.

The canProvision Method

This method filters provisioning requests by label. For example, a cloud targeting AWS resources might implement:

@Override
public boolean canProvision(CloudState state) {
    return state.getLabel() != null && state.getLabel().matches("aws");
}

The provision Method

The provision(CloudState state, int excessWorkload) method must return a Collection<PlannedNode>, where each PlannedNode contains:

• A display name string
• A Future<Node> that asynchronously yields a fully configured Node
• The number of executors the node will provide

@Override
public Collection<NodeProvisioner.PlannedNode> provision(CloudState state, int excessWorkload) {
    Future<Node> agentFuture = Executors.newSingleThreadExecutor().submit(() -> {
        Slave agent = new DumbSlave(
            "elastic-" + System.nanoTime(),
            "Elastic agent",
            "/home/jenkins",
            "1",
            Node.Mode.NORMAL,
            "",
            new JNLPLauncher(),
            new RetentionStrategy.Always(),
            Collections.emptyList()
        );
        agent.setLabelString(state.getLabel() != null ? state.getLabel().getName() : "");
        return agent;
    });
    
    return List.of(new NodeProvisioner.PlannedNode("elastic-agent", agentFuture, 1));
}

As the Future completes, Jenkins automatically adds the realized Node to the system.

Lifecycle Monitoring with CloudProvisioningListener

The core/src/main/java/hudson/slaves/CloudProvisioningListener.java file defines an extension point allowing plugins to react to provisioning events.

Key Callback Methods

Implementations receive notifications at critical transition points:

• onStarted(Cloud, Label, Collection<PlannedNode>): Fires when provision() is invoked but before infrastructure operations begin.
• onComplete(Cloud, Label, Collection<PlannedNode>): Fires when the Future<Node> completes successfully.
• onCommit(Cloud, Label, Collection<PlannedNode>): Fires when the node is officially added to Jenkins via Jenkins.addNode.
• onFailure/onRollback(...): Fires when provisioning fails or must be reversed.

@Extension
public class AuditListener extends CloudProvisioningListener {
    @Override
    public void onStarted(Cloud cloud, Label label, Collection<NodeProvisioner.PlannedNode> nodes) {
        Logger.getLogger(getClass().getName()).info(
            "Provisioning " + nodes.size() + " nodes from " + cloud.getDisplayName()
        );
    }
}

These callbacks enable auxiliary tasks like VM tagging, DNS registration, or credential injection.

Controlling Provisioning via QueueTaskDispatcher

Plugins can suppress automatic provisioning for specific builds by implementing QueueTaskDispatcher at core/src/main/java/hudson/model/queue/QueueTaskDispatcher.java.

When canRun(Queue.Item) returns a non-null CauseOfBlockage, Jenkins places the item in a blocked state. This signals NodeProvisioner to ignore the workload, preventing elastic agent creation for that task.

@Extension
public class ProvisioningGate extends QueueTaskDispatcher {
    @Override
    public CauseOfBlockage canRun(Queue.Item item) {
        if (item.task.getName().contains("restricted")) {
            return new CauseOfBlockage() {
                public String getShortDescription() { 
                    return "Elastic agents disabled for restricted jobs"; 
                }
            };
        }
        return null;
    }
}

This mechanism provides fine-grained control over infrastructure costs by preventing unnecessary agent spin-up for specific job types.

Summary

• Jenkins cloud provisioning plugins implement hudson.slaves.Cloud to provide elastic agents on-demand.
• The NodeProvisioner at core/src/main/java/hudson/slaves/NodeProvisioner.java calculates excess workload and triggers provision() calls when capacity is insufficient.
• Plugins must return PlannedNode objects containing a Future<Node> that resolves asynchronously.
• CloudProvisioningListener extensions hook into provisioning lifecycle events for monitoring and resource management.
• QueueTaskDispatcher implementations can block provisioning for specific queue items to control infrastructure expenditures.

Frequently Asked Questions

What is the difference between a static agent and an elastic agent in Jenkins?

Static agents are permanently registered nodes defined in the Jenkins configuration, while elastic agents are ephemeral nodes created dynamically by cloud provisioning plugins when the build queue exceeds current capacity. Elastic agents typically terminate automatically after a period of idleness, whereas static agents remain online until manually disabled.

How does NodeProvisioner determine exactly when to provision new elastic agents?

The NodeProvisioner.update() method analyzes load statistics per label and applies the StandardStrategyImpl formula: excessWorkload = queueLength - plannedCapacity - connectingCapacity. When this value exceeds 1 - margin (where margin is configurable), Jenkins invokes Cloud.provision() to create additional elastic agents. This ensures provisioning occurs only when queued builds genuinely exceed available and incoming capacity.

Can a plugin prevent Jenkins from provisioning elastic agents for specific jobs?

Yes. Implementing the QueueTaskDispatcher extension point allows plugins to return a CauseOfBlockage from the canRun(Queue.Item) method. When Jenkins detects a blocked item, the NodeProvisioner excludes that workload from excess calculations, effectively suppressing elastic agent creation for that specific build while allowing other jobs to trigger provisioning normally.

What must a cloud plugin return from the provision() method?

The provision(CloudState state, int excessWorkload) method must return a Collection<NodeProvisioner.PlannedNode>. Each PlannedNode must contain three elements: a display name string, a Future<Node> that asynchronously resolves to a fully configured agent, and an integer representing the number of executors that agent will provide. Jenkins monitors these futures and adds completed nodes to the system automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}