How to Use Jenkins for CI/CD: A Complete Guide to Pipeline Automation

Jenkins orchestrates CI/CD through code-defined pipelines executed by a CPS-based workflow engine that manages builds as Job objects and runs as lightweight, resumable threads.

Jenkins is an open-source automation server written in Java that enables teams to implement continuous integration and continuous delivery (CI/CD) pipelines. According to the jenkinsci/jenkins source code, the platform's architecture centers on a flexible job model and a robust execution engine that transforms version-controlled pipeline definitions into automated software delivery workflows.

Understanding the Jenkins Job Model

At the heart of Jenkins' CI/CD capability lies the abstract Job class, which serves as the foundation for all buildable projects. In core/src/main/java/hudson/model/Job.java, this generic class defines the common behavior for build parameters, run history, and lifecycle management that applies equally to Freestyle projects, Maven jobs, and modern Pipeline jobs.

When a Jenkins pipeline triggers, the system creates a Run object linked to the Job. This object records execution details, artifacts, and results while maintaining a complete history of every CI/CD execution.

Pipeline Execution Architecture

The CPS Engine and Process Management

Jenkins pipelines run on a specialized Continuation Passing Style (CPS) engine that converts Groovy DSL code into executable steps. As implemented in core/src/main/java/hudson/util/ProcessTree.java, the platform manages these executions through lightweight threads (CpsThread) that execute within a ProcessTree hierarchy.

This architecture enables critical CI/CD capabilities:

• Pause and resume: Pipelines can wait for human approval or external systems without consuming executor resources
• Safe termination: The CPS thread model allows Jenkins to stop specific pipeline steps without killing underlying OS processes
• Durability: Pipeline state persists across Jenkins restarts

Backward Compatibility and Configuration

The core maintains compatibility between traditional jobs and modern pipelines through adapter classes. In core/src/main/java/jenkins/model/BuildDiscarderProperty.java, the source code maps pipeline-specific properties (like org.jenkinsci.plugins.workflow.job.properties.BuildDiscarderProperty) to core equivalents, ensuring that the workflow engine persists configuration using the same XStream serialization mechanisms as the rest of Jenkins.

Defining Your CI/CD Pipeline as Code

Modern Jenkins usage centers on the Jenkinsfile—a text file containing pipeline definition written in Groovy DSL that lives in your source control repository. The jenkinsci/jenkins repository itself includes a minimal example in Jenkinsfile demonstrating how teams express build-test-deploy workflows as code.

Below is a complete Declarative Pipeline example that implements a typical Java CI/CD workflow:

pipeline {
    agent any                         // Run on any available executor
    options {
        timeout(time: 30, unit: 'MINUTES')
        buildDiscarder(logRotator(numToKeepStr: '10'))
    }
    environment {
        DOCKER_REGISTRY = 'registry.example.com'
        IMAGE_NAME      = "my-app:${env.BUILD_NUMBER}"
    }
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package -DskipTests'
            }
        }
        stage('Test') {
            steps {
                sh 'mvn test'
                junit '**/target/surefire-reports/*.xml'   // Publish JUnit results
            }
        }
        stage('Archive') {
            steps {
                archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
            }
        }
        stage('Docker Build & Push') {
            steps {
                script {
                    docker.withRegistry("https://${DOCKER_REGISTRY}", 'docker-credentials-id') {
                        docker.build("${IMAGE_NAME}", '.').push()
                    }
                }
            }
        }
    }
    post {
        always {
            cleanWs()                     // Clean workspace after each run
        }
        success {
            echo '✅ Build succeeded!'
        }
        failure {
            mail to: 'team@example.com',
                 subject: "Build ${env.JOB_NAME} #${env.BUILD_NUMBER} failed",
                 body: "Check console output at ${env.BUILD_URL}"
        }
    }
}

This example demonstrates key CI/CD practices: source control checkout, Maven build and test execution, JUnit result publication, artifact archival, Docker image creation with registry push, and post-build notifications.

Triggering and Scaling Jenkins CI/CD

Jenkins integrates with source control management (SCM) systems through webhooks or polling mechanisms. When commits arrive, Jenkins automatically instantiates a new Run associated with the Job, checks out the Jenkinsfile at that commit, and executes the defined stages.

The platform's extensibility comes from over 2,000 plugins that load at runtime. While core/src/main/java/hudson/model/Job.java provides the fundamental job lifecycle and the CPS engine handles execution, plugins deliver specific CI/CD capabilities such as:

• SCM integrations: Git, Subversion, and GitHub Enterprise connectors
• Test reporters: Parsing and displaying JUnit, TestNG, or Cucumber results
• Deployment targets: Kubernetes, AWS, Azure, and on-premise server orchestration
• Security: Credential binding and secrets management

Summary

• Job abstraction: All Jenkins CI/CD workflows inherit from the Job class in core/src/main/java/hudson/model/Job.java, providing consistent lifecycle management across project types.
• CPS execution: The workflow engine uses Continuation Passing Style threads managed through ProcessTree.java to enable pausable, durable pipelines.
• Pipeline as code: Jenkins stores CI/CD definitions in version-controlled Jenkinsfile scripts using Groovy DSL, ensuring reproducible builds.
• Plugin extensibility: Core architecture delegates specific CI/CD tasks to plugins, allowing teams to customize integrations without modifying the job model.

Frequently Asked Questions

What is the difference between Declarative and Scripted pipelines in Jenkins?

Declarative pipelines provide a structured, opinionated syntax via the pipeline block ideal for standard CI/CD workflows, while Scripted pipelines offer unlimited Groovy flexibility through the node block for complex logic. Both execute on the same CPS engine, but Declarative syntax enforces stage-based organization and provides built-in validation.

How does Jenkins handle pipeline durability and restart capabilities?

Jenkins persists pipeline state through the CPS engine's serialization mechanism. As implemented in the workflow engine, CpsThread objects capture execution context, allowing Jenkins to resume interrupted pipelines after controller restarts. The ProcessTree.java implementation separately tracks OS processes to prevent orphaned builds during termination.

Where does Jenkins store pipeline configuration and build history?

Jenkins persists job configurations, including BuildDiscarderProperty settings from core/src/main/java/jenkins/model/BuildDiscarderProperty.java, using XStream XML serialization in the JENKINS_HOME/jobs/ directory. Build history, artifacts, and logs associate with specific Run instances linked to their parent Job, while Jenkinsfile content resides in your SCM repository.

How can I trigger Jenkins pipelines automatically on code commits?

Configure your repository webhooks to POST to JENKINS_URL/github-webhook/ (for GitHub) or equivalent endpoints for GitLab, Bitbucket, or Gerrit. The Jenkins core receives these notifications, instantiates a new Run for the associated Job, and executes the pipeline stages defined in the Jenkinsfile at that specific commit SHA.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}