DevOps 7 min read

Jenkins Pipeline Tutorial: Complete CI/CD Guide

Master Jenkins pipelines for CI/CD automation. Learn declarative and scripted pipelines, stages, parallel execution, and deployment strategies.

MR

Moshiour Rahman

#jenkins #ci-cd #devops #automation #pipeline #ai

Advertisement

What is Jenkins Pipeline?

Jenkins Pipeline is a suite of plugins that enables implementing continuous delivery pipelines as code. It provides an extensible set of tools for modeling delivery pipelines.

Pipeline vs Freestyle

FeatureFreestylePipeline
DefinitionUI-basedCode as Jenkinsfile
Version controlLimitedFull Git integration
ComplexitySimple jobsComplex workflows
ReusabilityCopy/pasteShared libraries
VisualizationBasicStage view

Setup

Install Jenkins with Docker

# docker-compose.yml
version: '3.8'

services:
  jenkins:
    image: jenkins/jenkins:lts
    privileged: true
    user: root
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - jenkins_home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - JAVA_OPTS=-Djenkins.install.runSetupWizard=false

volumes:
  jenkins_home:

Required Plugins

  • Pipeline
  • Git
  • Docker Pipeline
  • Credentials
  • Blue Ocean (optional, better UI)

Declarative Pipeline

Basic Structure

// Jenkinsfile
pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building...'
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            steps {
                echo 'Testing...'
                sh 'mvn test'
            }
        }

        stage('Deploy') {
            steps {
                echo 'Deploying...'
                sh 'mvn deploy'
            }
        }
    }
}

Agent Options

pipeline {
    // Run on any available agent
    agent any

    // Run on agent with specific label
    agent { label 'linux' }

    // Run in Docker container
    agent {
        docker {
            image 'maven:3.8-openjdk-17'
            args '-v /root/.m2:/root/.m2'
        }
    }

    // No global agent (define per stage)
    agent none

    stages {
        stage('Build') {
            agent {
                docker { image 'node:18' }
            }
            steps {
                sh 'npm install'
            }
        }
    }
}

Environment Variables

pipeline {
    agent any

    environment {
        APP_NAME = 'my-application'
        APP_VERSION = '1.0.0'
        DOCKER_REGISTRY = 'docker.io/myuser'
        DEPLOY_ENV = "${params.ENVIRONMENT ?: 'dev'}"
    }

    stages {
        stage('Build') {
            environment {
                // Stage-specific variables
                BUILD_ID = "${env.BUILD_NUMBER}-${env.GIT_COMMIT?.take(7)}"
            }
            steps {
                echo "Building ${APP_NAME} version ${APP_VERSION}"
                echo "Build ID: ${BUILD_ID}"
                sh 'printenv | sort'
            }
        }
    }
}

Parameters

pipeline {
    agent any

    parameters {
        string(name: 'BRANCH', defaultValue: 'main', description: 'Branch to build')
        choice(name: 'ENVIRONMENT', choices: ['dev', 'staging', 'prod'], description: 'Deployment environment')
        booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests')
        password(name: 'API_KEY', description: 'API key for deployment')
    }

    stages {
        stage('Build') {
            steps {
                echo "Building branch: ${params.BRANCH}"
                echo "Target environment: ${params.ENVIRONMENT}"
            }
        }

        stage('Test') {
            when {
                expression { params.RUN_TESTS == true }
            }
            steps {
                sh 'mvn test'
            }
        }
    }
}

Credentials

pipeline {
    agent any

    environment {
        // Single credential
        DOCKER_CREDENTIALS = credentials('docker-hub-credentials')

        // Username/password separately
        DOCKER_USER = credentials('docker-username')
        DOCKER_PASS = credentials('docker-password')
    }

    stages {
        stage('Docker Login') {
            steps {
                // Using withCredentials block
                withCredentials([usernamePassword(
                    credentialsId: 'docker-hub-credentials',
                    usernameVariable: 'DOCKER_USER',
                    passwordVariable: 'DOCKER_PASS'
                )]) {
                    sh 'docker login -u $DOCKER_USER -p $DOCKER_PASS'
                }
            }
        }

        stage('Deploy with SSH') {
            steps {
                withCredentials([sshUserPrivateKey(
                    credentialsId: 'ssh-key',
                    keyFileVariable: 'SSH_KEY',
                    usernameVariable: 'SSH_USER'
                )]) {
                    sh '''
                        ssh -i $SSH_KEY $SSH_USER@server.example.com 'deploy.sh'
                    '''
                }
            }
        }
    }
}

Conditional Execution

When Directive

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }

        // Only on main branch
        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                sh './deploy-staging.sh'
            }
        }

        // Based on environment
        stage('Deploy to Production') {
            when {
                allOf {
                    branch 'main'
                    environment name: 'DEPLOY_TO_PROD', value: 'true'
                }
            }
            steps {
                sh './deploy-prod.sh'
            }
        }

        // Based on changeset
        stage('Build Frontend') {
            when {
                changeset 'frontend/**'
            }
            steps {
                sh 'cd frontend && npm run build'
            }
        }

        // Expression
        stage('Notify') {
            when {
                expression { currentBuild.result == 'SUCCESS' }
            }
            steps {
                echo 'Build successful!'
            }
        }
    }
}

Parallel Execution

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'mvn test -Dtest=*UnitTest'
                    }
                }

                stage('Integration Tests') {
                    steps {
                        sh 'mvn test -Dtest=*IntegrationTest'
                    }
                }

                stage('E2E Tests') {
                    agent {
                        docker { image 'cypress/included:latest' }
                    }
                    steps {
                        sh 'npm run test:e2e'
                    }
                }
            }
        }

        stage('Deploy') {
            steps {
                sh './deploy.sh'
            }
        }
    }
}

Post Actions

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
    }

    post {
        always {
            // Always execute
            echo 'Pipeline completed'
            cleanWs()  // Clean workspace
        }

        success {
            echo 'Build succeeded!'
            // Archive artifacts
            archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            // Publish test results
            junit 'target/surefire-reports/*.xml'
        }

        failure {
            echo 'Build failed!'
            // Send notification
            mail to: 'team@example.com',
                 subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
                 body: "Build failed: ${env.BUILD_URL}"
        }

        unstable {
            echo 'Build unstable (test failures)'
        }

        changed {
            echo 'Build status changed from previous build'
        }
    }
}

Complete CI/CD Pipeline

Java/Spring Boot Pipeline

pipeline {
    agent any

    tools {
        maven 'Maven-3.8'
        jdk 'JDK-17'
    }

    environment {
        APP_NAME = 'spring-boot-app'
        DOCKER_REGISTRY = 'docker.io/myuser'
        DOCKER_IMAGE = "${DOCKER_REGISTRY}/${APP_NAME}"
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
                script {
                    env.GIT_COMMIT_SHORT = sh(
                        script: 'git rev-parse --short HEAD',
                        returnStdout: true
                    ).trim()
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'mvn test'
                    }
                    post {
                        always {
                            junit 'target/surefire-reports/*.xml'
                        }
                    }
                }

                stage('Code Quality') {
                    steps {
                        withSonarQubeEnv('SonarQube') {
                            sh 'mvn sonar:sonar'
                        }
                    }
                }
            }
        }

        stage('Package') {
            steps {
                sh 'mvn package -DskipTests'
                archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    docker.build("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}")
                }
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    docker.withRegistry('https://docker.io', 'docker-hub-credentials') {
                        docker.image("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}").push()
                        docker.image("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}").push('latest')
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                sh """
                    kubectl set image deployment/${APP_NAME} \
                        ${APP_NAME}=${DOCKER_IMAGE}:${GIT_COMMIT_SHORT} \
                        -n staging
                """
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
            }
            steps {
                sh """
                    kubectl set image deployment/${APP_NAME} \
                        ${APP_NAME}=${DOCKER_IMAGE}:${GIT_COMMIT_SHORT} \
                        -n production
                """
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        success {
            slackSend channel: '#deployments',
                      color: 'good',
                      message: "Deployment successful: ${APP_NAME} (${GIT_COMMIT_SHORT})"
        }
        failure {
            slackSend channel: '#deployments',
                      color: 'danger',
                      message: "Deployment failed: ${APP_NAME} - ${BUILD_URL}"
        }
    }
}

Node.js Pipeline

pipeline {
    agent {
        docker {
            image 'node:18-alpine'
        }
    }

    environment {
        npm_config_cache = 'npm-cache'
        CI = 'true'
    }

    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }

        stage('Lint') {
            steps {
                sh 'npm run lint'
            }
        }

        stage('Test') {
            steps {
                sh 'npm test -- --coverage'
            }
            post {
                always {
                    publishHTML([
                        allowMissing: false,
                        reportDir: 'coverage/lcov-report',
                        reportFiles: 'index.html',
                        reportName: 'Coverage Report'
                    ])
                }
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }

        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh 'npm run deploy'
            }
        }
    }
}

Shared Libraries

Directory Structure

(root)
├── src/
│   └── org/
│       └── mycompany/
│           └── Utils.groovy
├── vars/
│   ├── buildJavaApp.groovy
│   └── deployToKubernetes.groovy
└── resources/
    └── templates/
        └── deployment.yaml

vars/buildJavaApp.groovy

def call(Map config = [:]) {
    pipeline {
        agent any

        tools {
            maven config.mavenVersion ?: 'Maven-3.8'
            jdk config.jdkVersion ?: 'JDK-17'
        }

        stages {
            stage('Build') {
                steps {
                    sh 'mvn clean compile'
                }
            }

            stage('Test') {
                steps {
                    sh 'mvn test'
                }
            }

            stage('Package') {
                steps {
                    sh 'mvn package -DskipTests'
                }
            }
        }
    }
}

Using Shared Library

// Jenkinsfile
@Library('my-shared-library') _

buildJavaApp(
    mavenVersion: 'Maven-3.8',
    jdkVersion: 'JDK-17'
)

Best Practices

Pipeline Tips

pipeline {
    agent any

    options {
        // Timeout for entire pipeline
        timeout(time: 30, unit: 'MINUTES')

        // Discard old builds
        buildDiscarder(logRotator(numToKeepStr: '10'))

        // Prevent concurrent builds
        disableConcurrentBuilds()

        // Timestamps in console output
        timestamps()

        // Skip checkout if not needed
        skipDefaultCheckout()
    }

    stages {
        stage('Build') {
            options {
                // Stage-specific timeout
                timeout(time: 10, unit: 'MINUTES')
                retry(3)
            }
            steps {
                sh 'mvn clean package'
            }
        }
    }
}

Summary

FeatureDescription
DeclarativeSimpler syntax, opinionated
ScriptedFull Groovy power, flexible
StagesLogical groupings of steps
ParallelConcurrent execution
PostCleanup and notifications

Jenkins Pipeline enables powerful, maintainable CI/CD workflows defined as code.

Advertisement

MR

Moshiour Rahman

Software Architect & AI Engineer

Share:
MR

Moshiour Rahman

Software Architect & AI Engineer

Enterprise software architect with deep expertise in financial systems, distributed architecture, and AI-powered applications. Building large-scale systems at Fortune 500 companies. Specializing in LLM orchestration, multi-agent systems, and cloud-native solutions. I share battle-tested patterns from real enterprise projects.

Related Articles

DevOps

GitHub Actions CI/CD Pipeline: Complete Tutorial

Build automated CI/CD pipelines with GitHub Actions. Learn workflows, jobs, actions, and deploy applications automatically with practical examples.

DevOps

GitHub Actions CI/CD: Automate Your Development Workflow

Master GitHub Actions for CI/CD pipelines. Learn workflows, jobs, actions, secrets management, and deploy to any cloud platform automatically.

DevOps

GitHub Actions CI/CD: From Push to Production in 10 Minutes

Build a complete CI/CD pipeline for Spring Boot: automated tests, Docker builds, staging deployments, and production releases with approval gates.

Comments

Comments are powered by GitHub Discussions.

Configure Giscus at giscus.app to enable comments.