CI/CD Pipeline Design

Overview

CI/CD pipeline design structures the automated processes that build, test, and deploy software changes from source code to production environments. A pipeline transforms code commits into deployable artifacts through a series of automated stages, each validating different aspects of the change.

The pipeline concept originated from manufacturing assembly lines, where products move through sequential stations. In software development, code changes flow through stages like compilation, testing, security scanning, and deployment. Each stage acts as a quality gate that must pass before proceeding.

Three distinct practices exist within CI/CD:

Continuous Integration (CI) merges code changes into a shared repository multiple times per day. Each integration triggers an automated build and test suite, detecting integration conflicts and bugs early. Developers receive rapid feedback about whether their changes break the build or fail tests.

Continuous Delivery (CD) extends CI by ensuring code remains in a deployable state. After passing all automated tests, changes reach a staging environment identical to production. The deployment to production requires manual approval, maintaining human control over release timing.

Continuous Deployment removes the manual approval step, automatically deploying every change that passes all pipeline stages directly to production. This practice requires high confidence in automated testing and monitoring.

Pipeline design determines build speed, reliability, deployment frequency, and the organization's ability to deliver features. A poorly designed pipeline creates bottlenecks, false positives, and deployment anxiety. An effective pipeline provides fast feedback, catches errors early, and enables frequent releases.

# Basic three-stage pipeline concept
stages:
  - build    # Compile code, resolve dependencies
  - test     # Run automated test suites
  - deploy   # Push to target environment

The pipeline architecture must balance speed against thoroughness. Running all tests in parallel provides fast feedback but consumes more resources. Sequential execution conserves resources but increases feedback time. Modern pipelines often combine both approaches, running fast unit tests first and slower integration tests in parallel.

Key Principles

Pipeline as Code stores pipeline definitions in version control alongside application code. This practice treats infrastructure configuration with the same rigor as application code, enabling code review, versioning, and rollback. Teams can reproduce the entire build process from the repository without external documentation.

Fail Fast philosophy places faster, cheaper checks early in the pipeline. Syntax validation and linting run before compilation. Unit tests execute before integration tests. Security scanning precedes deployment. This ordering provides rapid feedback and conserves resources by avoiding expensive operations when basic checks fail.

Idempotency ensures pipeline stages produce identical results given identical inputs. Running the same commit through the pipeline multiple times generates the same artifacts. This property enables reliable debugging, rollback, and parallel execution without side effects.

Isolation separates pipeline stages into independent units that don't affect each other. Each stage receives a clean environment, preventing state from one stage bleeding into another. Artifacts pass explicitly between stages rather than relying on shared filesystem state.

Artifact Promotion builds deployable artifacts once during the build stage, then promotes the same artifact through subsequent environments. The binary deployed to production is identical to the one tested in staging. This eliminates "works on my machine" problems caused by rebuilding for each environment.

The Build-Measure-Learn cycle provides feedback loops at multiple levels:

Code Commit → Build → Test → Deploy → Monitor
     ↑                                    |
     └────────── Feedback Loop ───────────┘

Monitoring production behavior informs future development, creating a continuous improvement cycle.

Deterministic Builds produce identical outputs from identical inputs, regardless of build time or environment. Dependency versions are pinned, timestamps are excluded from artifacts, and random number generators use fixed seeds during builds. Determinism enables caching, debugging, and confidence in artifact integrity.

Stage Gates define criteria that must be met before advancing to the next stage. Gates can be automated (all tests pass) or manual (security review approved). Gates prevent flawed code from reaching production but should be minimized to maintain flow.

Parallelization executes independent tasks concurrently, reducing total pipeline time. Test suites split across multiple runners, different deployment targets update simultaneously, and independent validation steps run in parallel. However, parallelization increases complexity and resource usage.

Observability provides visibility into pipeline execution. Logs, metrics, and traces expose what happened, how long it took, and why failures occurred. Teams can diagnose problems without reproducing failures locally.

Trunk-Based Development complements CI/CD by maintaining a single main branch that's always in a deployable state. Short-lived feature branches merge frequently, reducing integration complexity. This practice contrasts with long-lived feature branches that accumulate divergence from the main codebase.

Implementation Approaches

Scripted Pipelines define stages using imperative scripts in languages like Groovy, Bash, or Ruby. Scripts provide maximum flexibility but require more maintenance and testing. Teams familiar with the scripting language can implement complex logic, conditional execution, and dynamic stage generation.

# Rakefile defining a scripted pipeline
namespace :ci do
  desc "Run the full CI pipeline"
  task pipeline: [:build, :test, :security_scan, :deploy]
  
  task :build do
    sh "bundle install --deployment"
    sh "rake assets:precompile"
  end
  
  task :test do
    sh "bundle exec rspec"
    sh "bundle exec rubocop"
  end
  
  task :security_scan do
    sh "bundle exec brakeman --no-pager"
    sh "bundle audit check --update"
  end
  
  task :deploy do
    env = ENV['DEPLOY_ENV'] || 'staging'
    sh "cap #{env} deploy"
  end
end

Declarative Pipelines specify desired end states using configuration files (YAML, JSON, or DSLs). The CI/CD platform interprets the configuration and executes the necessary steps. Declarative approaches reduce complexity but limit flexibility.

# .gitlab-ci.yml declarative pipeline
stages:
  - build
  - test
  - deploy

variables:
  BUNDLE_PATH: vendor/bundle

build:
  stage: build
  script:
    - bundle install
    - rake assets:precompile
  artifacts:
    paths:
      - vendor/bundle
      - public/assets

test:
  stage: test
  script:
    - bundle exec rspec
    - bundle exec rubocop
  coverage: '/\(\d+\.\d+\%\) covered/'

deploy_staging:
  stage: deploy
  script:
    - bundle exec cap staging deploy
  environment:
    name: staging
  only:
    - main

Container-Based Pipelines execute each stage inside Docker containers, ensuring consistent environments across local development, CI, and production. Container images define the exact versions of system libraries, language runtimes, and tools.

# GitHub Actions with containers
name: CI Pipeline

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: ruby:3.2
    
    services:
      postgres:
        image: postgres:14
        env:
          POSTGRES_PASSWORD: postgres
    
    steps:
      - uses: actions/checkout@v3
      - name: Install dependencies
        run: |
          gem install bundler
          bundle install
      - name: Run tests
        run: bundle exec rspec
        env:
          DATABASE_URL: postgresql://postgres:postgres@postgres/test

Matrix Builds test code against multiple configurations simultaneously. Ruby applications might test against different Ruby versions, database engines, or operating systems. Matrix strategies identify compatibility issues early.

# Testing across Ruby versions
strategy:
  matrix:
    ruby: [3.0, 3.1, 3.2, 3.3]
    database: [postgresql, mysql]
    
steps:
  - uses: ruby/setup-ruby@v1
    with:
      ruby-version: ${{ matrix.ruby }}
      bundler-cache: true

Pipeline Templates create reusable pipeline definitions that multiple projects inherit. Templates centralize common patterns, enforce organizational standards, and reduce duplication. Projects extend templates with project-specific customization.

Blue-Green Deployment Pipelines maintain two identical production environments. The pipeline deploys to the inactive environment, runs smoke tests, then switches traffic. If problems occur, traffic switches back to the previous version.

Canary Deployment Pipelines gradually roll out changes to a subset of users. The pipeline deploys to a small percentage of servers, monitors metrics, then expands the rollout if metrics remain healthy. Automated rollback triggers if error rates increase.

Feature Flag Pipelines deploy code to production with new features disabled. The pipeline deploys frequently, but feature activation occurs separately through configuration. This decouples deployment risk from feature release.

Tools & Ecosystem

Jenkins provides a self-hosted automation server with extensive plugin ecosystem. Pipelines define using Groovy-based DSL or declarative syntax. Jenkins scales to thousands of pipelines but requires infrastructure management.

// Jenkinsfile
pipeline {
    agent { docker { image 'ruby:3.2' } }
    stages {
        stage('Build') {
            steps {
                sh 'bundle install'
            }
        }
        stage('Test') {
            steps {
                sh 'bundle exec rspec'
            }
        }
    }
}

GitHub Actions integrates directly with GitHub repositories, executing workflows on GitHub-hosted or self-hosted runners. YAML workflow files define pipelines with access to a marketplace of pre-built actions.

GitLab CI/CD provides integrated pipelines within GitLab, using YAML configuration files. GitLab includes built-in container registry, package registry, and deployment environments. The platform supports both SaaS and self-hosted installations.

CircleCI offers cloud-hosted CI/CD with local debugging capabilities. The circleci CLI reproduces pipeline execution on developer machines. CircleCI emphasizes caching strategies to accelerate builds.

# .circleci/config.yml
version: 2.1

executors:
  ruby-executor:
    docker:
      - image: cimg/ruby:3.2
      
jobs:
  build:
    executor: ruby-executor
    steps:
      - checkout
      - restore_cache:
          keys:
            - bundle-v1-{{ checksum "Gemfile.lock" }}
      - run: bundle install --path vendor/bundle
      - save_cache:
          key: bundle-v1-{{ checksum "Gemfile.lock" }}
          paths:
            - vendor/bundle

Buildkite separates pipeline orchestration from build execution. Organizations run their own build agents while Buildkite manages job scheduling and the web interface. This architecture provides control over build environments while outsourcing pipeline management.

Ruby-Specific Tools:

Rake automates task execution with dependency management. Many Ruby projects define CI tasks in Rakefiles that CI platforms invoke.

# Rakefile with CI tasks
require 'rspec/core/rake_task'
require 'rubocop/rake_task'

RSpec::Core::RakeTask.new(:spec)
RuboCop::RakeTask.new

task ci: [:spec, :rubocop]

Bundler manages Ruby gem dependencies, ensuring consistent gem versions across environments. The Gemfile.lock file commits to version control, guaranteeing identical dependencies in CI and production.

Capistrano deploys Ruby applications to servers. The pipeline's deploy stage invokes Capistrano tasks to push code, restart services, and run migrations.

# config/deploy.rb
set :application, "my_app"
set :repo_url, "git@github.com:company/my_app.git"

namespace :deploy do
  after :published, :restart_app do
    on roles(:app) do
      execute :systemctl, "restart", "my_app"
    end
  end
end

RSpec, Minitest provide testing frameworks that CI pipelines execute. Pipelines parse test output to determine success/failure and extract coverage metrics.

Rubocop enforces code style. Pipelines fail builds that violate style rules, maintaining consistency across the codebase.

Brakeman scans Rails applications for security vulnerabilities. The pipeline incorporates security scanning before deployment stages.

# Security scanning in pipeline
security:
  stage: test
  script:
    - bundle exec brakeman --no-pager --format json -o brakeman.json
    - bundle audit check --update
  artifacts:
    reports:
      sast: brakeman.json

SimpleCov generates code coverage reports. Pipelines can enforce minimum coverage thresholds, failing builds that don't meet requirements.

Common Patterns

Fan-Out/Fan-In splits a single trigger into multiple parallel jobs that later merge results. Tests run in parallel across multiple runners, then results aggregate before deployment.

# Parallel test execution
test:
  stage: test
  parallel: 4
  script:
    - bundle exec rspec --tag ~slow
    
integration_test:
  stage: test
  script:
    - bundle exec rspec --tag slow

deploy:
  stage: deploy
  needs: [test, integration_test]  # Wait for both
  script:
    - bundle exec cap production deploy

Stage Promotion advances artifacts through environments: development → staging → production. Each environment runs progressively comprehensive validation.

deploy_staging:
  stage: deploy_staging
  script: bundle exec cap staging deploy
  only: [main]

deploy_production:
  stage: deploy_production
  script: bundle exec cap production deploy
  when: manual
  only: [main]

Dependency Caching stores build dependencies between pipeline runs. Subsequent builds restore cached gems, reducing installation time.

# GitLab caching
cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - vendor/bundle

before_script:
  - bundle install --path vendor/bundle

Artifact Storage preserves build outputs for later stages. The build stage generates compiled assets or packaged gems that deploy stages use.

Conditional Execution runs stages only when specific conditions are met. Deploy stages execute only on the main branch, security scans run nightly, performance tests trigger on release tags.

# Conditional Rake tasks
task :deploy do
  if ENV['CI_COMMIT_BRANCH'] == 'main'
    sh "bundle exec cap production deploy"
  else
    puts "Skipping deploy on non-main branch"
  end
end

Pipeline Triggers initiate pipelines from external events: scheduled jobs, API calls, or completion of other pipelines. Nightly builds run comprehensive test suites, dependency update pipelines trigger weekly.

Smoke Tests run quick validation after deployment to verify basic functionality. The pipeline deploys code, then executes smoke tests before marking the deployment successful.

# spec/smoke/deployment_spec.rb
RSpec.describe "Deployment smoke tests" do
  it "serves the homepage" do
    response = HTTP.get("https://#{ENV['APP_DOMAIN']}")
    expect(response.status).to eq(200)
  end
  
  it "connects to database" do
    expect { User.count }.not_to raise_error
  end
end

Rollback Stages provide quick reversion to previous versions when deployments fail. The pipeline detects failures through smoke tests or monitoring alerts, then executes rollback procedures.

Notification Integration sends alerts when pipelines fail or deployments complete. Integrations with Slack, email, or PagerDuty keep teams informed.

Approval Gates pause the pipeline pending human review. Production deployments often require manual approval, balancing automation with control.

Practical Examples

Basic Ruby Application Pipeline:

# .github/workflows/ci.yml
name: Ruby CI

on:
  push:
    branches: [main, develop]
  pull_request:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          bundler-cache: true
      - run: bundle exec rubocop
      
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:14
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          bundler-cache: true
          
      - name: Setup database
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost/test
        run: |
          bundle exec rails db:create
          bundle exec rails db:schema:load
          
      - name: Run tests
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost/test
        run: bundle exec rspec
        
      - name: Upload coverage
        uses: codecov/codecov-action@v3
        
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          bundler-cache: true
      - run: |
          bundle exec brakeman --no-pager
          bundle audit check --update

Multi-Environment Deployment Pipeline:

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy_staging
  - deploy_production

variables:
  BUNDLE_PATH: vendor/bundle
  RAILS_ENV: test

build:
  stage: build
  script:
    - bundle install
    - bundle exec rails assets:precompile
  artifacts:
    paths:
      - public/assets
      - vendor/bundle
    expire_in: 1 hour

test:
  stage: test
  services:
    - postgres:14
  variables:
    DATABASE_URL: postgresql://postgres:postgres@postgres/test
  script:
    - bundle install
    - bundle exec rails db:create db:schema:load
    - bundle exec rspec
    - bundle exec rubocop
  coverage: '/\(\d+\.\d+\%\) covered/'

deploy_staging:
  stage: deploy_staging
  script:
    - eval $(ssh-agent -s)
    - echo "$STAGING_DEPLOY_KEY" | tr -d '\r' | ssh-add -
    - bundle exec cap staging deploy
  environment:
    name: staging
    url: https://staging.example.com
  only:
    - main

deploy_production:
  stage: deploy_production
  script:
    - eval $(ssh-agent -s)
    - echo "$PRODUCTION_DEPLOY_KEY" | tr -d '\r' | ssh-add -
    - bundle exec cap production deploy
    - bundle exec rake smoke_tests:production
  environment:
    name: production
    url: https://example.com
  when: manual
  only:
    - main

Gem Publishing Pipeline:

# .github/workflows/gem-release.yml
name: Publish Gem

on:
  push:
    tags:
      - 'v*'

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        ruby: [3.0, 3.1, 3.2, 3.3]
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: ${{ matrix.ruby }}
          bundler-cache: true
      - run: bundle exec rspec

  publish:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          
      - name: Build gem
        run: gem build *.gemspec
        
      - name: Publish to RubyGems
        env:
          GEM_HOST_API_KEY: ${{ secrets.RUBYGEMS_API_KEY }}
        run: |
          mkdir -p ~/.gem
          echo "---" > ~/.gem/credentials
          echo ":rubygems_api_key: ${GEM_HOST_API_KEY}" >> ~/.gem/credentials
          chmod 0600 ~/.gem/credentials
          gem push *.gem

Database Migration Pipeline:

# lib/tasks/ci.rake
namespace :ci do
  desc "Run migrations in CI environment"
  task migrate_check: :environment do
    # Verify migrations run without errors
    begin
      Rake::Task['db:migrate'].invoke
      
      # Check for pending migrations
      ActiveRecord::Base.connection.migration_context.needs_migration?
      
      # Rollback and re-run to verify reversibility
      Rake::Task['db:rollback'].invoke
      Rake::Task['db:migrate'].invoke
      
      puts "Migration check passed"
    rescue => e
      puts "Migration check failed: #{e.message}"
      exit 1
    end
  end
  
  desc "Verify schema matches migrations"
  task schema_check: :environment do
    Rake::Task['db:schema:dump'].invoke
    
    if system("git diff --exit-code db/schema.rb")
      puts "Schema matches migrations"
    else
      puts "ERROR: Schema does not match migrations"
      puts "Run 'rails db:migrate' and commit the schema changes"
      exit 1
    end
  end
end

Design Considerations

Build Speed vs Thoroughness presents a fundamental trade-off. Comprehensive test suites increase confidence but slow feedback. Teams must balance coverage against developer productivity. Fast unit tests run on every commit, while slower integration tests trigger on merge to main or nightly.

Push vs Pull Deployment affects security and infrastructure complexity. Push deployments have the CI server connect to production servers, requiring the CI environment to hold production credentials. Pull deployments have production servers poll for new versions and self-update, limiting credential exposure but increasing production complexity.

Hosted vs Self-Hosted CI/CD platforms involve different trade-offs. Hosted solutions (GitHub Actions, CircleCI) eliminate infrastructure management but limit customization and may restrict sensitive data handling. Self-hosted options (Jenkins, GitLab self-managed) provide full control but require maintenance and scaling expertise.

Monorepo vs Multi-Repo pipelines scale differently. Monorepos run a single large pipeline that tests all code, while multi-repo architectures run multiple smaller pipelines. Monorepos simplify cross-project changes but slow down individual pipelines. Multi-repo provides faster feedback but complicates dependency management.

Deployment Frequency influences pipeline design. Organizations deploying hourly need fast, reliable pipelines with automated rollback. Those deploying weekly can tolerate slower pipelines with more manual gates. The pipeline should match the organization's risk tolerance and operational maturity.

Environment Parity determines how closely staging matches production. Identical environments catch deployment issues early but double infrastructure costs. Scaled-down staging environments reduce costs but miss load-related problems. The pipeline should validate in environments that match production's critical characteristics.

Test Pyramid guidance suggests many fast unit tests, fewer integration tests, and minimal end-to-end tests. Pipelines following this pattern provide quick feedback from unit tests while catching integration issues with targeted higher-level tests. Inverting the pyramid creates slow, flaky pipelines.

Secret Management strategies affect security posture. Environment variables provide simple secret injection but complicate rotation. Dedicated secret managers (HashiCorp Vault, AWS Secrets Manager) offer better security and rotation but increase complexity. The pipeline should access secrets without logging or persisting them.

Artifact Retention policies balance storage costs against debugging needs. Keeping all artifacts enables investigation of old deployments but consumes storage. Retaining recent artifacts and tagged releases provides debugging capability while controlling costs.

Pipeline Complexity grows with project requirements. Simple projects need basic build-test-deploy pipelines. Complex applications might require parallel testing, multiple deployment targets, gradual rollouts, and automated rollback. Adding complexity without necessity slows development and increases maintenance burden.

Failure Handling determines how pipelines respond to errors. Fail-fast approaches halt execution immediately, providing quick feedback but potentially wasting work from parallel jobs. Fail-late approaches complete all jobs, collecting comprehensive failure information but delaying feedback.

Reference

Pipeline Stage Reference

Common CI/CD Variables

Pipeline Configuration Checklist

Ruby Testing Commands

Deployment Strategies Comparison

Common Pipeline Metrics