Docker Concepts

Overview

Docker provides operating system-level virtualization through containerization, packaging applications with their dependencies into isolated, portable units. Containers share the host operating system kernel while maintaining isolated process spaces, file systems, and network interfaces. This architecture delivers consistent runtime environments across development, testing, and production without the overhead of full virtual machines.

The Docker platform emerged from Linux container technologies (LXC, cgroups, namespaces) to standardize container creation, distribution, and execution. A Docker container runs as a process on the host system, isolated through kernel features rather than hardware virtualization. This process-level isolation allows containers to start in milliseconds and consume minimal resources compared to virtual machines.

Docker addresses the "works on my machine" problem by encapsulating an application's complete runtime environment: code, system libraries, dependencies, configuration files, and environment variables. The same container image produces identical behavior whether running on a developer's laptop, CI/CD server, or production cluster.

For Ruby applications, Docker eliminates dependency conflicts between system Ruby versions, gem dependencies, and native extensions. A containerized Ruby application includes the exact Ruby version, bundled gems, and compiled native extensions, guaranteeing consistent execution across environments.

# Without Docker: System-dependent execution
require 'nokogiri'  # May fail if libxml2 version incompatible
require 'pg'        # Requires PostgreSQL client libraries

# With Docker: Self-contained execution
# All dependencies packaged in container image
# Same behavior on macOS dev machine and Linux production

Key Principles

Docker operates through three fundamental components: images, containers, and registries. An image serves as a read-only template containing the application code, runtime, libraries, and configuration. A container represents a running instance of an image, executing as an isolated process. Registries store and distribute images, enabling sharing across teams and deployment targets.

Images compose from layers, each representing a filesystem change. When building an image, Docker creates a new layer for each instruction in the Dockerfile. Layers stack using a union filesystem, presenting multiple read-only layers plus one writable container layer as a single filesystem. This layering enables efficient storage through layer reuse—multiple images sharing common layers consume disk space only once.

Container isolation relies on Linux kernel features: namespaces and cgroups. Namespaces provide process, network, mount, and user isolation, giving each container its own view of system resources. Control groups (cgroups) limit resource consumption, preventing containers from exhausting host CPU, memory, or I/O.

The Docker daemon manages container lifecycle: pulling images, creating containers, starting/stopping processes, and cleaning up resources. The Docker client communicates with the daemon through a REST API, accepting commands and returning results. This client-server architecture allows remote container management.

Copy-on-write filesystem semantics optimize storage and performance. When a container modifies a file from an image layer, Docker copies the file to the container's writable layer before making changes. The original image layer remains unchanged, allowing multiple containers to share the same base layers while maintaining independent writable state.

Docker networking creates virtual networks connecting containers. The default bridge network assigns each container a private IP address, enabling inter-container communication. Custom networks provide DNS-based service discovery—containers reference each other by name rather than IP address. Host networking removes isolation, allowing containers direct access to host network interfaces.

Volume management persists data beyond container lifecycle. Named volumes store data in Docker-managed locations, surviving container deletion. Bind mounts map host filesystem paths into containers, useful for development workflows where code changes should immediately affect running containers. Volumes and bind mounts bypass the copy-on-write filesystem, improving I/O performance for data-intensive operations.

# Ruby application with database dependency
# Container isolation means each service runs independently

# config/database.yml in containerized Rails app
production:
  adapter: postgresql
  host: postgres  # DNS name from Docker network
  database: app_production
  username: <%= ENV['DB_USER'] %>
  password: <%= ENV['DB_PASSWORD'] %>

Ruby Implementation

Ruby applications integrate with Docker through multiple approaches: containerizing Ruby applications in Docker images, using Ruby Docker SDKs for programmatic container management, and building development environments with Docker Compose.

The docker-api gem provides Ruby bindings for the Docker Remote API, enabling container lifecycle management from Ruby code. This gem wraps HTTP endpoints, presenting an object-oriented interface for images, containers, volumes, and networks.

require 'docker-api'

# Connect to Docker daemon
Docker.url = 'unix:///var/run/docker.sock'
Docker.options = { read_timeout: 300 }

# Pull image
image = Docker::Image.create('fromImage' => 'ruby:3.2-alpine')

# Create and start container
container = Docker::Container.create(
  'Image' => 'ruby:3.2-alpine',
  'Cmd' => ['ruby', '-e', 'puts RUBY_VERSION'],
  'HostConfig' => {
    'Memory' => 536870912,  # 512MB
    'CpuShares' => 512
  }
)

container.start
output = container.logs(stdout: true)
puts output  # => 3.2.x

container.stop
container.delete

Ruby Dockerfiles follow conventions balancing image size, build performance, and security. Alpine Linux base images minimize size, though they require additional packages for gems with native extensions. Debian-based images provide broader compatibility at the cost of larger image size.

# Multi-stage Dockerfile for Rails application
FROM ruby:3.2-alpine AS builder

# Install build dependencies
RUN apk add --no-cache \
  build-base \
  postgresql-dev \
  nodejs \
  yarn

WORKDIR /app

# Install gems (cached layer)
COPY Gemfile Gemfile.lock ./
RUN bundle config set --local without 'development test' && \
    bundle install --jobs=4

# Install JavaScript dependencies
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile

# Copy application code
COPY . .

# Precompile assets
RUN RAILS_ENV=production \
    SECRET_KEY_BASE=dummy \
    bundle exec rails assets:precompile

# Production stage
FROM ruby:3.2-alpine

RUN apk add --no-cache \
  postgresql-client \
  tzdata \
  nodejs

WORKDIR /app

# Copy installed gems from builder
COPY --from=builder /usr/local/bundle /usr/local/bundle

# Copy application and assets
COPY --from=builder /app /app

# Create non-root user
RUN addgroup -g 1000 rails && \
    adduser -D -u 1000 -G rails rails && \
    chown -R rails:rails /app

USER rails

EXPOSE 3000

CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]

The Docker Ruby SDK supports streaming operations for container logs and execution:

require 'docker-api'

container = Docker::Container.get('rails-app')

# Stream logs in real-time
container.streaming_logs(stdout: true, follow: true) do |stream, chunk|
  puts chunk
end

# Execute commands inside running container
stdout, stderr, status = container.exec(['bundle', 'exec', 'rails', 'console']) do |stream, chunk|
  puts "#{stream}: #{chunk}"
end

Managing Docker from Ruby enables dynamic test environments, creating and destroying containers for each test suite:

require 'docker-api'
require 'minitest/autorun'

class DockerIntegrationTest < Minitest::Test
  def setup
    # Create PostgreSQL container for testing
    @db_container = Docker::Container.create(
      'Image' => 'postgres:15-alpine',
      'Env' => [
        'POSTGRES_PASSWORD=test',
        'POSTGRES_DB=test_db'
      ],
      'HostConfig' => {
        'PortBindings' => {
          '5432/tcp' => [{ 'HostPort' => '5433' }]
        }
      }
    )
    @db_container.start
    
    # Wait for PostgreSQL to accept connections
    sleep 2
  end
  
  def teardown
    @db_container.stop
    @db_container.delete(force: true)
  end
  
  def test_database_operations
    # Test code using database on port 5433
  end
end

Dockerized Ruby applications configure through environment variables rather than configuration files, following twelve-factor methodology:

# config/initializers/redis.rb
REDIS_URL = ENV.fetch('REDIS_URL', 'redis://localhost:6379/0')
$redis = Redis.new(url: REDIS_URL)

# Dockerfile
ENV REDIS_URL=redis://redis:6379/0

# docker-compose.yml sets this automatically through service linking

Practical Examples

Containerizing a Sinatra application demonstrates basic Docker workflow from development through deployment. The application requires Ruby runtime, gems, and configuration:

# app.rb
require 'sinatra'
require 'json'

set :bind, '0.0.0.0'
set :port, ENV.fetch('PORT', 4567)

get '/health' do
  content_type :json
  { status: 'ok', timestamp: Time.now.to_i }.to_json
end

get '/env' do
  content_type :json
  { environment: ENV['RACK_ENV'] || 'development' }.to_json
end

FROM ruby:3.2-slim

WORKDIR /app

COPY Gemfile Gemfile.lock ./
RUN bundle install --without development test

COPY . .

EXPOSE 4567

CMD ["ruby", "app.rb"]

Building and running this container:

docker build -t sinatra-app:1.0 .
docker run -d -p 8080:4567 --name web sinatra-app:1.0
curl http://localhost:8080/health
# => {"status":"ok","timestamp":1234567890}

Multi-container applications use Docker Compose to orchestrate services. A Rails application with PostgreSQL and Redis requires coordinated startup, networking, and volume management:

# docker-compose.yml
version: '3.8'

services:
  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: app_production
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    command: redis-server --appendonly yes

  web:
    build:
      context: .
      dockerfile: Dockerfile
    command: bundle exec puma -C config/puma.rb
    environment:
      RAILS_ENV: production
      DATABASE_URL: postgresql://app:secret@postgres/app_production
      REDIS_URL: redis://redis:6379/0
      SECRET_KEY_BASE: ${SECRET_KEY_BASE}
    ports:
      - "3000:3000"
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_started
    volumes:
      - ./log:/app/log
      - ./tmp:/app/tmp

  worker:
    build:
      context: .
      dockerfile: Dockerfile
    command: bundle exec sidekiq -C config/sidekiq.yml
    environment:
      RAILS_ENV: production
      DATABASE_URL: postgresql://app:secret@postgres/app_production
      REDIS_URL: redis://redis:6379/0
    depends_on:
      - postgres
      - redis

volumes:
  postgres_data:
  redis_data:

Development environments benefit from hot-reloading, mounting source code as volumes:

# docker-compose.dev.yml
version: '3.8'

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile.dev
    command: bundle exec rails server -b 0.0.0.0
    environment:
      RAILS_ENV: development
    volumes:
      - .:/app
      - bundle_cache:/usr/local/bundle
    ports:
      - "3000:3000"
    stdin_open: true
    tty: true

volumes:
  bundle_cache:

# Dockerfile.dev
FROM ruby:3.2

WORKDIR /app

# Install dependencies once
COPY Gemfile Gemfile.lock ./
RUN bundle install

# Source code mounted as volume at runtime
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

Running tests in containers isolates test dependencies from the host system:

# test/test_helper.rb
require 'minitest/autorun'
require 'docker-api'

class DockerTestHelper
  def self.setup_test_database
    @container = Docker::Container.create(
      'Image' => 'postgres:15-alpine',
      'Env' => ['POSTGRES_PASSWORD=test'],
      'HostConfig' => {
        'AutoRemove' => true,
        'PortBindings' => { '5432/tcp' => [{ 'HostPort' => '0' }] }
      }
    )
    @container.start
    
    # Get dynamically assigned port
    port_info = @container.json['NetworkSettings']['Ports']['5432/tcp']
    port = port_info.first['HostPort']
    
    ENV['TEST_DATABASE_URL'] = "postgresql://postgres:test@localhost:#{port}/postgres"
    
    # Wait for PostgreSQL startup
    30.times do
      break if system("pg_isready -h localhost -p #{port} -q")
      sleep 0.5
    end
  end
  
  def self.teardown_test_database
    @container.stop if @container
  end
end

Minitest.after_run { DockerTestHelper.teardown_test_database }

Container health checks enable reliable service dependencies. A Rails application waits for database availability before starting:

FROM ruby:3.2

WORKDIR /app

COPY Gemfile Gemfile.lock ./
RUN bundle install

COPY . .

# Health check pings Rails app
HEALTHCHECK --interval=30s --timeout=3s --start-period=40s \
  CMD curl -f http://localhost:3000/health || exit 1

CMD ["bundle", "exec", "puma"]

# config/routes.rb
Rails.application.routes.draw do
  get '/health', to: proc { 
    status = ActiveRecord::Base.connection.active? ? 200 : 503
    [status, { 'Content-Type' => 'application/json' }, 
     [{ database: status == 200 }.to_json]]
  }
end

Implementation Approaches

Containerization strategies vary based on application architecture, deployment target, and operational requirements. Monolithic applications typically deploy in single containers, while microservices architectures distribute across multiple containers with service-specific optimization.

Single-stage builds create images in one Dockerfile, installing all dependencies and copying code sequentially. This approach suits simple applications but produces larger images containing build tools and intermediate artifacts:

FROM ruby:3.2

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
  build-essential \
  libpq-dev \
  nodejs

# Install application dependencies
COPY Gemfile Gemfile.lock ./
RUN bundle install

COPY . .

CMD ["bundle", "exec", "puma"]

Multi-stage builds separate build-time dependencies from runtime requirements. The builder stage compiles code and installs dependencies, then copies only necessary artifacts to the final stage:

FROM ruby:3.2 AS builder

RUN apt-get update && apt-get install -y build-essential libpq-dev
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle install --jobs=4

FROM ruby:3.2-slim

RUN apt-get update && apt-get install -y libpq5
WORKDIR /app
COPY --from=builder /usr/local/bundle /usr/local/bundle
COPY . .

CMD ["bundle", "exec", "puma"]

Base image selection impacts image size, security surface, and dependency availability. Alpine Linux produces minimal images but requires manual package installation for gems with native extensions. Debian provides comprehensive package repositories with larger base image size. Distroless images eliminate package managers and shells, reducing attack surface for production deployments:

# Alpine: smallest, requires explicit dependencies
FROM ruby:3.2-alpine
RUN apk add --no-cache postgresql-client

# Debian slim: balanced size and compatibility  
FROM ruby:3.2-slim
RUN apt-get update && apt-get install -y libpq5

# Full Debian: largest, complete tooling
FROM ruby:3.2

Layer caching optimization arranges Dockerfile instructions to maximize cache hits. Dependencies change infrequently compared to application code, so copying dependency files before source code allows Docker to reuse cached layers when only code changes:

# Inefficient: code changes invalidate all subsequent layers
COPY . .
RUN bundle install

# Efficient: dependency layer cached unless Gemfile changes
COPY Gemfile Gemfile.lock ./
RUN bundle install
COPY . .

Container orchestration platforms manage multi-container deployments across clusters. Docker Swarm provides native clustering, while Kubernetes offers advanced scheduling, service discovery, and self-healing. Orchestration abstracts infrastructure, allowing applications to scale horizontally through container replication.

Development workflows balance between running services locally versus in containers. Docker Compose enables consistent local environments matching production, eliminating "works on my machine" discrepancies. Volume mounting supports rapid iteration, reflecting code changes without rebuilding images:

services:
  app:
    build: .
    volumes:
      - .:/app        # Hot reload application code
      - gems:/bundle  # Cache installed gems
    environment:
      BUNDLE_PATH: /bundle

Continuous integration builds container images as build artifacts, versioning through tags. Each commit produces a tagged image pushed to a registry. Deployment stages pull specific image tags rather than rebuilding, ensuring tested artifacts reach production unchanged:

# .gitlab-ci.yml
build:
  stage: build
  script:
    - docker build -t registry.example.com/app:$CI_COMMIT_SHA .
    - docker push registry.example.com/app:$CI_COMMIT_SHA
    
deploy_staging:
  stage: deploy
  script:
    - docker pull registry.example.com/app:$CI_COMMIT_SHA
    - docker tag registry.example.com/app:$CI_COMMIT_SHA registry.example.com/app:staging
    - docker service update --image registry.example.com/app:staging app_staging

Tools & Ecosystem

Docker Compose orchestrates multi-container applications through declarative YAML configuration. Service definitions specify image, build context, environment variables, ports, volumes, and dependencies. Compose creates isolated networks for service communication and manages container lifecycle collectively:

services:
  app:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - db
    environment:
      DATABASE_URL: postgresql://db/app
  
  db:
    image: postgres:15
    volumes:
      - db_data:/var/lib/postgresql/data

volumes:
  db_data:

Docker Hub serves as the default public registry for image distribution. Organizations host private registries for internal images, using authentication to control access. Registry mirrors cache frequently pulled images, reducing bandwidth and improving pull performance in distributed teams:

# Push to Docker Hub
docker tag myapp:latest username/myapp:1.0
docker push username/myapp:1.0

# Push to private registry
docker tag myapp:latest registry.company.com/myapp:1.0
docker push registry.company.com/myapp:1.0

BuildKit modernizes Docker image building with improved performance, build caching, and security. Enabled through environment variables or daemon configuration, BuildKit parallelizes independent build stages, mounts secrets without including them in image layers, and provides better cache invalidation:

# syntax=docker/dockerfile:1.4

FROM ruby:3.2

# Mount secret during build without storing in image
RUN --mount=type=secret,id=bundle_config \
  bundle config --local gems.example.com $(cat /run/secrets/bundle_config)

# Cache bundle install across builds
RUN --mount=type=cache,target=/usr/local/bundle \
  bundle install

Docker Desktop provides GUI management on macOS and Windows, packaging the Docker daemon with Kubernetes support. The CLI remains primary for automation, while the desktop application simplifies resource allocation, volume browsing, and container inspection.

Watchtower automates container updates by monitoring registries for new image tags. When detecting updated images, Watchtower pulls the new version, stops the existing container, and starts a replacement with identical configuration:

services:
  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --interval 300  # Check every 5 minutes

Dive analyzes Docker images layer-by-layer, identifying inefficiencies and wasted space. The tool visualizes layer composition, showing which Dockerfile instructions contributed most to image size:

dive myapp:latest
# Shows layer breakdown, highlighting large files added to each layer

Hadolint lints Dockerfiles against best practices, catching common mistakes before building images:

hadolint Dockerfile
# DL3008: Pin versions in apt-get install
# DL3009: Delete apt-get lists after installing
# DL3015: Avoid additional packages in yum install

Container registries beyond Docker Hub include GitHub Container Registry, GitLab Container Registry, Amazon ECR, and Google Container Registry. These integrate with respective platforms' authentication and CI/CD systems:

# Authenticate to GitHub Container Registry
require 'docker-api'

Docker.authenticate!(
  'username' => ENV['GITHUB_USERNAME'],
  'password' => ENV['GITHUB_TOKEN'],
  'serveraddress' => 'ghcr.io'
)

image = Docker::Image.create('fromImage' => 'ghcr.io/owner/image:tag')

Common Pitfalls

Layer caching gotchas occur when developers assume Docker detects file content changes. Docker caches based on file metadata (modification time, inode), not content. Touching a file without changing its content invalidates the cache for that layer and all subsequent layers:

# Cache invalidated even if Gemfile unchanged
RUN touch Gemfile
COPY Gemfile Gemfile.lock ./
RUN bundle install  # Runs every build despite unchanged dependencies

Image tag mutability causes deployment inconsistencies. Tags like latest or stable point to different images over time. Pulling ruby:latest today yields a different image than pulling it next month. Production deployments should reference immutable tags or digest hashes:

# Mutable tag - different image tomorrow
docker pull ruby:latest

# Immutable digest - always same image
docker pull ruby@sha256:abc123...

# Versioned tag - more stable but still mutable
docker pull ruby:3.2.2

Volume permission issues arise from user ID mismatches between container and host. A container running as user 1000 cannot write to host directories owned by user 1001. Alpine-based images compound this by lacking user synchronization tools:

# Create matching user ID
RUN addgroup -g 1000 app && \
    adduser -D -u 1000 -G app app

USER app

# Now writes to volumes succeed if host directories owned by UID 1000

Container networking confusion stems from port publishing versus exposure. EXPOSE documents ports without making them accessible from the host. Publishing (-p flag) actually forwards traffic:

EXPOSE 3000  # Documentation only

# Port inaccessible from host
docker run myapp

# Port accessible on host:3000
docker run -p 3000:3000 myapp

Build context size impacts build performance. Dockerfile COPY . . sends entire directory to Docker daemon, including node_modules, .git, and other unnecessary files. .dockerignore excludes files from build context:

# .dockerignore
.git
node_modules
tmp
log
*.log
.env

Environment variable injection through ENV bakes values into images, exposing secrets in image layers. Runtime variable injection through docker run -e or Compose keeps secrets out of images:

# Bad: Secret in image layer, visible in docker history
ENV DATABASE_PASSWORD=secret123

# Good: Document variable without value
ENV DATABASE_PASSWORD=

# Inject at runtime
docker run -e DATABASE_PASSWORD=secret123 myapp

Resource limits prevent container resource exhaustion affecting host stability. Without limits, containers consume unlimited CPU and memory, potentially crashing the host:

services:
  app:
    image: myapp
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 512M

Timezone discrepancies occur because containers default to UTC regardless of host timezone. Applications generating timestamps or scheduling tasks require explicit timezone configuration:

ENV TZ=America/New_York
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

Signal handling failures prevent graceful shutdowns. Docker sends SIGTERM to container PID 1, but shell script wrappers don't forward signals by default. Applications must handle SIGTERM for clean shutdown:

# Bad: Shell doesn't forward signals
CMD bundle exec puma

# Good: exec replaces shell with application
CMD exec bundle exec puma

# Better: Use JSON array syntax (no shell)
CMD ["bundle", "exec", "puma"]

Container filesystem persistence confusion leads to data loss. Container filesystems are ephemeral—data written outside volumes disappears when containers stop. Critical data requires volume mounting:

# config/initializers/uploads.rb
# Bad: Writes to ephemeral container filesystem
CarrierWave.configure do |config|
  config.root = Rails.root.join('public/uploads')
end

# Good: Writes to persistent volume
CarrierWave.configure do |config|
  config.root = '/mnt/uploads'  # Mounted volume
end

Dependency version pinning prevents broken builds from upstream changes. Unpinned base images or system packages introduce breaking changes:

# Unpinned: Breaks when ruby:3.2 points to incompatible version
FROM ruby:3.2

# Pinned: Reproducible builds
FROM ruby:3.2.2-alpine3.18

# System packages need pinning too
RUN apk add --no-cache postgresql-client=15.3-r0

Reference

Core Components

Dockerfile Instructions

Docker CLI Commands

Docker Compose Commands

Volume Types

Network Types

Health Check Parameters

Resource Limits

Image Tag Conventions