Configuration Management

Overview

Configuration management controls how applications access settings, credentials, environment-specific values, and operational parameters. Software systems require different configurations for development, testing, staging, and production environments. Each environment needs distinct database connections, API endpoints, feature flags, and resource limits. Configuration management separates these environment-specific values from application code.

The practice emerged from the need to deploy identical code across multiple environments while varying behavior based on context. Early applications hardcoded values directly in source files, requiring code changes for each deployment target. This created maintenance problems and deployment risks. Modern configuration management externalizes settings, making applications portable and deployable without code modification.

Configuration management handles several categories of data. Environment variables control runtime behavior like debug modes and log levels. Connection strings specify database and service endpoints. Credentials include API keys, passwords, and certificates. Feature flags enable or disable functionality without code deployment. Resource limits define memory, timeout, and concurrency constraints.

The distinction between configuration and code defines the boundary. Configuration data changes between deployments of the same codebase. Code defines application logic and remains constant across environments. A database hostname is configuration; the database query logic is code. An API timeout value is configuration; the HTTP client implementation is code.

# Configuration - varies by environment
DATABASE_URL = ENV['DATABASE_URL']
MAX_CONNECTIONS = ENV.fetch('MAX_CONNECTIONS', 10).to_i

# Code - consistent across environments
class DatabasePool
  def initialize(url, max_connections)
    @url = url
    @max_connections = max_connections
  end
end

pool = DatabasePool.new(DATABASE_URL, MAX_CONNECTIONS)

Configuration management intersects with security, deployment automation, and application architecture. Insecure configuration storage exposes credentials. Manual configuration updates create deployment bottlenecks. Tightly coupled configuration complicates testing and development.

Key Principles

The twelve-factor app methodology established core configuration principles. Store configuration in environment variables, separate from code. Never commit credentials or environment-specific values to version control. Make configuration immediately visible and modifiable without code changes.

Environment parity requires identical configuration structure across all stages. Development, staging, and production use the same configuration keys with different values. This prevents configuration drift where environments diverge over time. Applications read DATABASE_URL in all environments, though the actual database servers differ.

Explicit dependency declaration ensures applications state all configuration requirements. Applications fail fast at startup when required configuration is missing rather than failing unpredictably during operation. Validation occurs before accepting traffic.

class Config
  def initialize
    @database_url = ENV.fetch('DATABASE_URL')
    @redis_url = ENV.fetch('REDIS_URL')
    @api_key = ENV.fetch('API_KEY')
    @log_level = ENV.fetch('LOG_LEVEL', 'info')
  rescue KeyError => e
    raise ConfigurationError, "Missing required configuration: #{e.message}"
  end
end

Configuration hierarchy establishes precedence when multiple sources provide values. Command-line arguments override environment variables, which override configuration files, which override application defaults. This layering allows specific deployments to override defaults without modifying base configuration.

Immutability treats configuration as read-only after application startup. Applications read configuration once during initialization and never modify values during runtime. This prevents temporal coupling where application behavior changes unexpectedly mid-execution. Configuration changes require process restart.

Type safety validates configuration values match expected types. String "true" differs from boolean true. Numeric limits require integers, not strings. Type coercion at configuration load time prevents runtime type errors.

class TypedConfig
  def initialize
    @port = Integer(ENV.fetch('PORT', 3000))
    @debug = ENV.fetch('DEBUG', 'false').downcase == 'true'
    @timeout = Float(ENV.fetch('TIMEOUT', 30.0))
    @workers = Integer(ENV.fetch('WORKERS', 4))
  end
rescue ArgumentError => e
  raise ConfigurationError, "Invalid configuration type: #{e.message}"
end

Secret separation distinguishes sensitive credentials from non-sensitive settings. Secrets include passwords, private keys, and API tokens. Non-sensitive configuration includes timeouts, URLs, and feature flags. Secrets require encryption at rest and restricted access. Non-sensitive configuration can use simpler storage.

Configuration versioning tracks changes to configuration schemas and values over time. When applications add or remove configuration requirements, version identifiers prevent mismatches between application code and configuration. Schema evolution requires backward compatibility periods.

Implementation Approaches

File-based configuration stores settings in structured files deployed alongside application code. YAML, JSON, TOML, or custom formats define configuration hierarchies. Applications read files at startup and parse values. File-based approaches work well for non-sensitive, relatively static configuration.

# config/settings.yml
development:
  database:
    host: localhost
    pool: 5
  cache:
    enabled: false

production:
  database:
    host: db.example.com
    pool: 25
  cache:
    enabled: true

Files support complex nested structures and comments for documentation. Version control tracks configuration changes. However, files complicate secret management and environment-specific overrides. Each environment requires separate files or templating.

Environment variable injection provides configuration through process environment. The operating system or container runtime sets variables before starting the application. Applications read variables using standard language facilities. This approach follows twelve-factor principles and works across languages and platforms.

Environment variables handle simple key-value pairs. Complex nested configuration requires flattening or encoding. Variable naming conventions prevent collisions. Prefixing with application identifiers namespaces variables: APP_DATABASE_URL, APP_CACHE_ENABLED.

class EnvConfig
  def self.load
    {
      database: {
        url: ENV['APP_DATABASE_URL'],
        pool: ENV.fetch('APP_DATABASE_POOL', 5).to_i
      },
      cache: {
        enabled: ENV.fetch('APP_CACHE_ENABLED', 'false') == 'true',
        ttl: ENV.fetch('APP_CACHE_TTL', 3600).to_i
      }
    }
  end
end

Configuration services centralize configuration in external systems. Applications query configuration servers at startup or runtime. Services like Consul, etcd, or cloud provider parameter stores provide distributed configuration. This enables dynamic updates and centralized management.

Configuration services support multiple applications sharing configuration. Central administration updates values without touching application deployments. Service discovery integrates with configuration, providing endpoint information. However, services add dependencies and latency. Applications must handle service unavailability.

Layered configuration combines multiple sources with precedence rules. Base configuration files provide defaults. Environment variables override file values. Command-line arguments override everything. Applications merge sources into final configuration.

class LayeredConfig
  def initialize
    @config = load_defaults
      .merge(load_file)
      .merge(load_environment)
      .merge(load_arguments)
  end

  private

  def load_defaults
    { port: 3000, log_level: 'info', workers: 4 }
  end

  def load_file
    return {} unless File.exist?('config.yml')
    YAML.load_file('config.yml')
  end

  def load_environment
    {
      port: ENV['PORT']&.to_i,
      log_level: ENV['LOG_LEVEL'],
      workers: ENV['WORKERS']&.to_i
    }.compact
  end

  def load_arguments
    # Parse ARGV for command-line overrides
    {}
  end
end

Templated configuration generates environment-specific files from templates. Template engines like ERB or Mustache substitute variables into configuration file templates. Build or deployment processes render templates with environment-specific values.

Templates maintain configuration structure while varying values. Single template generates multiple environment files. However, templating adds build complexity and can obscure actual deployed values.

Ruby Implementation

Ruby applications commonly use environment variables through ENV hash. The ENV object provides read and write access to process environment variables. Reading missing variables returns nil. The fetch method raises KeyError for required variables.

# Basic environment variable access
database_url = ENV['DATABASE_URL']
port = ENV.fetch('PORT', 3000).to_i

# Raising on missing required values
api_key = ENV.fetch('API_KEY') # Raises KeyError if missing

# Type conversion
debug_mode = ENV['DEBUG'] == 'true'
max_connections = Integer(ENV.fetch('MAX_CONNECTIONS', 10))

The dotenv gem loads environment variables from .env files during development. This simplifies local development without setting system-wide environment variables. The gem reads .env files and populates ENV before application startup.

# Gemfile
gem 'dotenv-rails', groups: [:development, :test]

# .env file (not committed to version control)
DATABASE_URL=postgresql://localhost/myapp_development
REDIS_URL=redis://localhost:6379
API_KEY=dev_api_key_12345

# config/application.rb
require 'dotenv/load'

The config gem provides layered YAML-based configuration with environment support. Configuration files live in config/settings directory. The gem merges settings files, environment-specific files, and local overrides.

# Gemfile
gem 'config'

# config/settings.yml
defaults: &defaults
  database:
    pool: 5
  cache:
    ttl: 3600

development:
  <<: *defaults
  database:
    host: localhost

production:
  <<: *defaults
  database:
    host: <%= ENV['DATABASE_HOST'] %>
    pool: 25

# Accessing configuration
Settings.database.host
Settings.cache.ttl
Settings.database.pool

Rails provides built-on configuration through config/application.rb and environment files. The Rails.configuration object holds framework and application settings. Environment-specific files override base configuration.

# config/application.rb
module MyApp
  class Application < Rails::Application
    config.time_zone = 'UTC'
    config.active_record.default_timezone = :utc
  end
end

# config/environments/production.rb
Rails.application.configure do
  config.log_level = :info
  config.cache_store = :redis_cache_store, { url: ENV['REDIS_URL'] }
end

# Accessing configuration
Rails.configuration.time_zone
Rails.configuration.log_level

Custom configuration classes encapsulate configuration logic and validation. These classes centralize configuration reading, type conversion, and validation.

class AppConfig
  class << self
    def database
      @database ||= DatabaseConfig.new
    end

    def cache
      @cache ||= CacheConfig.new
    end

    def api
      @api ||= APIConfig.new
    end
  end

  class DatabaseConfig
    attr_reader :url, :pool, :timeout

    def initialize
      @url = ENV.fetch('DATABASE_URL')
      @pool = Integer(ENV.fetch('DATABASE_POOL', 5))
      @timeout = Integer(ENV.fetch('DATABASE_TIMEOUT', 5000))
      validate!
    end

    private

    def validate!
      raise ConfigError, "Invalid pool size" if @pool < 1
      raise ConfigError, "Invalid timeout" if @timeout < 0
    end
  end

  class CacheConfig
    attr_reader :url, :ttl, :enabled

    def initialize
      @url = ENV['REDIS_URL']
      @ttl = Integer(ENV.fetch('CACHE_TTL', 3600))
      @enabled = ENV.fetch('CACHE_ENABLED', 'true') == 'true'
    end
  end
end

# Usage
AppConfig.database.url
AppConfig.cache.enabled

The dry-configurable gem provides a declarative DSL for defining configuration schemas. Settings include types, defaults, and constructors for value processing.

require 'dry-configurable'

class AppSettings
  extend Dry::Configurable

  setting :database_url, constructor: ->(v) { v.to_s }
  setting :database_pool, default: 5, constructor: ->(v) { Integer(v) }
  setting :cache_enabled, default: false, constructor: ->(v) { v == 'true' }
  setting :log_level, default: 'info'
end

AppSettings.config.database_url = ENV['DATABASE_URL']
AppSettings.config.database_pool = ENV.fetch('DATABASE_POOL', 5)
AppSettings.config.cache_enabled = ENV.fetch('CACHE_ENABLED', 'false')

AppSettings.config.database_url
AppSettings.config.database_pool

Tools & Ecosystem

Environment variable management tools simplify setting and organizing variables across environments. The direnv tool automatically loads and unloads environment variables based on directory. When entering a project directory, direnv loads variables from .envrc file. Leaving the directory unloads them.

Docker and container platforms inject environment variables through deployment manifests. Docker Compose defines environment sections in docker-compose.yml. Kubernetes uses ConfigMaps and Secrets. Container orchestration platforms provide variable management interfaces.

# docker-compose.yml
services:
  web:
    image: myapp:latest
    environment:
      - DATABASE_URL=postgresql://db:5432/myapp
      - REDIS_URL=redis://redis:6379
      - LOG_LEVEL=debug
    env_file:
      - .env.production

Secret management systems store and control access to sensitive configuration. HashiCorp Vault provides encrypted secret storage with access control policies. AWS Secrets Manager and Google Secret Manager offer cloud-native secret storage. Applications retrieve secrets at runtime through authenticated API calls.

These systems audit secret access, rotate credentials, and enforce least-privilege access. Secrets never appear in configuration files or environment variables visible to unauthorized users.

# Using Vault with vault-ruby gem
require 'vault'

Vault.address = ENV['VAULT_ADDR']
Vault.token = ENV['VAULT_TOKEN']

# Read secret
secret = Vault.logical.read('secret/data/database')
database_password = secret.data[:data][:password]

# Dynamic secrets that auto-rotate
db_creds = Vault.logical.read('database/creds/myapp')
username = db_creds.data[:username]
password = db_creds.data[:password]

Configuration management databases (CMDBs) track configuration across infrastructure and applications. Tools like Chef, Puppet, and Ansible manage server configuration as code. These tools define desired configuration state and enforce it across server fleets.

Infrastructure as Code (IaC) tools provision and configure cloud resources. Terraform defines infrastructure configuration in declarative files. CloudFormation and ARM templates configure cloud services. These tools version infrastructure configuration alongside application configuration.

Parameter store services provide hierarchical configuration storage. AWS Systems Manager Parameter Store and Azure App Configuration centralize configuration with versioning and access control. Applications read parameters through cloud SDK calls.

# AWS Parameter Store
require 'aws-sdk-ssm'

ssm = Aws::SSM::Client.new(region: 'us-east-1')

# Get single parameter
response = ssm.get_parameter(
  name: '/myapp/production/database/password',
  with_decryption: true
)
password = response.parameter.value

# Get parameters by path
response = ssm.get_parameters_by_path(
  path: '/myapp/production/',
  recursive: true,
  with_decryption: true
)

config = response.parameters.each_with_object({}) do |param, hash|
  key = param.name.split('/').last
  hash[key] = param.value
end

Practical Examples

A Rails application separates development and production configuration using environment variables and Rails conventions. Development uses local services while production connects to cloud infrastructure.

# config/database.yml
default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("DB_POOL", 5) %>
  timeout: 5000

development:
  <<: *default
  database: myapp_development
  host: localhost

test:
  <<: *default
  database: myapp_test
  host: localhost

production:
  <<: *default
  url: <%= ENV['DATABASE_URL'] %>
  pool: <%= ENV.fetch("DB_POOL", 25) %>

# config/environments/production.rb
Rails.application.configure do
  config.cache_store = :redis_cache_store, {
    url: ENV['REDIS_URL'],
    expires_in: 90.minutes,
    namespace: 'myapp'
  }
  
  config.action_mailer.delivery_method = :smtp
  config.action_mailer.smtp_settings = {
    address: ENV['SMTP_HOST'],
    port: ENV.fetch('SMTP_PORT', 587),
    user_name: ENV['SMTP_USER'],
    password: ENV['SMTP_PASSWORD'],
    authentication: :plain,
    enable_starttls_auto: true
  }
  
  config.active_storage.service = :amazon
end

# config/storage.yml
amazon:
  service: S3
  access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
  secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
  region: <%= ENV['AWS_REGION'] %>
  bucket: <%= ENV['S3_BUCKET'] %>

A microservices deployment uses centralized configuration with environment-specific overrides. Base configuration provides defaults while deployment manifests inject environment-specific values.

# config/settings.yml (base configuration)
service:
  name: user-service
  port: 8080
  timeout: 30

database:
  pool: 5
  timeout: 5000

cache:
  ttl: 3600
  enabled: true

# Kubernetes ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: user-service-config
data:
  SERVICE_PORT: "8080"
  DATABASE_POOL: "25"
  CACHE_ENABLED: "true"
  CACHE_TTL: "7200"

# Kubernetes Secret
apiVersion: v1
kind: Secret
metadata:
  name: user-service-secrets
type: Opaque
data:
  database-url: cG9zdGdyZXNxbDovL2RiLmV4YW1wbGUuY29tL3VzZXJz
  api-key: c2VjcmV0X2tleV8xMjM0NQ==

# Application reads merged configuration
class ServiceConfig
  def self.load
    base = YAML.load_file('config/settings.yml')
    
    {
      service: {
        name: base['service']['name'],
        port: ENV.fetch('SERVICE_PORT', base['service']['port']).to_i,
        timeout: ENV.fetch('SERVICE_TIMEOUT', base['service']['timeout']).to_i
      },
      database: {
        url: ENV.fetch('DATABASE_URL'),
        pool: ENV.fetch('DATABASE_POOL', base['database']['pool']).to_i,
        timeout: ENV.fetch('DATABASE_TIMEOUT', base['database']['timeout']).to_i
      },
      cache: {
        enabled: ENV.fetch('CACHE_ENABLED', base['cache']['enabled'].to_s) == 'true',
        ttl: ENV.fetch('CACHE_TTL', base['cache']['ttl']).to_i
      },
      api_key: ENV.fetch('API_KEY')
    }
  end
end

Feature flag configuration enables runtime behavior changes without code deployment. Flags control feature rollout, A/B testing, and emergency feature disabling.

class FeatureFlags
  def initialize
    @flags = load_flags
  end

  def enabled?(flag_name, user_id: nil, percentage: nil)
    flag = @flags[flag_name.to_sym]
    return false unless flag
    
    return flag[:enabled] unless percentage
    
    # Percentage-based rollout
    return false unless flag[:enabled]
    hash = Digest::MD5.hexdigest("#{flag_name}:#{user_id}").to_i(16)
    (hash % 100) < percentage
  end

  private

  def load_flags
    {
      new_dashboard: {
        enabled: ENV.fetch('FEATURE_NEW_DASHBOARD', 'false') == 'true'
      },
      payment_v2: {
        enabled: ENV.fetch('FEATURE_PAYMENT_V2', 'false') == 'true'
      },
      beta_features: {
        enabled: ENV.fetch('FEATURE_BETA', 'false') == 'true'
      }
    }
  end
end

flags = FeatureFlags.new

# Simple enable/disable
if flags.enabled?(:new_dashboard)
  render :new_dashboard
else
  render :legacy_dashboard
end

# Percentage rollout
if flags.enabled?(:payment_v2, user_id: current_user.id, percentage: 25)
  process_payment_v2
else
  process_payment_v1
end

Security Implications

Credentials in version control create permanent security vulnerabilities. Committed secrets remain in repository history even after deletion. Attackers scan public repositories for exposed credentials. Never commit API keys, passwords, certificates, or private keys.

Use .gitignore to exclude configuration files containing secrets. Environment-specific files and .env files should never appear in version control.

# .gitignore
.env
.env.local
.env.*.local
config/credentials/*.key
config/master.key
config/secrets.yml

Environment variable exposure through application logs or error messages leaks credentials. Stack traces, debug output, and log messages can print environment contents. Filter sensitive variables from logs and error reports.

# config/initializers/filter_parameters.rb
Rails.application.config.filter_parameters += [
  :password,
  :password_confirmation,
  :api_key,
  :secret_key,
  :access_token,
  :private_key
]

# Custom logger filtering
class SecureLogger
  SENSITIVE_PATTERNS = [
    /password/i,
    /api[_-]?key/i,
    /secret/i,
    /token/i
  ].freeze

  def self.log(message)
    filtered = message.dup
    ENV.each do |key, value|
      if SENSITIVE_PATTERNS.any? { |pattern| key.match?(pattern) }
        filtered.gsub!(value, '[FILTERED]') if value
      end
    end
    Rails.logger.info(filtered)
  end
end

Insufficient access controls on configuration files allow unauthorized reads. Configuration files need restrictive permissions. Secrets require encryption at rest. Cloud provider secret services provide access control through IAM policies.

# Encrypt sensitive configuration with Rails credentials
# Generates config/credentials.yml.enc and config/master.key
rails credentials:edit

# config/credentials.yml.enc (encrypted)
aws:
  access_key_id: AKIAIOSFODNN7EXAMPLE
  secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
stripe:
  secret_key: sk_live_51Example...

# Access in application
Rails.application.credentials.aws[:access_key_id]
Rails.application.credentials.stripe[:secret_key]

# Master key comes from environment
ENV['RAILS_MASTER_KEY'] # In production

Secrets in container images persist after deployment. Building container images with hardcoded secrets embeds them in image layers. Use multi-stage builds and runtime secret injection.

# Bad - secrets in image
FROM ruby:3.2
ENV DATABASE_PASSWORD=secretpassword123

# Good - secrets from environment
FROM ruby:3.2
# No secrets in Dockerfile
# Inject at runtime via Docker/Kubernetes

Default credentials present in configuration files create known vulnerabilities. Change all defaults before production deployment. Validate that production configuration differs from development defaults.

class ConfigValidator
  INSECURE_DEFAULTS = {
    'DATABASE_PASSWORD' => ['password', 'secret', 'admin'],
    'API_KEY' => ['your_api_key_here', 'test_key'],
    'SECRET_KEY_BASE' => ['development_secret']
  }.freeze

  def self.validate!
    INSECURE_DEFAULTS.each do |key, insecure_values|
      value = ENV[key]
      if insecure_values.include?(value)
        raise SecurityError, "Insecure default value for #{key}"
      end
    end
  end
end

# Run at application startup in production
ConfigValidator.validate! if Rails.env.production?

Common Pitfalls

Type confusion occurs when string environment variables receive numeric or boolean interpretation. ENV['PORT'] returns string "3000", not integer 3000. Comparisons and arithmetic require explicit conversion.

# Wrong - string comparison
if ENV['DEBUG'] == true  # Always false
  enable_debugging
end

# Wrong - string arithmetic
max_connections = ENV['MAX_CONNECTIONS'] + 5  # String concatenation "105"

# Correct - explicit conversion
if ENV['DEBUG'] == 'true'
  enable_debugging
end

max_connections = ENV.fetch('MAX_CONNECTIONS', 10).to_i + 5

Missing configuration validation causes runtime failures. Applications start successfully with invalid configuration and fail during request processing. Validate all required configuration at startup.

# Fails at startup with clear error
class Config
  def initialize
    @port = Integer(ENV.fetch('PORT'))
    @database_url = ENV.fetch('DATABASE_URL')
    validate_database_url!
  end

  private

  def validate_database_url!
    uri = URI.parse(@database_url)
    unless uri.scheme == 'postgresql' && uri.host
      raise ConfigError, "Invalid database URL"
    end
  rescue URI::InvalidURIError => e
    raise ConfigError, "Malformed database URL: #{e.message}"
  end
end

Configuration drift occurs when environments diverge over time. Production receives updates that development and staging miss. Document all configuration changes and apply consistently across environments.

Sensitive data in configuration files persists in backups and logs. Encrypted configuration files leave decryption keys vulnerable. Separate secret management from configuration management.

Circular dependencies between configuration and application code prevent initialization. Configuration depends on database schema, but database connection requires configuration. Break circular dependencies through initialization order.

# Bad - circular dependency
class Config
  def database_url
    # Queries database for configuration
    Setting.find_by(key: 'database_url').value
  end
end

# Good - staged initialization
class Config
  def self.bootstrap
    # Load bootstrap configuration from environment
    @database_url = ENV.fetch('DATABASE_URL')
  end

  def self.load_runtime_config
    # Load additional configuration from database after connection established
    @settings = Setting.all.each_with_object({}) do |setting, hash|
      hash[setting.key] = setting.value
    end
  end
end

Config.bootstrap
establish_database_connection(Config.database_url)
Config.load_runtime_config

Overwriting production configuration during deployment destroys working settings. Always backup configuration before deployment. Use configuration version control or immutable configuration.

Reference

Configuration Sources

Environment Variable Patterns

Configuration File Formats

Common Configuration Keys

Security Checklist

Ruby Configuration Gems

Validation Examples

# Required value validation
ENV.fetch('DATABASE_URL')  # Raises if missing

# Type validation
Integer(ENV.fetch('PORT'))  # Raises on non-integer

# Range validation
port = Integer(ENV.fetch('PORT'))
raise ConfigError unless (1..65535).include?(port)

# Format validation
url = ENV.fetch('DATABASE_URL')
raise ConfigError unless url.start_with?('postgresql://')

# Enum validation
log_level = ENV.fetch('LOG_LEVEL', 'info')
valid_levels = %w[debug info warn error fatal]
raise ConfigError unless valid_levels.include?(log_level)