Message Passing

Overview

Message passing represents a fundamental communication paradigm where independent components exchange information through discrete messages. Unlike shared memory approaches where components access common data structures, message passing treats communication as an explicit transfer of data between senders and receivers.

The concept manifests across multiple programming contexts. In object-oriented systems, message passing describes how objects invoke methods on each other. In concurrent programming, processes or threads communicate by exchanging messages through channels or queues. In distributed systems, separate machines coordinate through network messages.

Ruby implements message passing as its core object interaction mechanism. When code calls a method on an object, Ruby interprets this as sending a message to that object. The object receives the message and determines how to respond based on its class and method definitions. This interpretation distinguishes Ruby from languages where method calls represent direct procedure invocations.

Message passing provides several architectural benefits. Components remain loosely coupled because they communicate through defined message interfaces rather than accessing each other's internal state. The sender and receiver can exist in different processes, threads, or machines without changing the message passing code. Systems gain flexibility because components can be replaced or modified without affecting message senders, as long as message interfaces remain stable.

# Message passing in Ruby
calculator = Calculator.new
result = calculator.add(5, 3)  # Sends "add" message with arguments 5, 3
# => 8

Key Principles

Message passing operates on several fundamental principles that distinguish it from other communication mechanisms.

Encapsulation and Independence: Components interact only through messages, never accessing each other's internal state directly. Each component maintains its own private data and exposes functionality through message interfaces. This encapsulation ensures that internal implementation changes do not propagate to message senders.

Asynchronous Potential: Message passing naturally supports asynchronous communication where senders continue execution without waiting for receiver response. The sender places a message in a queue or channel, and the receiver processes it when available. Synchronous message passing also exists, where senders block until receiving a response.

Location Transparency: The message passing abstraction hides whether sender and receiver exist in the same process, different threads, or separate machines. Code sends messages using the same interface regardless of receiver location. The underlying system handles message routing, serialization, and delivery.

Message Structure: Messages contain all information needed for the receiver to process the request. A message typically includes an identifier indicating the requested operation, arguments providing input data, and potentially metadata like sender identity or timestamps.

Dynamic Dispatch: In object-oriented message passing, the receiver determines which method executes based on the message and the receiver's class. Multiple objects can respond to the same message differently through polymorphism. The sender does not control or know which specific method executes.

# Different objects respond to the same message differently
[1, 2, 3].reverse       # Sends "reverse" message to Array
# => [3, 2, 1]

"hello".reverse         # Sends "reverse" message to String
# => "olleh"

Selective Reception: Receivers can choose which messages they handle. Objects define methods for messages they understand and can ignore or reject unknown messages. In concurrent systems, receivers often filter messages based on type or content.

Message Ordering: Some message passing systems guarantee message order (FIFO delivery), while others allow reordering for performance. Order guarantees affect system complexity and performance characteristics.

Ruby Implementation

Ruby implements message passing through its method dispatch mechanism. Every method call translates to a message send operation where Ruby resolves which method to execute based on the receiver's class hierarchy.

Basic Message Sending: The standard method call syntax represents message passing in Ruby. When code calls object.method_name(args), Ruby sends a message named method_name to object with the provided arguments.

class Greeter
  def hello(name)
    "Hello, #{name}!"
  end
end

greeter = Greeter.new
greeter.hello("Alice")  # Message passing through standard syntax
# => "Hello, Alice!"

Explicit Message Sending: Ruby provides the send method for dynamic message dispatch. This allows constructing message names at runtime or calling private methods explicitly.

class Calculator
  def add(a, b)
    a + b
  end
  
  private
  
  def secret_multiply(a, b)
    a * b
  end
end

calc = Calculator.new
calc.send(:add, 10, 5)              # Dynamic message send
# => 15

calc.send(:secret_multiply, 4, 3)   # Bypasses private visibility
# => 12

operation = :add
calc.send(operation, 7, 2)          # Runtime message name construction
# => 9

Public Send: The public_send method provides safer dynamic dispatch by respecting method visibility. It raises an error when attempting to call private or protected methods.

calc = Calculator.new
calc.public_send(:add, 5, 5)                # Works for public methods
# => 10

calc.public_send(:secret_multiply, 2, 3)    # Raises NoMethodError
# NoMethodError: private method `secret_multiply' called

Method Missing: Ruby invokes method_missing when an object receives a message it does not recognize. Classes override this method to handle unknown messages dynamically.

class DynamicHandler
  def method_missing(method_name, *args, &block)
    if method_name.to_s.start_with?('greet_')
      language = method_name.to_s.sub('greet_', '')
      "Greeting in #{language}: #{args.first}"
    else
      super
    end
  end
  
  def respond_to_missing?(method_name, include_private = false)
    method_name.to_s.start_with?('greet_') || super
  end
end

handler = DynamicHandler.new
handler.greet_spanish("Hola")     # Handles unknown message
# => "Greeting in spanish: Hola"

handler.greet_french("Bonjour")
# => "Greeting in french: Bonjour"

Message Introspection: Ruby provides methods to query what messages an object accepts before sending them.

obj = "hello"
obj.respond_to?(:reverse)    # Check if object accepts message
# => true

obj.respond_to?(:fly)
# => false

obj.methods                   # List all messages object accepts
# => [:reverse, :upcase, :downcase, ...]

obj.methods(false)            # Only messages defined by object's class
# => [specific String methods]

Concurrent Message Passing: Ruby supports concurrent message passing through Queue and thread communication.

require 'thread'

message_queue = Queue.new

# Producer thread sends messages
producer = Thread.new do
  5.times do |i|
    message_queue.push("Message #{i}")
    sleep(0.1)
  end
  message_queue.push(:done)
end

# Consumer thread receives messages
consumer = Thread.new do
  loop do
    msg = message_queue.pop
    break if msg == :done
    puts "Received: #{msg}"
  end
end

producer.join
consumer.join

Actor Pattern Implementation: Ruby gems like Celluloid implement actor-based message passing where each actor processes messages sequentially.

# Conceptual actor implementation (simplified)
class Actor
  def initialize
    @mailbox = Queue.new
    @thread = Thread.new { process_messages }
  end
  
  def send_message(msg)
    @mailbox.push(msg)
  end
  
  private
  
  def process_messages
    loop do
      msg = @mailbox.pop
      handle_message(msg)
    end
  end
  
  def handle_message(msg)
    # Subclasses implement message handling
  end
end

Common Patterns

Message passing systems employ several established patterns for organizing communication between components.

Request-Reply Pattern: The sender sends a message and waits for a response from the receiver. This synchronous pattern blocks the sender until the reply arrives. The pattern ensures that senders receive confirmation that receivers processed their requests.

class RequestReplyServer
  def handle_request(request)
    case request[:action]
    when :add
      request[:a] + request[:b]
    when :multiply
      request[:a] * request[:b]
    else
      { error: "Unknown action" }
    end
  end
end

server = RequestReplyServer.new
request = { action: :add, a: 10, b: 5 }
reply = server.handle_request(request)
# => 15

Fire-and-Forget Pattern: The sender dispatches a message without waiting for acknowledgment or response. The sender continues immediately after sending, assuming the receiver will process the message eventually. This asynchronous pattern improves sender throughput but provides no delivery guarantees.

class Logger
  def initialize
    @queue = Queue.new
    @worker = Thread.new { process_logs }
  end
  
  def log(message)
    @queue.push(message)  # Fire and forget
  end
  
  private
  
  def process_logs
    loop do
      msg = @queue.pop
      File.open('app.log', 'a') { |f| f.puts(msg) }
    end
  end
end

logger = Logger.new
logger.log("Event occurred")  # Returns immediately

Publish-Subscribe Pattern: Publishers send messages to topics without knowing subscribers. Subscribers register interest in topics and receive all messages published to those topics. This pattern decouples publishers from subscribers, allowing dynamic subscription changes.

class EventBus
  def initialize
    @subscribers = Hash.new { |h, k| h[k] = [] }
  end
  
  def subscribe(topic, subscriber)
    @subscribers[topic] << subscriber
  end
  
  def publish(topic, message)
    @subscribers[topic].each do |subscriber|
      subscriber.call(message)
    end
  end
end

bus = EventBus.new

# Subscribers register interest
bus.subscribe(:user_created, ->(msg) { puts "Email: #{msg[:email]}" })
bus.subscribe(:user_created, ->(msg) { puts "Log: New user #{msg[:name]}" })

# Publisher sends message
bus.publish(:user_created, { name: "Alice", email: "alice@example.com" })
# Output:
# Email: alice@example.com
# Log: New user Alice

Point-to-Point Pattern: Messages route from one sender to exactly one receiver through a queue. Multiple receivers can consume from the same queue, but each message goes to only one receiver. This pattern distributes work among receiver instances.

class WorkQueue
  def initialize
    @queue = Queue.new
    @workers = []
  end
  
  def add_worker(&block)
    @workers << Thread.new do
      loop do
        task = @queue.pop
        block.call(task)
      end
    end
  end
  
  def enqueue(task)
    @queue.push(task)
  end
end

work_queue = WorkQueue.new

# Add multiple workers
3.times do |i|
  work_queue.add_worker do |task|
    puts "Worker #{i} processing: #{task}"
  end
end

# Enqueue tasks - each processed by one worker
10.times { |i| work_queue.enqueue("Task #{i}") }

Command Pattern: Messages encapsulate operations as objects containing all information needed to execute the operation. Commands support queuing, logging, and undo operations because they represent executable requests as first-class objects.

class Command
  def execute
    raise NotImplementedError
  end
end

class AddCommand < Command
  def initialize(receiver, value)
    @receiver = receiver
    @value = value
  end
  
  def execute
    @receiver.add(@value)
  end
end

class Calculator
  attr_reader :total
  
  def initialize
    @total = 0
  end
  
  def add(value)
    @total += value
  end
end

calc = Calculator.new
command = AddCommand.new(calc, 10)
command.execute
calc.total
# => 10

Message Router Pattern: An intermediary receives messages and routes them to appropriate receivers based on content, type, or routing rules. Routers enable complex message flows without senders knowing receiver locations.

class MessageRouter
  def initialize
    @routes = {}
  end
  
  def register_route(message_type, handler)
    @routes[message_type] = handler
  end
  
  def route(message)
    handler = @routes[message[:type]]
    handler&.call(message)
  end
end

router = MessageRouter.new
router.register_route(:email, ->(msg) { puts "Sending email to #{msg[:to]}" })
router.register_route(:sms, ->(msg) { puts "Sending SMS to #{msg[:phone]}" })

router.route({ type: :email, to: "user@example.com", body: "Hello" })
# Output: Sending email to user@example.com

Message Filter Pattern: Receivers examine messages and decide whether to process, forward, or discard them based on filtering criteria. Filters enable selective message processing without sender awareness.

class MessageFilter
  def initialize(&filter_block)
    @filter = filter_block
    @next_handler = nil
  end
  
  def set_next(handler)
    @next_handler = handler
  end
  
  def handle(message)
    if @filter.call(message)
      @next_handler&.handle(message)
    end
  end
end

priority_filter = MessageFilter.new { |msg| msg[:priority] == :high }
category_filter = MessageFilter.new { |msg| msg[:category] == :important }

priority_filter.set_next(category_filter)

final_handler = Class.new do
  def handle(msg)
    puts "Processing: #{msg[:content]}"
  end
end.new

category_filter.set_next(final_handler)

priority_filter.handle({ priority: :high, category: :important, content: "Urgent task" })
# Output: Processing: Urgent task

priority_filter.handle({ priority: :low, category: :important, content: "Regular task" })
# No output - filtered by priority

Practical Examples

Message passing applies across numerous programming scenarios, from basic object interaction to complex distributed systems.

Dynamic Method Invocation: Applications construct method names dynamically and invoke them using message passing. This pattern appears in framework code that maps external requests to handler methods.

class RestController
  def get_users
    "Fetching users"
  end
  
  def post_users
    "Creating user"
  end
  
  def delete_users
    "Deleting user"
  end
  
  def handle_request(http_method, resource)
    method_name = "#{http_method}_#{resource}".to_sym
    
    if respond_to?(method_name)
      send(method_name)
    else
      "Method not allowed"
    end
  end
end

controller = RestController.new
controller.handle_request(:get, :users)
# => "Fetching users"

controller.handle_request(:post, :users)
# => "Creating user"

controller.handle_request(:patch, :users)
# => "Method not allowed"

Building a DSL with Method Missing: Domain-specific languages use message passing to create expressive APIs where method names correspond to domain concepts.

class QueryBuilder
  def initialize(table)
    @table = table
    @conditions = []
    @limit_value = nil
  end
  
  def where(condition)
    @conditions << condition
    self
  end
  
  def limit(value)
    @limit_value = value
    self
  end
  
  def to_sql
    sql = "SELECT * FROM #{@table}"
    sql += " WHERE #{@conditions.join(' AND ')}" unless @conditions.empty?
    sql += " LIMIT #{@limit_value}" if @limit_value
    sql
  end
  
  def method_missing(method_name, *args)
    if method_name.to_s.start_with?('where_')
      column = method_name.to_s.sub('where_', '')
      where("#{column} = '#{args.first}'")
    else
      super
    end
  end
  
  def respond_to_missing?(method_name, include_private = false)
    method_name.to_s.start_with?('where_') || super
  end
end

query = QueryBuilder.new('users')
  .where_email('alice@example.com')
  .where_status('active')
  .limit(10)

query.to_sql
# => "SELECT * FROM users WHERE email = 'alice@example.com' AND status = 'active' LIMIT 10"

Background Job Processing: Applications use message queues to offload time-consuming tasks to background workers, improving response times for web requests.

class JobQueue
  def initialize
    @queue = Queue.new
    @workers = []
    @running = true
  end
  
  def start_workers(count = 3)
    count.times do
      @workers << Thread.new { worker_loop }
    end
  end
  
  def enqueue(job)
    @queue.push(job)
  end
  
  def stop
    @running = false
    @workers.each { |w| w.join }
  end
  
  private
  
  def worker_loop
    while @running
      begin
        job = @queue.pop(true)  # Non-blocking
        process_job(job)
      rescue ThreadError
        sleep(0.1)
      end
    end
  end
  
  def process_job(job)
    job.call
  end
end

# Usage
job_queue = JobQueue.new
job_queue.start_workers(3)

# Enqueue tasks
job_queue.enqueue(-> { puts "Sending email"; sleep(1) })
job_queue.enqueue(-> { puts "Processing image"; sleep(2) })
job_queue.enqueue(-> { puts "Generating report"; sleep(1.5) })

sleep(4)
job_queue.stop

Event-Driven Architecture: Systems coordinate through events, with components publishing events when state changes and other components subscribing to relevant events.

class EventManager
  def initialize
    @listeners = Hash.new { |h, k| h[k] = [] }
  end
  
  def on(event_name, &handler)
    @listeners[event_name] << handler
  end
  
  def trigger(event_name, data = {})
    @listeners[event_name].each do |handler|
      handler.call(data)
    end
  end
end

class OrderSystem
  attr_reader :events
  
  def initialize
    @events = EventManager.new
  end
  
  def create_order(order_data)
    order = { id: rand(1000), **order_data }
    events.trigger(:order_created, order)
    order
  end
end

# Setup system and listeners
order_system = OrderSystem.new

order_system.events.on(:order_created) do |order|
  puts "Inventory: Reserving items for order #{order[:id]}"
end

order_system.events.on(:order_created) do |order|
  puts "Email: Sending confirmation to #{order[:customer_email]}"
end

order_system.events.on(:order_created) do |order|
  puts "Analytics: Recording order event"
end

# Create order triggers all listeners
order_system.create_order(
  customer_email: "customer@example.com",
  items: ["Widget A", "Gadget B"]
)
# Output:
# Inventory: Reserving items for order 742
# Email: Sending confirmation to customer@example.com
# Analytics: Recording order event

Inter-Thread Communication: Threads coordinate through message passing using queues and channels, avoiding shared state and race conditions.

class Pipeline
  def initialize
    @stages = []
  end
  
  def add_stage(&processor)
    input_queue = Queue.new
    output_queue = Queue.new
    
    thread = Thread.new do
      loop do
        item = input_queue.pop
        break if item == :done
        result = processor.call(item)
        output_queue.push(result)
      end
      output_queue.push(:done)
    end
    
    @stages << { input: input_queue, output: output_queue, thread: thread }
    self
  end
  
  def process(items)
    results = []
    
    # Start pipeline
    items.each { |item| @stages.first[:input].push(item) }
    @stages.first[:input].push(:done)
    
    # Connect stages
    @stages.each_cons(2) do |current, next_stage|
      Thread.new do
        loop do
          item = current[:output].pop
          break if item == :done
          next_stage[:input].push(item)
        end
        next_stage[:input].push(:done)
      end
    end
    
    # Collect results
    loop do
      result = @stages.last[:output].pop
      break if result == :done
      results << result
    end
    
    @stages.each { |s| s[:thread].join }
    results
  end
end

pipeline = Pipeline.new
  .add_stage { |x| x * 2 }
  .add_stage { |x| x + 10 }
  .add_stage { |x| x.to_s }

results = pipeline.process([1, 2, 3, 4, 5])
# => ["12", "14", "16", "18", "20"]

Design Considerations

Selecting appropriate message passing approaches requires evaluating several design dimensions based on system requirements and constraints.

Synchronous vs Asynchronous Communication: Synchronous message passing blocks senders until receivers respond. This simplifies error handling and maintains request-response ordering but limits throughput and couples sender execution to receiver availability. Asynchronous message passing allows senders to continue immediately, improving throughput and resilience to receiver failures, but complicates error handling and ordering guarantees.

Choose synchronous messaging when immediate feedback is required, transactions span sender and receiver, or message ordering must be preserved. Choose asynchronous messaging for fire-and-forget operations, when sender throughput matters more than immediate confirmation, or when receivers might be temporarily unavailable.

Direct vs Indirect Communication: Direct message passing sends messages straight to receivers, minimizing latency and simplifying debugging. Indirect message passing routes messages through intermediaries like queues or message brokers, adding latency but enabling sender-receiver decoupling, load balancing, and message persistence.

Direct communication works well for simple systems with static topologies and low message volumes. Indirect communication through message brokers suits systems needing dynamic scaling, multiple receiver instances, message replay capabilities, or guaranteed delivery.

Message Delivery Guarantees: Systems provide different delivery semantics. At-most-once delivery sends messages without confirmation, offering low overhead but no delivery guarantees. At-least-once delivery retries until confirmed, ensuring delivery but potentially causing duplicates. Exactly-once delivery guarantees single processing, requiring complex coordination.

At-most-once delivery suffices for non-critical notifications or monitoring data. At-least-once delivery works when operations are idempotent or duplicate detection is possible. Exactly-once delivery is necessary for financial transactions or critical state changes where duplicates cause problems.

Message Format: Structured message formats like JSON or Protocol Buffers enable schema validation and cross-language interoperability but add serialization overhead. Custom formats minimize size and parsing cost but require maintaining serialization code.

# Structured message format
message = {
  type: "user_action",
  timestamp: Time.now.to_i,
  user_id: 12345,
  action: "purchase",
  metadata: { item_id: 67890, price: 29.99 }
}

# Compact custom format (string encoding)
compact_msg = "UA|#{Time.now.to_i}|12345|purchase|67890|29.99"

Error Handling Strategy: Message passing systems must decide how to handle failures. Immediate error propagation sends error responses back to senders, requiring sender error handling. Dead letter queues capture failed messages for later analysis. Retry mechanisms attempt redelivery with backoff strategies.

Error handling complexity increases with system distribution. Local message passing within a process can use exceptions. Distributed message passing requires explicit error messages or status codes.

Message Ordering Requirements: Some systems require strict message ordering while others allow reordering for performance. FIFO ordering guarantees messages process in send order but limits parallelization. Unordered delivery enables parallel processing but complicates dependent operations.

Maintain ordering for stateful operations where later messages depend on earlier ones. Allow reordering when messages represent independent events or when receivers can handle out-of-order delivery.

Coupling vs Flexibility Trade-off: Tight message coupling provides compile-time safety and IDE support but reduces flexibility. Loose coupling enables dynamic message routing and runtime flexibility but loses type safety and makes errors discoverable only at runtime.

# Tight coupling - explicit method call
class DirectNotifier
  def notify(user, message)
    user.send_email(message)  # Directly coupled to User#send_email
  end
end

# Loose coupling - message passing
class FlexibleNotifier
  def notify(recipient, message)
    recipient.send(:deliver, message)  # Works with any object responding to deliver
  end
end

State Management: Message passing systems must decide whether receivers maintain state or operate statelessly. Stateful receivers remember previous messages, enabling complex workflows but complicating recovery and scaling. Stateless receivers treat each message independently, simplifying scaling and recovery but requiring messages to contain complete context.

Performance Considerations

Message passing performance characteristics vary significantly based on implementation approach and system architecture.

Message Serialization Overhead: Converting objects to messages and back adds CPU cost and latency. Simple string or numeric messages serialize quickly. Complex object graphs require traversal and encoding. Binary formats like MessagePack or Protocol Buffers reduce serialization time and message size compared to JSON or YAML.

require 'json'
require 'benchmark'

data = {
  user_id: 12345,
  name: "Alice Johnson",
  orders: Array.new(100) { { id: rand(1000), total: rand(100.0) } }
}

Benchmark.bm do |x|
  x.report("JSON:") { 10_000.times { JSON.generate(data); JSON.parse(JSON.generate(data)) } }
  x.report("Marshal:") { 10_000.times { Marshal.dump(data); Marshal.load(Marshal.dump(data)) } }
end

Queue Operation Cost: Queue operations (push/pop) introduce synchronization overhead in concurrent systems. Lock-free queues reduce contention but increase implementation complexity. Queue size affects memory usage and can cause backpressure when producers outpace consumers.

Message Granularity: Fine-grained messages (one message per operation) increase message count and overhead but enable parallel processing. Coarse-grained messages (batching multiple operations) reduce overhead but limit parallelism and increase latency for individual operations.

# Fine-grained messages
orders.each do |order|
  message_queue.push({ type: :process_order, order: order })
end

# Coarse-grained messages (batching)
orders.each_slice(100) do |batch|
  message_queue.push({ type: :process_order_batch, orders: batch })
end

Synchronization Cost: Synchronous message passing incurs context switching and blocking costs. Each synchronous message send requires thread synchronization and potentially thread parking/unparking. Asynchronous message passing amortizes synchronization costs across message batches.

Memory Allocation: Each message typically requires memory allocation for the message object and its payload. High message rates stress garbage collection. Object pooling reuses message objects to reduce allocation pressure.

class MessagePool
  def initialize(size = 100)
    @pool = Array.new(size) { Message.new }
    @available = Queue.new
    @pool.each { |msg| @available.push(msg) }
  end
  
  def acquire
    @available.pop
  end
  
  def release(message)
    message.reset
    @available.push(message)
  end
end

class Message
  attr_accessor :type, :payload
  
  def reset
    @type = nil
    @payload = nil
  end
end

Network Latency: Distributed message passing adds network round-trip time. Local process or thread communication operates in microseconds while network messages take milliseconds. Message batching amortizes network overhead across multiple logical messages.

Message Broker Performance: Using message brokers introduces additional hops and processing. Brokers provide persistence, routing, and delivery guarantees at the cost of increased latency and resource requirements. Direct messaging eliminates broker overhead but loses broker capabilities.

Backpressure Handling: When receivers process messages slower than senders produce them, queues grow unbounded. Bounded queues block or reject senders when full, creating backpressure. Queue size affects memory usage and latency—larger queues increase memory but smooth traffic bursts.

class BoundedQueue
  def initialize(max_size)
    @queue = Queue.new
    @max_size = max_size
    @mutex = Mutex.new
  end
  
  def push(item, timeout = nil)
    deadline = timeout ? Time.now + timeout : nil
    
    loop do
      @mutex.synchronize do
        if @queue.size < @max_size
          @queue.push(item)
          return true
        end
      end
      
      return false if deadline && Time.now >= deadline
      sleep(0.01)
    end
  end
  
  def pop
    @queue.pop
  end
end

Method Dispatch Overhead: Ruby's dynamic method dispatch adds overhead compared to static method calls. The send method incurs additional cost over direct method calls. Method caching reduces repeated lookup costs but cannot eliminate all overhead.

# Performance comparison
require 'benchmark'

class Calculator
  def add(a, b)
    a + b
  end
end

calc = Calculator.new

Benchmark.bm do |x|
  x.report("Direct call:") { 1_000_000.times { calc.add(5, 3) } }
  x.report("Send:") { 1_000_000.times { calc.send(:add, 5, 3) } }
end

Reference

Message Passing Approaches

Ruby Message Passing Methods

Message Delivery Semantics

Common Message Patterns

Performance Factors

Error Handling Strategies

Design Decision Matrix