Inter-Process Communication

Overview

Inter-Process Communication (IPC) refers to the mechanisms that operating systems provide for processes to manage shared data and coordinate their execution. Unlike threads within the same process that share memory space, processes operate in isolated memory spaces. IPC mechanisms bridge this isolation, enabling processes to exchange data, synchronize actions, and coordinate complex workflows.

IPC forms the foundation of modern software architecture. Web servers handle requests across multiple processes, database systems coordinate between query executors and storage managers, and microservices architectures depend entirely on process-to-process communication. Operating systems implement IPC through various mechanisms, each with distinct characteristics suited to different communication patterns.

The choice of IPC mechanism affects system architecture, performance, and reliability. A message queue provides different guarantees than shared memory, and a Unix socket offers different capabilities than a named pipe. Understanding these differences enables informed architectural decisions.

Ruby provides access to most IPC mechanisms through its standard library and native extensions. While Ruby's Global Interpreter Lock affects threading, it does not restrict process-based concurrency, making IPC particularly relevant for Ruby applications requiring parallelism.

# Basic process creation establishes the foundation for IPC
pid = fork do
  puts "Child process: #{Process.pid}"
  sleep 2
end

puts "Parent process: #{Process.pid}"
Process.wait(pid)
# Parent process: 12345
# Child process: 12346

Key Principles

IPC mechanisms operate through several fundamental approaches. The operating system mediates all inter-process communication, enforcing isolation and access control. Processes cannot directly access each other's memory, so the OS provides controlled channels for data transfer.

Message Passing transfers data explicitly from one process to another. The sender packages data into a message and invokes an OS service to transmit it. The receiver invokes a corresponding service to retrieve the message. This approach makes data flow explicit and facilitates reasoning about system behavior. Pipes, sockets, and message queues implement message passing with different characteristics.

Shared Memory allows multiple processes to map the same physical memory region into their address spaces. Processes can then read and write shared data structures directly without OS mediation for each operation. This provides the highest performance but requires explicit synchronization to prevent race conditions. The operating system manages the mapping but does not control individual reads and writes.

Synchronization Primitives coordinate process execution without transferring data. Semaphores, mutexes, and condition variables allow processes to signal each other and coordinate access to shared resources. These mechanisms prevent race conditions and deadlocks in concurrent systems.

Remote Procedure Calls abstract communication as procedure invocations. One process calls a function that appears local but executes in another process. The RPC system handles marshaling arguments, transmitting the request, executing the remote procedure, and returning results. This pattern simplifies distributed system programming.

Process identity and isolation determine IPC security. Operating systems assign each process a user ID and group ID, enforcing permissions on IPC resources. A process can only access IPC objects it has permission to use, preventing unauthorized communication between processes.

Data serialization becomes necessary when processes communicate across machine boundaries or between different language runtimes. The sending process must convert data structures into a byte stream, and the receiving process must reconstruct them. Format choices affect performance, compatibility, and maintainability.

# Pipes demonstrate unidirectional message passing
reader, writer = IO.pipe

pid = fork do
  writer.close  # Child closes write end
  data = reader.read
  puts "Child received: #{data}"
  reader.close
end

reader.close  # Parent closes read end
writer.puts "Message from parent"
writer.close

Process.wait(pid)
# Child received: Message from parent

Process lifetime management affects IPC reliability. When a process terminates unexpectedly, the OS typically closes its open IPC resources. Other processes must detect and handle this condition. Proper error handling distinguishes robust IPC implementations from fragile ones.

Implementation Approaches

Pipes provide unidirectional byte streams between processes with a parent-child relationship. Anonymous pipes exist only while both ends remain open, making them suitable for communication between processes created through forking. Named pipes (FIFOs) persist in the filesystem, allowing unrelated processes to connect.

Pipes buffer data in kernel memory, blocking writers when the buffer fills and blocking readers when empty. This flow control prevents fast producers from overwhelming slow consumers. The kernel manages buffering automatically, but buffer size limits affect achievable throughput.

Unix Domain Sockets extend pipe functionality with bidirectional communication and more flexible connection patterns. Processes establish socket connections through filesystem paths, supporting both stream-oriented (TCP-like) and datagram (UDP-like) communication. Unix domain sockets outperform TCP sockets for local communication by eliminating network protocol overhead.

TCP/IP Sockets enable communication between processes on different machines or between processes that need network protocol features. The socket API provides a uniform interface for both local and remote communication. Connection-oriented TCP sockets provide reliable, ordered delivery, while connectionless UDP sockets offer lower latency at the cost of reliability guarantees.

Message Queues decouple senders and receivers temporally. A sender places a message in the queue and continues execution without waiting for a receiver. Multiple receivers can consume messages from the same queue, enabling load distribution. The queue persists messages even when no receivers are active, providing durability.

System V message queues integrate with traditional Unix IPC facilities, while POSIX message queues offer a more modern interface. Message queues provide message boundaries—each send operation produces a discrete message that arrives intact at the receiver. This contrasts with byte stream abstractions where message framing requires application-level protocols.

Shared Memory offers the highest performance by eliminating data copying. After mapping shared memory, processes access data with normal memory operations. However, processes must implement synchronization explicitly to prevent race conditions. Semaphores or memory-mapped mutexes coordinate access.

POSIX shared memory objects use filesystem paths for naming, while System V shared memory uses integer keys. Both approaches provide memory segments that multiple processes can map. Changes written by one process become immediately visible to others sharing the region.

Memory-Mapped Files extend shared memory concepts to persistent storage. Processes map a file into their address space and access it with memory operations. The operating system handles synchronizing memory contents with disk storage. This approach simplifies file I/O and enables efficient data sharing through persistent storage.

Signals provide asynchronous notifications between processes. The kernel delivers signals to processes based on events like timer expiration, terminal interrupts, or explicit kill commands. Signal handlers execute asynchronously, complicating reasoning about program state. Signals work best for simple notifications rather than complex communication.

Ruby Implementation

Ruby exposes IPC mechanisms through the IO, Socket, and Process classes in the standard library. The fork method creates child processes, establishing the foundation for process-based parallelism.

Pipes in Ruby use IO.pipe, which returns a connected reader and writer. After forking, processes close the unused end and communicate through the remaining end. Ruby's IO objects provide buffering and encoding conversion automatically.

def pipe_communication(data)
  reader, writer = IO.pipe
  
  pid = fork do
    writer.close
    result = reader.read.upcase
    puts "Transformed: #{result}"
    reader.close
    exit!(0)
  end
  
  reader.close
  writer.write(data)
  writer.close
  Process.wait(pid)
end

pipe_communication("hello world")
# Transformed: HELLO WORLD

Unix Sockets through UNIXSocket and UNIXServer classes provide connection-oriented communication. The server creates a listening socket bound to a filesystem path, and clients connect to that path. Data transfer works identically to network sockets.

require 'socket'

# Server process
server_path = '/tmp/ruby_ipc.sock'
File.delete(server_path) if File.exist?(server_path)

server_pid = fork do
  server = UNIXServer.new(server_path)
  client = server.accept
  
  data = client.gets
  response = "Processed: #{data.strip}"
  client.puts(response)
  
  client.close
  server.close
end

sleep 0.1  # Allow server to start

# Client process
client = UNIXSocket.new(server_path)
client.puts("test data")
result = client.gets
puts result
client.close

Process.wait(server_pid)
File.delete(server_path)
# Processed: test data

TCP Sockets use TCPSocket and TCPServer classes. The API matches Unix sockets but operates over network protocols. Ruby handles host resolution, connection establishment, and protocol details.

require 'socket'

server_pid = fork do
  server = TCPServer.new('localhost', 9999)
  client = server.accept
  
  message = client.read
  client.write("Echo: #{message}")
  
  client.close
  server.close
end

sleep 0.1

client = TCPSocket.new('localhost', 9999)
client.write("network message")
client.close_write

response = client.read
puts response
client.close

Process.wait(server_pid)
# Echo: network message

DRb (Distributed Ruby) provides object-oriented remote procedure calls. Objects in one Ruby process become accessible to other Ruby processes. DRb handles serialization, network communication, and method dispatch transparently.

require 'drb'

# Server object
class Calculator
  def add(a, b)
    a + b
  end
  
  def multiply(a, b)
    a * b
  end
end

server_pid = fork do
  DRb.start_service('druby://localhost:8787', Calculator.new)
  DRb.thread.join
end

sleep 0.5

# Client access
DRb.start_service
calculator = DRbObject.new_nil('druby://localhost:8787')

result1 = calculator.add(5, 3)
result2 = calculator.multiply(4, 7)

puts "Add: #{result1}, Multiply: #{result2}"
# Add: 8, Multiply: 28

Process.kill('TERM', server_pid)
Process.wait(server_pid)

IO.popen combines process creation with pipe communication. It executes a command in a child process and returns an IO object connected to its standard input, output, or both. This simplifies interaction with external programs.

# Read from command output
IO.popen(['echo', 'test']) do |io|
  output = io.read
  puts "Output: #{output}"
end
# Output: test

# Write to command input and read output
IO.popen('bc', 'r+') do |calc|
  calc.puts('5 + 3')
  calc.close_write
  result = calc.read
  puts "Calculation: #{result}"
end
# Calculation: 8

Process Communication with Marshal serializes Ruby objects for IPC. Marshal converts objects to byte streams that can transmit through any IPC mechanism, then reconstructs them in the receiving process.

reader, writer = IO.pipe

pid = fork do
  writer.close
  data = Marshal.load(reader)
  puts "Received: #{data.inspect}"
  reader.close
end

reader.close
complex_data = {name: 'test', values: [1, 2, 3], time: Time.now}
Marshal.dump(complex_data, writer)
writer.close

Process.wait(pid)
# Received: {:name=>"test", :values=>[1, 2, 3], time=>2025-10-07 14:32:10...}

Practical Examples

Worker Pool with Pipes distributes work across multiple processes. The parent process sends tasks through pipes to worker children, which return results. This pattern enables parallelism in CPU-bound Ruby applications.

def worker_pool(tasks, worker_count)
  workers = worker_count.times.map do
    task_reader, task_writer = IO.pipe
    result_reader, result_writer = IO.pipe
    
    pid = fork do
      task_writer.close
      result_reader.close
      
      while task_data = task_reader.gets
        task = Marshal.load(task_data)
        result = yield(task)  # Process task
        Marshal.dump(result, result_writer)
        result_writer.flush
      end
      
      task_reader.close
      result_writer.close
    end
    
    task_reader.close
    result_writer.close
    
    {pid: pid, task_writer: task_writer, result_reader: result_reader}
  end
  
  results = []
  tasks.each_with_index do |task, index|
    worker = workers[index % worker_count]
    Marshal.dump(task, worker[:task_writer])
    worker[:task_writer].flush
    results << Marshal.load(worker[:result_reader])
  end
  
  workers.each do |worker|
    worker[:task_writer].close
    worker[:result_reader].close
    Process.wait(worker[:pid])
  end
  
  results
end

tasks = (1..10).to_a
results = worker_pool(tasks, 3) do |n|
  n * n  # Square each number
end

puts "Results: #{results.inspect}"
# Results: [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Request-Reply Pattern with Unix Sockets implements a service that processes requests from multiple clients. The server handles each connection in sequence or spawns child processes for concurrent handling.

require 'socket'
require 'json'

def run_server(socket_path)
  File.delete(socket_path) if File.exist?(socket_path)
  server = UNIXServer.new(socket_path)
  
  fork do
    loop do
      client = server.accept
      
      request = JSON.parse(client.gets)
      response = {
        status: 'ok',
        result: process_request(request),
        timestamp: Time.now.to_i
      }
      
      client.puts(JSON.generate(response))
      client.close
    end
  end
end

def process_request(request)
  case request['action']
  when 'reverse'
    request['data'].reverse
  when 'upcase'
    request['data'].upcase
  else
    'unknown action'
  end
end

def send_request(socket_path, action, data)
  client = UNIXSocket.new(socket_path)
  
  request = {action: action, data: data}
  client.puts(JSON.generate(request))
  
  response = JSON.parse(client.gets)
  client.close
  
  response
end

socket_path = '/tmp/service.sock'
server_pid = run_server(socket_path)
sleep 0.1

response1 = send_request(socket_path, 'reverse', 'hello')
response2 = send_request(socket_path, 'upcase', 'world')

puts "Response 1: #{response1['result']}"
puts "Response 2: #{response2['result']}"
# Response 1: olleh
# Response 2: WORLD

Process.kill('TERM', server_pid)
Process.wait(server_pid)
File.delete(socket_path)

Pub-Sub with Named Pipes implements a simple message broadcast system. Multiple subscribers open a named pipe for reading, and a publisher writes messages that all subscribers receive.

require 'thread'

def create_named_pipe(path)
  system("mkfifo #{path}") unless File.exist?(path)
end

def publisher(pipe_path, messages)
  messages.each do |msg|
    # Open for each write to avoid blocking
    File.open(pipe_path, 'w') do |pipe|
      pipe.puts(msg)
    end
    sleep 0.1
  end
end

def subscriber(pipe_path, id)
  fork do
    File.open(pipe_path, 'r') do |pipe|
      while line = pipe.gets
        puts "Subscriber #{id} received: #{line.strip}"
      end
    end
  end
end

pipe_path = '/tmp/pubsub_pipe'
create_named_pipe(pipe_path)

# Start subscribers
sub1 = subscriber(pipe_path, 1)
sub2 = subscriber(pipe_path, 2)

sleep 0.5

# Publish messages
publisher(pipe_path, ['message 1', 'message 2', 'message 3'])

sleep 1

Process.kill('TERM', sub1)
Process.kill('TERM', sub2)
Process.wait(sub1)
Process.wait(sub2)
File.delete(pipe_path)

Coordinated Shutdown with Signals manages graceful process termination. Parent processes signal children to shut down cleanly, allowing them to finish current operations and release resources.

def worker_with_shutdown
  shutdown = false
  
  trap('TERM') do
    shutdown = true
  end
  
  fork do
    counter = 0
    until shutdown
      counter += 1
      sleep 0.5
    end
    puts "Worker shutting down after #{counter} iterations"
  end
end

workers = 3.times.map { worker_with_shutdown }

sleep 2

puts "Initiating shutdown"
workers.each { |pid| Process.kill('TERM', pid) }
workers.each { |pid| Process.wait(pid) }

puts "All workers terminated"
# Worker shutting down after 4 iterations
# Worker shutting down after 4 iterations
# Worker shutting down after 4 iterations
# All workers terminated

Performance Considerations

IPC mechanism selection directly impacts throughput and latency. Shared memory provides the fastest data transfer by eliminating copying, while message passing mechanisms incur overhead from kernel mediation and buffer management.

Shared Memory achieves the lowest latency for large data transfers. After initial mapping overhead, processes access shared data at memory speed. A 1MB shared memory region allows processes to exchange data in microseconds. However, coordination overhead from synchronization primitives can dominate for small transfers.

Unix Domain Sockets outperform TCP sockets for local communication by 30-50% in typical scenarios. The kernel optimizes Unix socket transfers by avoiding network stack processing. For applications communicating on the same machine, Unix sockets should be the default choice over TCP.

Pipes perform well for unidirectional data flows. Kernel buffering reduces system call overhead, and the simple unidirectional model enables optimizations. However, pipe buffers are limited—typically 64KB on Linux—causing writers to block when receivers fall behind.

System call frequency dominates IPC overhead. Each send or receive operation invokes the kernel, incurring context switch costs. Batching multiple messages into larger transfers amortizes this overhead. An application sending 1000 small messages should buffer them into fewer larger sends.

# Inefficient: many small writes
1000.times do |i|
  socket.write("message #{i}\n")
end

# Efficient: batch into larger write
buffer = 1000.times.map { |i| "message #{i}\n" }.join
socket.write(buffer)

Serialization overhead affects high-throughput applications. Marshal provides convenience but adds CPU cost for encoding and decoding. Binary protocols like MessagePack reduce serialization overhead compared to text formats like JSON. For maximum performance, custom binary protocols eliminate unnecessary encoding.

Process creation cost impacts IPC architecture decisions. Forking a Ruby process takes milliseconds, amortizing over long-lived workers but prohibitive for short-lived tasks. Pre-forking server models create worker processes during initialization, avoiding fork overhead per request.

Buffer management affects memory efficiency. Message queues and socket buffers consume kernel memory, limiting concurrent connections. Applications handling thousands of connections must manage buffer sizes carefully to avoid memory exhaustion.

Copy elimination improves throughput for large data transfers. Unix socket credential passing allows sending file descriptors between processes, enabling zero-copy data sharing. Instead of reading data and sending it through a socket, a process can send the file descriptor directly.

require 'socket'

# Send file descriptor through Unix socket
def send_fd(socket, file)
  socket.sendmsg('', 0, nil, [:SOCKET, :RIGHTS, file.fileno])
end

def receive_fd(socket)
  msg, _, _, cred = socket.recvmsg(1)
  fd = cred.unix_rights[0]
  IO.new(fd)
end

Contention reduction in shared memory systems requires careful synchronization design. Fine-grained locking allows higher concurrency than coarse-grained locks but increases complexity. Lock-free algorithms using atomic operations eliminate lock contention entirely for appropriate use cases.

Security Implications

IPC mechanisms bypass network security controls, requiring explicit protection. Processes on the same machine can communicate through Unix sockets or shared memory without firewall mediation. Applications must implement authentication and authorization at the IPC layer.

Unix Socket Permissions control access through filesystem permissions. Creating a Unix socket with restrictive permissions (0600) limits connections to the socket owner. However, this provides coarse-grained control—either full access or no access.

require 'socket'

socket_path = '/tmp/secure_socket.sock'
File.delete(socket_path) if File.exist?(socket_path)

server = UNIXServer.new(socket_path)
File.chmod(0600, socket_path)  # Owner only

# Socket now accessible only to creating user

Credential Passing through Unix sockets enables authentication. The operating system provides the connecting process's user ID, group ID, and process ID to the receiving process. This allows servers to verify client identity and enforce access control.

require 'socket'

def get_peer_credentials(socket)
  cred = socket.getsockopt(:SOCKET, :PEERCRED)
  # Returns struct with pid, uid, gid
  pid, uid, gid = cred.unpack('III')
  {pid: pid, uid: uid, gid: gid}
end

Input Validation remains critical despite process isolation. Malicious or compromised processes can send crafted data through IPC channels. Applications must validate all received data before use, checking types, ranges, and formats. Trusting IPC data based on source process identity alone leads to vulnerabilities.

Serialization formats affect attack surface. Marshal deserializes arbitrary Ruby objects, enabling code execution if attackers control serialized data. JSON or MessagePack restrict deserialization to simple data types, reducing exploit opportunities. Custom binary protocols with explicit type specifications provide the most control.

Denial of Service through IPC resource exhaustion requires rate limiting and resource quotas. A malicious process can flood Unix sockets with connection attempts or fill message queues with bogus messages. Servers must limit concurrent connections, message rates, and queue depths.

Shared memory segments persist after process termination unless explicitly removed. Abandoned shared memory consumes system resources and may contain sensitive data. Applications must implement cleanup procedures, ideally using POSIX shared memory with O_EXCL to detect reuse attempts.

Timing Attacks can leak information through IPC response times. Authentication checks that return immediately for invalid users but slowly for valid users reveal user existence. Constant-time comparison algorithms prevent these leaks.

Race conditions in IPC setup enable attacks. Creating a Unix socket in a shared directory allows attackers to create that path first, capturing connections intended for the legitimate server. Applications should create sockets in protected directories or use abstract socket names that exist only in the kernel.

Common Pitfalls

Forgetting to Close File Descriptors in child processes causes resource leaks. After forking, children inherit copies of all parent file descriptors. Unused descriptors must be closed explicitly to prevent exhaustion.

# Incorrect: descriptors leak in child
reader, writer = IO.pipe
fork do
  data = reader.read  # Writer still open
  puts data
end

# Correct: close unused end
reader, writer = IO.pipe
fork do
  writer.close
  data = reader.read
  reader.close
  puts data
end

Buffering Issues cause apparent data loss or delays. Ruby IO objects buffer data by default. Without explicit flushing, data may remain in buffers instead of transmitting. This particularly affects line-oriented protocols expecting immediate sends.

# Data may not transmit immediately
socket.puts("message")

# Force immediate transmission
socket.puts("message")
socket.flush

Deadlock from Circular Waits occurs when processes wait for each other. Process A writes to B and waits for a response, while B writes to A and waits for a response. Neither can proceed because both are blocked writing. Implementing non-blocking I/O or careful protocol design prevents this.

Zombie Processes accumulate when parents don't wait for children. After a child exits, it remains as a zombie until the parent calls Process.wait. Long-running parents must wait for all children to prevent zombie accumulation.

# Incorrect: zombies accumulate
100.times do
  fork { exit }
end
sleep 5
# 100 zombie processes exist

# Correct: wait for children
pids = 100.times.map { fork { exit } }
pids.each { |pid| Process.wait(pid) }

Marshal Incompatibility between Ruby versions breaks IPC. Marshal format changes across Ruby versions, causing deserialization failures when communicating between processes running different Ruby versions. Text-based formats like JSON provide better cross-version compatibility.

Race Conditions in File-based IPC occur without proper locking. Multiple processes accessing the same file-based resource (named pipes, Unix sockets) can conflict during creation or deletion. Using advisory locks or atomic operations prevents races.

Assuming Atomicity of network operations leads to bugs. Socket reads and writes are not atomic for multi-byte messages. A 1000-byte write may transmit in multiple segments, and reads may receive partial messages. Applications must implement framing protocols or use length prefixes.

# Incorrect: assumes complete transmission
message = "x" * 10000
socket.write(message)

# Correct: ensure complete transmission
message = "x" * 10000
total = 0
while total < message.length
  sent = socket.write(message[total..-1])
  total += sent
end

Ignoring EINTR from system calls causes failures. Signal handlers can interrupt system calls, returning EINTR. Ruby generally retries automatically, but explicit system call wrappers must handle this condition.

Buffer Overflows from unbounded reads exhaust memory. Reading from an untrusted source without size limits allows attackers to consume all memory. Always impose maximum message sizes and enforce them during reads.

Tools & Ecosystem

Redis provides advanced IPC patterns through its data structure server. Multiple processes communicate by reading and writing Redis keys, with atomic operations ensuring consistency. Redis pub/sub implements broadcast messaging, and lists enable job queues.

require 'redis'

redis = Redis.new

# Simple queue pattern
redis.lpush('jobs', 'task_1')
redis.lpush('jobs', 'task_2')

job = redis.brpop('jobs', timeout: 5)
puts "Processing: #{job[1]}"  # task_1

RabbitMQ and ActiveMQ implement AMQP message queuing with reliable delivery guarantees. The Bunny gem provides Ruby AMQP client functionality, enabling processes to exchange messages through durable queues that survive restarts.

dRuby (DRb) in Ruby standard library enables transparent remote object access. Objects in one Ruby process become method-callable from other processes, with DRb handling network communication and serialization automatically.

Sidekiq and Resque implement background job processing through Redis-backed queues. Web processes push jobs into queues, and worker processes fetch and execute them. This pattern decouples request handling from slow operations.

Unicorn and Puma web servers use preforking to handle concurrent requests through process-based parallelism. The master process accepts connections and distributes them to worker processes, achieving concurrency despite Ruby's GIL.

MessagePack provides efficient binary serialization, outperforming JSON and Marshal for IPC data transfer. The msgpack gem integrates MessagePack into Ruby applications, reducing serialization overhead.

Celluloid abstracts actor-based concurrency, simplifying concurrent system design. While primarily thread-based, Celluloid patterns apply to process-based systems, with actors mapping to processes and message passing to IPC.

gRPC enables efficient RPC with Protocol Buffers serialization. The grpc gem provides Ruby gRPC implementation, useful for high-performance service-to-service communication with strict type safety.

ZeroMQ offers brokerless message passing with rich patterns. The ffi-rzmq gem exposes ZeroMQ to Ruby, providing publish-subscribe, push-pull, and request-reply patterns without central brokers.

Reference

IPC Mechanism Comparison

Ruby IPC Classes

Common Patterns

Performance Characteristics

Security Checklist

Error Handling