Blocking vs Non-Blocking

Overview

Blocking and non-blocking execution models define how programs handle I/O operations and external resources. In blocking execution, program flow halts until an operation completes. The calling thread suspends execution and waits for the result before continuing. In non-blocking execution, operations return immediately, and the program continues executing while the operation completes in the background.

The distinction matters primarily for I/O-bound operations: reading files, making network requests, querying databases, or interacting with external services. CPU-bound operations execute synchronously regardless of the execution model, as computation cannot proceed until calculations complete.

Blocking operations dominate traditional programming models because they align with sequential reasoning. A program reads a file, the read completes, the program processes the data. This linear flow matches how developers think about program execution. Non-blocking operations emerged from the need to handle multiple concurrent I/O operations efficiently without dedicating a thread to each waiting operation.

# Blocking file read
content = File.read('data.txt')
process(content)
# Execution waits at File.read until entire file loads

# Non-blocking approach with async/await pattern
async_read('data.txt') do |content|
  process(content)
end
continue_execution
# Execution continues immediately, callback runs when read completes

The performance implications differ dramatically at scale. A blocking web server handling 1000 concurrent requests requires 1000 threads, each consuming memory for its stack. A non-blocking server handles the same load with a small number of threads by multiplexing I/O operations. However, non-blocking code introduces complexity through callbacks, promises, or async/await patterns that complicate error handling and debugging.

Key Principles

Blocking execution follows a synchronous model where operations execute sequentially. When code invokes a blocking operation, the operating system deschedules the thread and places it in a wait queue. The thread remains suspended until the operation signals completion, then the OS reschedules the thread for execution. This mechanism enables cooperative multitasking at the OS level but creates overhead when many threads wait simultaneously.

The blocking model guarantees operation completion before continuing. After socket.read returns, the data exists in memory. After File.write completes, the OS has committed data to disk buffers. This guarantee simplifies error handling because failures occur at the call site. If the operation fails, the exception or error code appears exactly where the program invoked the operation.

Non-blocking execution inverts this model. Operations return immediately with a promise or future representing eventual completion. The operation proceeds asynchronously while the program continues executing. Three primary mechanisms deliver results: callbacks, promises/futures, or async/await syntax. Each mechanism addresses the same fundamental challenge: how to execute code after an asynchronous operation completes.

Callbacks represent the simplest mechanism. The caller provides a function that executes when the operation completes:

http_client.get('https://api.example.com') do |response|
  parse_response(response)
end

Promises and futures provide a more structured approach. The operation returns an object representing the eventual result. Code can chain operations on this object, compose multiple asynchronous operations, or block waiting for completion:

promise = http_client.get('https://api.example.com')
promise.then { |response| parse_response(response) }
       .then { |data| store_in_database(data) }
       .catch { |error| handle_error(error) }

Event loops coordinate non-blocking operations. The loop monitors multiple I/O operations simultaneously using system calls like select, poll, epoll, or kqueue. When any operation completes, the event loop invokes the associated callback. This single-threaded model eliminates context switching overhead and synchronization complexity but requires that callbacks execute quickly to avoid blocking the event loop.

The distinction between blocking and non-blocking affects error handling fundamentally. Blocking operations raise exceptions at the call site. Non-blocking operations must propagate errors through the callback, promise, or async mechanism. An uncaught exception in a callback terminates the event loop or goes unnoticed, depending on the implementation:

# Blocking error handling
begin
  data = File.read('config.json')
  config = JSON.parse(data)
rescue Errno::ENOENT
  use_default_config
rescue JSON::ParserError => e
  log_error("Invalid config: #{e.message}")
end

# Non-blocking error handling
async_read('config.json')
  .then { |data| JSON.parse(data) }
  .catch(Errno::ENOENT) { use_default_config }
  .catch(JSON::ParserError) { |e| log_error("Invalid config: #{e.message}") }

Resource management differs between models. Blocking code uses deterministic cleanup with ensure blocks or resource acquisition patterns. The linear control flow guarantees cleanup code executes. Non-blocking code must track resource lifetimes across asynchronous operations and ensure cleanup occurs after all callbacks complete.

Ruby Implementation

Ruby's Global Interpreter Lock (GIL) constrains non-blocking implementation strategies. The GIL prevents true parallel execution of Ruby code across multiple threads, but releases the lock during I/O operations. This design makes blocking I/O operations safe for concurrent execution while limiting CPU parallelism.

Ruby's standard library provides blocking I/O by default. Methods like File.read, Net::HTTP.get, and TCPSocket#read block the calling thread until the operation completes. Ruby releases the GIL during these operations, allowing other Ruby threads to execute concurrently:

threads = 10.times.map do |i|
  Thread.new do
    data = Net::HTTP.get(URI("https://api.example.com/data/#{i}"))
    process_data(data)
  end
end
threads.each(&:join)
# All requests execute concurrently despite blocking operations

Fibers provide lightweight cooperative concurrency. Unlike threads, fibers require explicit yielding and do not incur OS-level context switching overhead. Libraries build non-blocking abstractions on fibers by yielding when operations would block:

require 'fiber'

def async_read(file_path)
  Fiber.new do
    # Simulated async read
    EventLoop.register_read(file_path) do |data|
      Fiber.yield(data)
    end
  end
end

fiber = async_read('data.txt')
fiber.resume # Start the read
data = fiber.resume # Get the result

The async gem provides comprehensive non-blocking I/O for Ruby. It wraps blocking operations with fiber-based scheduling, automatically yielding when operations would block and resuming when data arrives:

require 'async'
require 'async/http'

Async do
  internet = Async::HTTP::Internet.new
  
  # Multiple concurrent requests without threads
  responses = [
    internet.get("https://example.com/1"),
    internet.get("https://example.com/2"),
    internet.get("https://example.com/3")
  ]
  
  responses.each do |response|
    puts response.read
  end
ensure
  internet.close
end

EventMachine provided Ruby's first popular non-blocking framework. It implements an event loop pattern where operations register callbacks that execute on completion. EventMachine operates in a single thread but handles thousands of concurrent operations:

require 'eventmachine'

EM.run do
  http = EM::HttpRequest.new('https://api.example.com/data').get
  
  http.callback do
    puts http.response
    EM.stop
  end
  
  http.errback do
    puts "Request failed"
    EM.stop
  end
end

Celluloid provided actor-based concurrency, abstracting threads and enabling non-blocking patterns through message passing. Each actor processes messages sequentially but multiple actors run concurrently:

require 'celluloid/current'

class DataFetcher
  include Celluloid
  
  def fetch(url)
    # Blocking operation runs in actor's thread
    Net::HTTP.get(URI(url))
  end
end

pool = DataFetcher.pool(size: 10)
futures = urls.map { |url| pool.future.fetch(url) }
results = futures.map(&:value) # Blocks until all complete

Ruby 3.0 introduced Ractor for parallel execution without the GIL. Ractors provide true parallelism but require message passing for communication, as they cannot share most objects:

ractor = Ractor.new do
  data = Ractor.receive
  # Process data in parallel
  result = expensive_computation(data)
  result
end

ractor.send(input_data)
result = ractor.take # Blocks until computation completes

The nio4r gem provides cross-platform non-blocking I/O primitives. It wraps platform-specific system calls like epoll and kqueue in a consistent interface:

require 'nio'

selector = NIO::Selector.new

socket = TCPSocket.new('example.com', 80)
monitor = selector.register(socket, :w)

selector.select(1) # Wait up to 1 second for writeable socket
if monitor.writeable?
  socket.write("GET / HTTP/1.1\r\nHost: example.com\r\n\r\n")
end

Non-blocking database access requires specialized gems. The mysql2 gem provides async query execution when used with an event loop:

require 'mysql2'
require 'eventmachine'

EM.run do
  client = Mysql2::EM::Client.new(host: 'localhost', username: 'root')
  
  query = client.query("SELECT * FROM users")
  query.callback do |result|
    result.each { |row| puts row['name'] }
    EM.stop
  end
end

Practical Examples

A web scraper demonstrates the performance difference between blocking and non-blocking approaches. The blocking version processes URLs sequentially, waiting for each response before requesting the next:

# Blocking scraper
require 'net/http'

urls = [
  'https://example.com/page1',
  'https://example.com/page2',
  'https://example.com/page3'
]

results = urls.map do |url|
  uri = URI(url)
  response = Net::HTTP.get(uri)
  extract_data(response)
end
# Total time: sum of all request times

The non-blocking version issues all requests concurrently and processes responses as they arrive:

# Non-blocking scraper with async gem
require 'async'
require 'async/http/internet'

Async do
  internet = Async::HTTP::Internet.new
  
  tasks = urls.map do |url|
    Async do
      response = internet.get(url)
      extract_data(response.read)
    end
  end
  
  results = tasks.map(&:wait)
ensure
  internet.close
end
# Total time: maximum of all request times (concurrent execution)

A chat server illustrates connection management differences. The blocking version dedicates one thread per connection:

# Blocking chat server
require 'socket'

server = TCPServer.new(3000)

loop do
  client = server.accept # Blocks waiting for connection
  
  Thread.new(client) do |conn|
    loop do
      message = conn.gets # Blocks waiting for message
      break if message.nil?
      broadcast(message)
    end
    conn.close
  end
end
# Scales to ~1000 connections before thread overhead dominates

The non-blocking version multiplexes connections on a single thread:

# Non-blocking chat server
require 'socket'
require 'nio'

server = TCPServer.new(3000)
selector = NIO::Selector.new
clients = {}

selector.register(server, :r)

loop do
  selector.select do |monitor|
    if monitor.io == server
      client = server.accept
      selector.register(client, :r)
      clients[client] = ""
    else
      data = monitor.io.read_nonblock(1024, exception: false)
      
      if data == :wait_readable
        next
      elsif data.nil?
        selector.deregister(monitor.io)
        clients.delete(monitor.io)
        monitor.io.close
      else
        clients[monitor.io] << data
        broadcast(data) if data.include?("\n")
      end
    end
  end
end
# Scales to 10,000+ connections with minimal memory overhead

Background job processing shows how blocking operations integrate with non-blocking event loops. A job processor might execute blocking database queries within an async context:

require 'async'
require 'async/barrier'

class JobProcessor
  def initialize(queue)
    @queue = queue
  end
  
  def process_jobs
    Async do |task|
      barrier = Async::Barrier.new
      
      loop do
        job = @queue.pop # Blocking operation
        
        barrier.async do
          # Execute job in separate fiber
          execute_job(job)
        end
        
        # Limit concurrent jobs
        barrier.wait if barrier.size >= 10
      end
    ensure
      barrier.wait # Complete remaining jobs
    end
  end
  
  private
  
  def execute_job(job)
    # Blocking database query
    result = Database.query(job.sql)
    job.callback(result)
  end
end

File processing pipelines benefit from non-blocking I/O when reading multiple files concurrently:

require 'async'
require 'async/semaphore'

def process_files(file_paths)
  Async do
    semaphore = Async::Semaphore.new(5) # Limit concurrent reads
    
    tasks = file_paths.map do |path|
      semaphore.async do
        data = File.read(path) # Ruby releases GIL during read
        process_data(data)
      end
    end
    
    tasks.map(&:wait)
  end
end

API clients often mix blocking and non-blocking operations. The client uses non-blocking HTTP requests but blocks waiting for results at strategic points:

require 'async'
require 'async/http/internet'

class APIClient
  def initialize
    @internet = Async::HTTP::Internet.new
  end
  
  def fetch_user_data(user_id)
    Async do |task|
      # Issue parallel requests
      profile_task = task.async { fetch_profile(user_id) }
      posts_task = task.async { fetch_posts(user_id) }
      friends_task = task.async { fetch_friends(user_id) }
      
      # Block waiting for all results
      {
        profile: profile_task.wait,
        posts: posts_task.wait,
        friends: friends_task.wait
      }
    end
  end
  
  private
  
  def fetch_profile(user_id)
    response = @internet.get("https://api.example.com/users/#{user_id}")
    JSON.parse(response.read)
  end
end

Design Considerations

The choice between blocking and non-blocking execution depends on workload characteristics, scaling requirements, and development complexity tolerance. Blocking execution suits applications with predictable load, where thread-per-request overhead remains acceptable. Non-blocking execution becomes necessary when connection count exceeds available threads or when I/O latency dominates execution time.

Request patterns determine model suitability. Applications serving long-polling connections, websockets, or server-sent events maintain thousands of idle connections. Blocking models assign threads to these idle connections, consuming memory without performing work. Non-blocking models multiplex idle connections efficiently, checking for activity only when events occur.

CPU-bound versus I/O-bound workload ratios affect the decision. Pure I/O workloads benefit maximally from non-blocking execution because threads spend most time waiting. Mixed workloads with substantial CPU processing see diminishing returns, as non-blocking execution cannot parallelize computation without multiple threads or processes. When 80% of request time involves computation, the 20% I/O time saved through non-blocking execution provides minimal overall improvement.

Latency requirements influence model selection. Blocking execution introduces thread scheduling delays but maintains predictable per-request latency. Non-blocking execution reduces contention for threads but introduces queuing delays when the event loop falls behind. A system requiring 99th percentile latency under 50ms might choose blocking execution to avoid event loop congestion. A system optimizing for throughput over latency typically chooses non-blocking execution.

Memory constraints favor non-blocking execution. Each thread requires stack space, typically 1-2MB. Supporting 10,000 concurrent connections demands 10-20GB for thread stacks alone. Non-blocking execution handles the same connection count with a fixed number of threads, often 1-8, consuming minimal memory for connection state.

Development complexity increases substantially with non-blocking code. Debugging async code requires tracking execution across callbacks or promise chains. Stack traces become fragmented across event loop iterations. Testing requires mock event loops and careful coordination of async operations. Teams without async experience face a steep learning curve that can delay development significantly.

Library compatibility matters. Ruby gems expect blocking I/O. Using these gems in non-blocking code blocks the event loop, negating concurrency benefits. Applications requiring specific gems might need to wrap them in thread pools, hybrid blocking/non-blocking architectures, or seek async-compatible alternatives.

Error handling complexity multiplies in non-blocking code. Exceptions thrown in callbacks might not propagate to callers. Uncaught exceptions can terminate event loops silently. Implementing proper error boundaries, timeout handling, and error recovery requires careful planning. Blocking code handles errors with standard exception mechanisms and ensure blocks.

Migration costs influence decisions. Converting blocking codebases to non-blocking execution requires rewriting I/O operations, restructuring control flow, and implementing async patterns throughout. Incremental migration strategies exist but introduce architectural complexity. Starting new projects with non-blocking execution proves easier than migrating existing codebases.

Performance Considerations

Blocking execution scales linearly with connection count until thread limits appear. Each connection consumes thread resources: stack memory, kernel structures, and context switching overhead. Systems handle hundreds of concurrent connections comfortably. Thousands of connections cause thrashing as the OS spends more time context switching than executing application code.

Non-blocking execution scales differently. Connection count affects memory usage only for connection state, not thread stacks. A single event loop handles tens of thousands of connections, limited by event loop efficiency and application callback execution time rather than threading overhead. This scaling profile makes non-blocking execution mandatory for high-concurrency scenarios.

Throughput characteristics differ fundamentally. Blocking servers achieve throughput proportional to thread count times per-request processing speed. Doubling threads doubles throughput until contention appears. Non-blocking servers achieve throughput proportional to event loop efficiency times callback execution speed. Adding threads to non-blocking servers provides minimal benefit unless callbacks block or perform CPU-intensive work.

Latency patterns show different behavior. Blocking execution provides consistent per-request latency with occasional spikes during garbage collection or context switching. 99th percentile latency remains close to median latency. Non-blocking execution provides lower median latency but higher tail latency when the event loop accumulates backlog. 99th percentile latency can be 10x median latency during load spikes.

CPU utilization patterns reveal efficiency differences. Blocking servers show high CPU usage during request processing and low usage during I/O waits. The average CPU utilization appears moderate despite individual threads alternating between full utilization and idle. Non-blocking servers maintain consistent CPU usage, as the event loop continuously processes available work without idle periods.

Memory bandwidth affects both models differently. Blocking threads require memory for stacks and frequent cache invalidation during context switches. Non-blocking execution maintains better cache locality as a single thread processes multiple operations, but large connection counts stress memory bandwidth loading connection state.

Benchmark data illustrates practical differences:

# Benchmark blocking HTTP server
require 'benchmark'
require 'webrick'

server = WEBrick::HTTPServer.new(Port: 8000)
server.mount_proc('/') { |req, res| res.body = 'Hello' }

# Apache Bench results:
# 100 concurrent connections: 5,000 req/sec
# 1,000 concurrent connections: 4,000 req/sec (thread contention)
# 10,000 concurrent connections: server crashes (too many threads)

# Benchmark non-blocking HTTP server
require 'async'
require 'async/http/server'

server = Async::HTTP::Server.new do |request|
  Protocol::HTTP::Response[200, {}, ['Hello']]
end

# Apache Bench results:
# 100 concurrent connections: 8,000 req/sec
# 1,000 concurrent connections: 10,000 req/sec
# 10,000 concurrent connections: 12,000 req/sec (scales well)

Database query performance demonstrates I/O wait impact. Blocking execution with sequential queries:

# Blocking sequential queries
start = Time.now
users = User.find([1, 2, 3, 4, 5])
profiles = users.map { |u| Profile.find(u.profile_id) }
elapsed = Time.now - start
# Total time: 5 * query_latency (assuming 20ms queries = 100ms total)

Non-blocking parallel queries:

# Non-blocking parallel queries
Async do
  users = User.find([1, 2, 3, 4, 5])
  
  tasks = users.map do |u|
    Async { Profile.find(u.profile_id) }
  end
  
  profiles = tasks.map(&:wait)
end
# Total time: max(query_latencies) ≈ 20ms (all queries concurrent)

Optimization strategies differ between models. Blocking servers optimize through thread pool tuning, connection limits, and request queuing. Non-blocking servers optimize through callback efficiency, event loop tuning, and minimizing blocking operations within callbacks.

Common Pitfalls

Blocking the event loop destroys non-blocking performance. Any blocking operation within a callback prevents the event loop from processing other operations. A single 100ms blocking operation in a callback reduces throughput to 10 requests/second regardless of concurrent connection count:

# Event loop blocked - performance disaster
Async do
  http = Async::HTTP::Internet.new
  
  http.get('https://api.example.com') do |response|
    # This blocks the event loop for 1 second
    sleep(1)
    process(response)
  end
  
  # No other operations can proceed during sleep
end

The solution wraps blocking operations in separate threads or processes:

# Proper handling of blocking operations
Async do
  http = Async::HTTP::Internet.new
  
  http.get('https://api.example.com') do |response|
    # Execute blocking operation in thread pool
    Async do |task|
      task.async do
        Thread.new { process(response) }.value
      end
    end
  end
end

Error propagation fails silently in callbacks without explicit handling. Unhandled exceptions in callbacks might terminate the event loop or disappear completely:

# Silent error - exception disappears
Async do
  http = Async::HTTP::Internet.new
  
  Async do
    response = http.get('https://invalid-url')
    # Connection error raised here disappears
    process(response)
  end
end

Proper error handling catches exceptions at every async boundary:

# Correct error handling
Async do
  http = Async::HTTP::Internet.new
  
  Async do
    begin
      response = http.get('https://invalid-url')
      process(response)
    rescue => e
      log_error("Request failed: #{e.message}")
    end
  end
end

Resource leaks occur when async operations fail before releasing resources. A connection opened in a callback might never close if an exception interrupts execution:

# Resource leak risk
Async do
  socket = TCPSocket.new('example.com', 80)
  # Exception here leaks socket
  data = process_request
  socket.write(data)
  socket.close
end

Always use ensure blocks to guarantee resource cleanup:

# Guaranteed cleanup
Async do
  socket = TCPSocket.new('example.com', 80)
  begin
    data = process_request
    socket.write(data)
  ensure
    socket.close
  end
end

Callback ordering assumptions create race conditions. Operations complete in unpredictable order based on network timing, system load, and event loop scheduling:

# Race condition - order not guaranteed
results = []

Async do
  [1, 2, 3].each do |i|
    Async do
      response = fetch_data(i)
      results << response # Order undefined
    end
  end
end

Synchronize results explicitly when order matters:

# Guaranteed ordering
results = []

Async do |task|
  tasks = [1, 2, 3].map do |i|
    task.async { fetch_data(i) }
  end
  
  results = tasks.map(&:wait) # Results in order of tasks array
end

Timeout handling in async code requires explicit timeout mechanisms. Blocking operations timeout automatically after TCP timeout periods, but async operations might wait indefinitely without timeout handling:

# No timeout - might wait forever
Async do
  response = http.get('https://slow-server.com')
  process(response)
end

Implement timeouts at operation level:

# Explicit timeout handling
Async do
  Async::Task.current.with_timeout(5) do
    response = http.get('https://slow-server.com')
    process(response)
  end
rescue Async::TimeoutError
  handle_timeout
end

Mixing blocking and non-blocking libraries causes unpredictable behavior. A gem performing blocking I/O within an async context blocks the event loop:

# Blocking gem ruins async performance
require 'async'
require 'net/http' # Blocking library

Async do
  # This blocks the event loop
  response = Net::HTTP.get(URI('https://example.com'))
  process(response)
end

Use async-compatible libraries or isolate blocking operations:

# Async-compatible library
require 'async/http/internet'

Async do
  internet = Async::HTTP::Internet.new
  response = internet.get('https://example.com')
  process(response.read)
end

Debugging async code reveals incomplete stack traces. Exceptions occur in callback context, not the original calling context:

# Debugging challenge
Async do
  fetch_data.then do |data|
    process(data)
    # Exception here shows truncated stack trace
  end
end

Add contextual logging and error tracking throughout async chains:

# Improved debugging
Async do
  logger.debug("Starting fetch_data")
  
  begin
    fetch_data.then do |data|
      logger.debug("Processing data: #{data.size} bytes")
      process(data)
    end
  rescue => e
    logger.error("Failed: #{e.message}")
    logger.error(e.backtrace.join("\n"))
    raise
  end
end

Reference

Execution Model Comparison

Ruby Async Libraries

System Calls for Non-Blocking I/O

Async Pattern Methods

Performance Characteristics

Ruby Fiber API

Common Error Patterns

Async HTTP Client Examples

Event Loop Tuning