Long Polling

Overview

Long polling addresses the limitation of traditional HTTP request-response cycles in scenarios requiring real-time or near-real-time data updates. In standard HTTP communication, clients must repeatedly poll servers at intervals to check for new data, creating unnecessary network traffic and latency. Long polling maintains an open HTTP connection, with the server holding the request until new data arrives or a timeout expires.

The technique emerged as a workaround for HTTP's stateless, client-initiated communication model before WebSocket standardization. Applications requiring instant notifications—chat systems, live dashboards, stock tickers, collaborative editing tools—benefit from long polling when WebSocket support is unavailable or unnecessary.

The core mechanism involves three stages: the client sends a request, the server holds that request open without immediately responding, and when data becomes available (or timeout occurs), the server responds and the client immediately initiates a new request. This creates a persistent communication channel using standard HTTP infrastructure.

# Basic long polling flow
def long_poll_request
  loop do
    response = HTTP.get("https://api.example.com/events")
    process_data(response.body)
    # Immediately reconnect after receiving response
  end
end

Long polling differs from short polling (regular interval requests) and streaming (single persistent connection with multiple responses). Each approach suits different network conditions, server capabilities, and application requirements.

Key Principles

Long polling operates on several fundamental principles that distinguish it from other communication patterns. The server maintains request connections in a suspended state rather than responding immediately with "no data available" messages. This suspension requires the server to track pending requests and associate them with specific clients or channels.

The timeout mechanism prevents indefinite connection hangs. Servers typically close connections after 30-60 seconds if no data arrives, prompting clients to immediately reconnect. This timeout serves multiple purposes: preventing resource exhaustion from indefinitely held connections, detecting dead connections, and complying with proxy and load balancer timeout policies.

Request-response atomicity ensures each data payload gets delivered exactly once. When the server sends data, it closes the connection, and the client must initiate a new request to receive subsequent updates. This differs from streaming, where a single connection carries multiple data frames.

# Server-side principle: hold request until data or timeout
class EventsController < ApplicationController
  def poll
    timeout = 50.seconds
    start_time = Time.current
    
    loop do
      event = Event.pending_for_user(current_user).first
      return render json: event if event
      
      if Time.current - start_time > timeout
        return render json: { status: 'timeout' }, status: 204
      end
      
      sleep 0.5
    end
  end
end

The client-driven reconnection pattern places responsibility for maintaining the communication channel on the client. After receiving any response (data or timeout), the client immediately initiates a new long poll request. This creates a continuous loop of request-wait-response-request cycles.

Connection state management presents challenges since HTTP connections are stateless. Servers must implement mechanisms to associate long poll requests with user sessions, authentication tokens, or subscription channels. Each request includes credentials or session identifiers to maintain context across multiple request-response cycles.

Backoff strategies handle error scenarios where immediate reconnection would overwhelm servers. Clients implement exponential backoff when receiving error responses, gradually increasing delay between connection attempts from milliseconds to seconds.

# Client-side reconnection with exponential backoff
class LongPollClient
  MAX_BACKOFF = 30
  
  def start
    @backoff = 1
    
    loop do
      begin
        response = perform_long_poll
        @backoff = 1  # Reset on success
        handle_response(response)
      rescue => e
        sleep @backoff
        @backoff = [@backoff * 2, MAX_BACKOFF].min
      end
    end
  end
end

Ruby Implementation

Ruby web frameworks provide multiple approaches for implementing long polling. Rack-based applications can implement long polling through middleware, while Rails offers controller-based solutions. The key challenge involves preventing thread or process blocking while maintaining open connections.

Rack applications handle long polling through hijacking, a technique where the application takes control of the TCP socket from the web server. This allows fine-grained control over connection timing and response delivery.

# Rack middleware for long polling
class LongPollMiddleware
  def initialize(app)
    @app = app
    @subscribers = []
  end
  
  def call(env)
    return @app.call(env) unless env['PATH_INFO'] == '/poll'
    
    # Hijack the connection
    env['rack.hijack'].call
    io = env['rack.hijack_io']
    
    subscriber = Subscriber.new(io)
    @subscribers << subscriber
    
    # Clean up after timeout or disconnect
    Thread.new do
      sleep 60
      remove_subscriber(subscriber)
    end
    
    # Return async marker
    [-1, {}, []]
  end
  
  def broadcast(data)
    @subscribers.each do |sub|
      sub.send_data(data)
    end
    @subscribers.clear
  end
end

Rails controllers implement long polling using various techniques. The simplest approach uses sleep loops within controller actions, though this blocks application threads. More sophisticated implementations use threads or asynchronous execution.

# Rails controller with thread-based long polling
class MessagesController < ApplicationController
  def poll
    queue = MessageQueue.for_user(current_user)
    timeout = 45.seconds
    
    message = wait_for_message(queue, timeout)
    
    if message
      render json: message
    else
      head :no_content
    end
  end
  
  private
  
  def wait_for_message(queue, timeout)
    deadline = Time.current + timeout
    
    loop do
      message = queue.pop(non_block: true) rescue nil
      return message if message
      
      return nil if Time.current >= deadline
      
      sleep 0.2
    end
  end
end

The Concurrent Ruby gem provides thread-safe data structures that facilitate long polling implementations. Thread pools manage multiple concurrent long poll connections without creating excessive threads.

# Using concurrent-ruby for efficient long polling
require 'concurrent'

class LongPollServer
  def initialize
    @subscribers = Concurrent::Map.new
    @pool = Concurrent::FixedThreadPool.new(100)
  end
  
  def add_subscriber(user_id, connection)
    future = Concurrent::Future.execute(executor: @pool) do
      wait_for_event(user_id)
    end
    
    @subscribers[connection] = future
    
    future.then do |event|
      send_event(connection, event)
      @subscribers.delete(connection)
    end
  end
  
  def wait_for_event(user_id)
    timeout = 50
    
    Timeout::timeout(timeout) do
      Redis.current.blpop("events:#{user_id}", timeout: timeout)
    end
  rescue Timeout::Error
    nil
  end
end

Puma and other threaded Ruby servers handle long polling more efficiently than forking servers like Unicorn. Each long poll connection occupies a thread, so thread-based servers scale better for this pattern. Configuration requires increasing thread counts to accommodate held connections.

# Puma configuration for long polling
# config/puma.rb
workers 4
threads 50, 100  # Higher thread count for held connections

preload_app!

on_worker_boot do
  # Reconnect to Redis/database
  ActiveRecord::Base.establish_connection
end

EventMachine provides an asynchronous approach that avoids blocking threads entirely. Connections register callbacks that fire when data becomes available, allowing a single thread to manage thousands of concurrent long poll connections.

# EventMachine-based long polling
require 'eventmachine'
require 'em-http-request'

class EventMachineLongPoll
  def start
    EM.run do
      EM::HttpRequest.new('http://api.example.com/events').get(
        keepalive: true,
        timeout: 60
      ).callback do |http|
        process_response(http.response)
        start  # Reconnect
      end.errback do |http|
        EM.add_timer(2) { start }  # Retry with delay
      end
    end
  end
end

Design Considerations

Long polling represents one point in a spectrum of real-time communication approaches. The decision to use long polling depends on requirements for latency, server load, client compatibility, and infrastructure constraints.

WebSockets provide true bidirectional communication with lower overhead than long polling. Each long poll request includes full HTTP headers, while WebSocket frames contain minimal overhead after the initial handshake. However, WebSockets require specific server support, may face challenges with corporate proxies or firewalls, and add complexity to infrastructure that already handles HTTP well.

Server-Sent Events (SSE) offer a simpler alternative for server-to-client communication. SSE uses a single long-lived connection with the server pushing multiple events through one request. Long polling requires a new HTTP request for each message, generating more network overhead. SSE falls short when bidirectional communication is required, as it only supports server-to-client flow.

# Trade-off comparison through implementation
class RealtimeController < ApplicationController
  # Long polling: new request per message
  def long_poll
    message = wait_for_message(timeout: 50)
    render json: message
    # Client must reconnect
  end
  
  # SSE: multiple messages, one connection
  def stream
    response.headers['Content-Type'] = 'text/event-stream'
    sse = SSE.new(response.stream)
    
    loop do
      message = wait_for_message(timeout: 5)
      sse.write(message) if message
    end
  ensure
    sse.close
  end
end

Regular short polling suffices for applications with relaxed latency requirements. Polling every 5-30 seconds generates predictable server load and simpler client implementations. Long polling makes sense when latency requirements fall below 2-3 seconds and message frequency is unpredictable—checking every second wastes resources, while checking every 10 seconds misses real-time requirements.

Infrastructure compatibility influences the decision significantly. Long polling works through any proxy, load balancer, or firewall that supports standard HTTP. WebSockets and SSE may require specific proxy configurations or face blocking by corporate networks. Applications requiring broad compatibility across restrictive networks favor long polling despite its higher overhead.

Server resource consumption patterns differ between approaches. Long polling holds server connections but releases them regularly when timeouts expire or data arrives. WebSockets maintain persistent connections that never close unless explicitly disconnected. For applications where messages arrive sporadically, long polling may actually consume fewer resources since connections close during idle periods.

# Resource consumption pattern
class ResourceMonitor
  def measure_long_polling
    # Connection held 45s average (timeout or message)
    # 100 clients = 100 connections for 45s each
    # Then all close and reconnect
    connections_per_minute = (60.0 / 45) * 100  # ~133
  end
  
  def measure_websocket
    # Connection held indefinitely
    # 100 clients = 100 persistent connections
    # No reconnection overhead
    connections_per_minute = 100
  end
end

Implementation complexity scales with the approach chosen. Long polling requires handling reconnection logic, exponential backoff, and request deduplication, but uses familiar HTTP patterns. WebSockets add protocol negotiation, frame parsing, ping-pong heartbeats, and complex error recovery scenarios.

Implementation Approaches

Several architectural patterns address different long polling requirements and scale characteristics. The choice between approaches depends on message routing needs, concurrency requirements, and data consistency guarantees.

The direct database polling approach queries the database repeatedly within the held connection, checking for new records. This pattern suits applications with low connection counts where database queries are fast and indexed properly.

# Direct database polling
class NotificationsController < ApplicationController
  def poll
    last_id = params[:last_id]
    timeout = 45.seconds
    deadline = Time.current + timeout
    
    loop do
      notification = Notification
        .where(user_id: current_user.id)
        .where('id > ?', last_id)
        .order(id: :asc)
        .first
      
      return render json: notification if notification
      return head :no_content if Time.current >= deadline
      
      sleep 1
    end
  end
end

The message queue approach decouples data production from consumption. When new data arrives, producers push messages to a queue, and long poll handlers consume from that queue. Redis or RabbitMQ serve as intermediary message brokers, reducing database load and enabling fan-out to multiple subscribers.

# Message queue approach with Redis
class EventsController < ApplicationController
  def poll
    channel = "user:#{current_user.id}:events"
    timeout = 50
    
    # Blocking Redis pop with timeout
    result = REDIS.blpop(channel, timeout: timeout)
    
    if result
      _, data = result
      render json: JSON.parse(data)
    else
      head :no_content
    end
  end
end

# Producer side
class EventPublisher
  def publish(user_id, event_data)
    channel = "user:#{user_id}:events"
    REDIS.rpush(channel, event_data.to_json)
  end
end

The observer pattern maintains in-memory subscriber registries where long poll connections register interest in specific channels or topics. When events occur, the application notifies all registered subscribers. This approach delivers events immediately without polling delays but requires shared state across server processes.

# Observer pattern with shared subscribers
class EventHub
  @subscribers = Concurrent::Map.new
  
  class << self
    def subscribe(channel, connection)
      subscribers[channel] ||= Concurrent::Array.new
      subscribers[channel] << connection
      
      # Auto-cleanup after timeout
      Concurrent::ScheduledTask.execute(50) do
        unsubscribe(channel, connection)
      end
    end
    
    def publish(channel, data)
      return unless subscribers[channel]
      
      subscribers[channel].each do |connection|
        connection.send_data(data)
      end
      
      subscribers[channel].clear
    end
    
    def unsubscribe(channel, connection)
      subscribers[channel]&.delete(connection)
    end
    
    private
    
    attr_reader :subscribers
  end
end

The hybrid approach combines immediate delivery with fallback polling. Connections first check for buffered messages, subscribe to real-time notifications, then fall back to periodic polling if no events arrive. This handles edge cases where messages arrive during connection establishment.

# Hybrid approach
class HybridPollController < ApplicationController
  def poll
    last_id = params[:last_id].to_i
    
    # Check for buffered messages first
    buffered = Message.where(user_id: current_user.id)
                     .where('id > ?', last_id)
                     .order(:id)
                     .first
    
    return render json: buffered if buffered
    
    # Subscribe to real-time notifications
    queue = SubscriptionManager.subscribe(current_user.id)
    
    # Wait with timeout
    message = wait_with_fallback(queue, last_id)
    
    render json: message if message
    head :no_content unless message
  ensure
    SubscriptionManager.unsubscribe(current_user.id, queue)
  end
  
  private
  
  def wait_with_fallback(queue, last_id)
    deadline = Time.current + 45.seconds
    
    loop do
      # Check queue for real-time message
      return queue.pop(non_block: true) rescue nil if queue
      
      # Fallback: check database
      message = Message.where(user_id: current_user.id)
                      .where('id > ?', last_id)
                      .first
      return message if message
      
      return nil if Time.current >= deadline
      
      sleep 0.5
    end
  end
end

Distributed system implementations require coordination across multiple server instances. Redis Pub/Sub enables cross-server event distribution, allowing any server to publish events that all connected long poll handlers receive.

# Distributed long polling with Redis Pub/Sub
class DistributedLongPoll
  def initialize
    @redis = Redis.new
    @local_subscribers = Concurrent::Map.new
  end
  
  def start_subscriber_thread
    Thread.new do
      @redis.subscribe('events') do |on|
        on.message do |channel, message|
          data = JSON.parse(message)
          notify_local_subscribers(data)
        end
      end
    end
  end
  
  def add_connection(user_id, connection)
    @local_subscribers[user_id] ||= Concurrent::Array.new
    @local_subscribers[user_id] << connection
  end
  
  def notify_local_subscribers(data)
    user_id = data['user_id']
    return unless @local_subscribers[user_id]
    
    @local_subscribers[user_id].each do |connection|
      connection.send_data(data)
    end
    
    @local_subscribers.delete(user_id)
  end
  
  # Any server can publish
  def publish_event(user_id, event_data)
    @redis.publish('events', {
      user_id: user_id,
      data: event_data
    }.to_json)
  end
end

Performance Considerations

Long polling's performance characteristics stem from maintaining open connections and the overhead of repeated HTTP requests. Each pattern decision impacts resource consumption, latency, and scale limitations.

Connection count represents the primary scalability constraint. Each long poll connection consumes a thread, file descriptor, and memory for connection state. Ruby application servers like Puma configure maximum thread counts (typically 5-100 per worker), limiting concurrent long poll connections. A server with 4 workers and 50 threads handles at most 200 simultaneous long poll connections.

# Performance: connection capacity calculation
class CapacityPlanner
  def max_long_poll_connections
    workers = 4
    threads_per_worker = 50
    workers * threads_per_worker  # 200 connections
  end
  
  def required_capacity(active_users:, avg_connection_time:)
    # Average 45s connection time
    requests_per_minute = 60.0 / avg_connection_time
    # Each user makes requests_per_minute requests
    peak_concurrent = active_users / requests_per_minute
    
    # Add 20% buffer
    (peak_concurrent * 1.2).ceil
  end
end

Request overhead accumulates from repeated HTTP handshakes. Each long poll cycle includes full HTTP headers (typically 500-1500 bytes), TCP handshake overhead, and SSL negotiation if using HTTPS. At 1000 active users with 45-second timeouts, approximately 1333 requests per minute occur, each carrying header overhead. WebSockets eliminate this repeated overhead after initial connection.

Database or cache query frequency affects backend resource consumption. The direct polling approach queries the database every 0.5-2 seconds per connection. With 200 connections, this generates 100-400 queries per second. The message queue approach reduces database load by checking a central queue instead of individual database queries.

# Database load comparison
class LoadAnalysis
  def direct_polling_qps(connections:, poll_interval:)
    # Each connection queries every poll_interval seconds
    connections / poll_interval
    # 200 connections, 1s interval = 200 QPS
  end
  
  def queue_based_qps(connections:)
    # Redis BLPOP blocks until data or timeout
    # No repeated queries during wait
    connections * 0  # Zero query load while waiting
  end
end

Timeout selection balances latency and resource efficiency. Shorter timeouts (10-20 seconds) reduce maximum message delivery latency but increase connection churn and overhead. Longer timeouts (45-60 seconds) reduce overhead but may cause issues with proxies or load balancers that enforce their own timeouts.

Memory consumption grows with connection count and buffered data. Each held connection consumes memory for connection state, thread stack space (typically 1MB per thread), and any buffered message data. Applications holding 500 connections require approximately 500MB just for thread stacks.

# Memory profiling for long polling
class MemoryProfile
  def connection_overhead
    base_memory = ObjectSpace.memsize_of(Object.new)
    thread_stack = 1.megabyte
    connection_state = 4.kilobytes
    
    per_connection_mb = (thread_stack + connection_state) / 1.megabyte
    # ~1MB per connection
  end
  
  def total_memory_requirement(concurrent_connections)
    overhead = connection_overhead
    ruby_vm = 50.megabytes
    application = 200.megabytes
    
    ruby_vm + application + (overhead * concurrent_connections)
    # 250MB + 1MB per connection
  end
end

Response buffering at reverse proxies impacts long polling behavior. Nginx and other reverse proxies may buffer responses, preventing immediate delivery to clients. Configuration must disable buffering for long poll endpoints.

# Nginx configuration for long polling
# nginx.conf
location /poll {
  proxy_pass http://app_servers;
  proxy_buffering off;
  proxy_read_timeout 60s;
  proxy_connect_timeout 5s;
}

Scaling horizontally requires sticky sessions or state sharing. Without session affinity, each request may hit a different server, complicating in-memory subscriber management. Load balancers must route subsequent requests from the same client to the same server, or applications must use shared state (Redis Pub/Sub) across servers.

Common Pitfalls

Several failure patterns emerge when implementing long polling without careful consideration of edge cases and error conditions. These pitfalls cause resource leaks, duplicate messages, or connection failures.

Thundering herd occurs when many connections timeout simultaneously and all reconnect at once. If 1000 clients use identical 45-second timeouts starting at the same time, all 1000 reconnect simultaneously after timeout, overwhelming the server. Randomizing timeouts or staggering reconnection prevents this spike.

# Pitfall: thundering herd
class BadLongPoll
  def poll
    timeout = 45.seconds  # All clients timeout together
    # ...
  end
end

# Solution: randomized timeout
class GoodLongPoll
  def poll
    base_timeout = 45.seconds
    jitter = rand(-5..5).seconds
    timeout = base_timeout + jitter
    # Spreads reconnections across 10-second window
  end
end

Missing cleanup logic causes connection leaks. When clients disconnect ungracefully (browser closes, network drops), servers must detect the disconnection and clean up resources. Without cleanup, subscriber lists grow unbounded, memory leaks occur, and stale connections persist.

# Pitfall: no cleanup detection
class LeakyPoll
  def poll
    subscribers << connection
    wait_for_data
    # If client disconnects, connection remains in subscribers
  end
end

# Solution: connection monitoring
class CleanPoll
  def poll
    connection.on_close { cleanup(connection) }
    
    timeout = 45.seconds
    start = Time.current
    
    loop do
      return if connection.closed?  # Detect disconnection
      
      data = check_for_data
      return send_data(data) if data
      return timeout_response if Time.current - start > timeout
      
      sleep 0.5
    end
  ensure
    cleanup(connection)  # Always cleanup
  end
end

Duplicate message delivery happens when clients reconnect before receiving the previous response. If a client sends a second long poll request while the first is still pending, and a message arrives, both connections might receive the same message. Using sequence IDs or acknowledgment prevents duplicates.

# Pitfall: duplicate delivery
def poll
  message = wait_for_message
  render json: message
  # If client already reconnected, they get this message twice
end

# Solution: sequence-based delivery
def poll
  last_id = params[:last_id].to_i
  
  message = Message
    .where('id > ?', last_id)
    .where(user_id: current_user.id)
    .order(:id)
    .first
  
  if message
    render json: { id: message.id, data: message.data }
  else
    head :no_content
  end
end

Unbounded retry loops cause cascading failures. When servers return errors, clients that retry immediately without backoff multiply load on an already struggling system. Implementing exponential backoff with maximum retry limits prevents retry storms.

# Pitfall: immediate retry
class AggressiveClient
  def start
    loop do
      response = long_poll rescue nil
      # Immediate retry on error hammers server
    end
  end
end

# Solution: exponential backoff with circuit breaker
class ResilientClient
  MAX_FAILURES = 5
  
  def start
    failures = 0
    backoff = 1
    
    loop do
      if failures >= MAX_FAILURES
        sleep 60  # Circuit breaker: long pause
        failures = 0
      end
      
      begin
        response = long_poll
        failures = 0
        backoff = 1
      rescue => e
        failures += 1
        sleep backoff
        backoff = [backoff * 2, 30].min
      end
    end
  end
end

Authentication token expiration mid-connection causes confusing failures. If a long poll request stays open for 45 seconds but authentication tokens expire after 30 seconds, the response attempt fails with authentication errors. Refresh tokens before timeouts or use longer-lived tokens for long poll endpoints.

Proxy and load balancer timeouts interfere with long polling. Many proxies close idle connections after 30-60 seconds. If application timeouts exceed proxy timeouts, proxies close connections without notifying either client or server. Applications must use timeouts shorter than infrastructure limits.

# Pitfall: timeout longer than proxy
class ProxyKilledPoll
  def poll
    timeout = 120.seconds  # Proxy closes at 60s
    wait_for_data(timeout)
    # Connection already closed by proxy
  end
end

# Solution: respect infrastructure limits
class ProxyAwarePoll
  def poll
    proxy_timeout = 55.seconds  # Under proxy's 60s limit
    app_timeout = 50.seconds    # Leave margin for response
    wait_for_data(app_timeout)
  end
end

Missing connection state tracking causes resource exhaustion. Without limiting concurrent connections per user, malicious or buggy clients can exhaust server resources by opening hundreds of connections. Per-user connection limits protect against this scenario.

# Connection limiting
class RateLimitedPoll
  MAX_CONNECTIONS_PER_USER = 5
  
  def poll
    count = ConnectionTracker.count(current_user.id)
    
    if count >= MAX_CONNECTIONS_PER_USER
      return render json: { error: 'Too many connections' }, 
                    status: 429
    end
    
    ConnectionTracker.increment(current_user.id)
    
    begin
      # Perform long poll
    ensure
      ConnectionTracker.decrement(current_user.id)
    end
  end
end

Reference

Long Polling Pattern Components

Ruby Framework Comparison

Timeout Strategy Patterns

Common Configuration Parameters

Performance Metrics

Error Codes and Handling

Ruby Gems for Long Polling

Decision Matrix