Overview

Ruby's parsing ecosystem has undergone significant transformation with the introduction of Prism (formerly YARP) in Ruby 3.3. The traditional parse.y file, which used yacc/bison grammar to generate Ruby's parser, has been supplemented by a new error-tolerant, maintainable recursive descent parser that addresses long-standing issues with maintainability, error tolerance, portability, and performance.

Parse.y-based parsers suffered from several limitations: they were difficult to maintain across Ruby versions, lacked comprehensive error tolerance for editor and tooling support, had portability issues across different Ruby implementations, and created maintenance burdens for the numerous parsing tools in the Ruby ecosystem. The migration from parse.y represents both a technical improvement and a consolidation effort to reduce the fragmented state of Ruby parsing tools.

Prism provides compatibility layers through classes like Prism::Translation::Parser that allow existing tools to migrate gradually while maintaining their current APIs. Performance benchmarks show 4-6 times speedup compared to traditional parser gem implementations, making this migration both a compatibility requirement for newer Ruby versions and a performance optimization opportunity.

The migration landscape includes several scenarios: moving from the whitequark/parser gem to Prism, updating tools that relied on Ripper, migrating custom parse.y implementations, and adapting Ruby implementation parsers. Each scenario requires different strategies and considerations for maintaining compatibility while gaining the benefits of modern parsing infrastructure.

# Traditional parser gem approach
require 'parser/current'
buffer = Parser::Source::Buffer.new('example.rb')
buffer.source = code
ast = Parser::CurrentRuby.new.parse(buffer)

# Modern Prism approach with compatibility layer
require 'prism'
require 'prism/translation/parser'
buffer = Parser::Source::Buffer.new('example.rb') 
buffer.source = code
parser = Prism::Translation::Parser.new
ast = parser.parse(buffer)

# Direct Prism usage without compatibility layer
require 'prism'
result = Prism.parse(code, filepath: 'example.rb')
ast = result.value
comments = result.comments
errors = result.errors

Basic Usage

Migration patterns fall into three primary categories: compatibility layer usage, direct Prism adoption, and hybrid approaches. The compatibility layer provides the smoothest transition path for existing codebases, while direct Prism usage offers maximum performance and feature access.

The Prism::Translation::Parser class serves as a drop-in replacement for parser gem usage, inheriting from the base parser and overriding parse methods to use Prism internally while maintaining the same external API. This approach allows tools like RuboCop to migrate with minimal code changes while immediately benefiting from Prism's performance improvements.

# Migrating from parser gem to Prism compatibility layer
class CodeAnalyzer
  def initialize
    # Before: Parser::CurrentRuby.new
    @parser = Prism::Translation::Parser.new
  end
  
  def analyze_file(filepath)
    source = File.read(filepath)
    buffer = Parser::Source::Buffer.new(filepath)
    buffer.source = source
    
    # API remains identical
    ast = @parser.parse(buffer)
    process_nodes(ast)
  end
  
  private
  
  def process_nodes(node)
    # Node processing logic unchanged
    node.children.each { |child| process_nodes(child) if child.is_a?(Parser::AST::Node) }
  end
end

Direct Prism usage provides access to enhanced error handling, serialization capabilities, and improved node structures. Prism is designed to be portable, error tolerant, and maintainable, written in C99 with no dependencies, making it suitable for embedding in various Ruby implementations and tools.

# Direct Prism usage for new implementations
class PrismAnalyzer
  def analyze_file(filepath)
    source = File.read(filepath)
    result = Prism.parse(source, filepath: filepath)
    
    unless result.success?
      handle_parse_errors(result.errors)
      return nil if result.errors.any?(&:fatal?)
    end
    
    process_prism_nodes(result.value)
    {
      ast: result.value,
      comments: result.comments,
      warnings: result.warnings,
      source_map: build_source_map(result)
    }
  end
  
  private
  
  def process_prism_nodes(node)
    visitor = Prism::Visitor.new
    node.accept(visitor)
  end
  
  def handle_parse_errors(errors)
    errors.each do |error|
      puts "Parse error at #{error.location}: #{error.message}"
    end
  end
end

Version compatibility considerations require careful handling when supporting multiple Ruby versions. The parser gem is no longer being updated for Ruby 3.4+ syntax, making migration to Prism necessary for supporting newer Ruby features.

# Version-aware migration strategy
class VersionAwareParsing
  def self.create_parser(ruby_version = RUBY_VERSION)
    if Gem::Version.new(ruby_version) >= Gem::Version.new('3.3')
      if defined?(Prism::Translation::Parser)
        Prism::Translation::Parser.new
      else
        require 'prism/translation/parser'
        Prism::Translation::Parser.new
      end
    else
      require 'parser/current'
      Parser::CurrentRuby.new
    end
  end
  
  def self.parse_with_fallback(source, filepath: nil)
    begin
      if defined?(Prism)
        result = Prism.parse(source, filepath: filepath)
        return result if result.success?
      end
    rescue => e
      warn "Prism parsing failed, falling back: #{e.message}"
    end
    
    # Fallback to traditional parser
    require 'parser/current'
    buffer = Parser::Source::Buffer.new(filepath || '(string)')
    buffer.source = source
    Parser::CurrentRuby.new.parse(buffer)
  end
end

Incremental migration allows gradual adoption without disrupting existing functionality. This approach works well for large codebases or tools with extensive test suites that need validation at each migration step.

# Incremental migration with feature flagging
class GradualMigration
  def initialize(use_prism: ENV['USE_PRISM'] == 'true')
    @use_prism = use_prism
    setup_parser
  end
  
  def parse(source, filepath: nil)
    if @use_prism
      parse_with_prism(source, filepath)
    else
      parse_with_parser_gem(source, filepath)
    end
  end
  
  private
  
  def setup_parser
    if @use_prism
      require 'prism/translation/parser'
      @parser = Prism::Translation::Parser.new
    else
      require 'parser/current' 
      @parser = Parser::CurrentRuby.new
    end
  end
  
  def parse_with_prism(source, filepath)
    buffer = create_buffer(source, filepath)
    @parser.parse(buffer)
  end
  
  def parse_with_parser_gem(source, filepath)
    buffer = create_buffer(source, filepath)
    @parser.parse(buffer)
  end
  
  def create_buffer(source, filepath)
    buffer = Parser::Source::Buffer.new(filepath || '(string)')
    buffer.source = source
    buffer
  end
end

Migration & Compatibility

The parser gem reached end-of-life for Ruby 3.4+ support, requiring active migration for projects that need to support newer Ruby syntax. Migration strategies vary based on current parser usage, target Ruby versions, and compatibility requirements with existing toolchains.

Parser Gem to Prism Translation Layer

Prism::Translation::Parser provides the most seamless migration path by implementing the parser gem's API while using Prism internally. This compatibility layer handles offset conversions, node structure mapping, and error reporting differences between the two systems.

# Before: parser gem implementation
require 'parser/current'

class LegacyCodeAnalysis
  def initialize
    @parser = Parser::CurrentRuby.new
  end
  
  def extract_method_calls(source, filename = nil)
    buffer = Parser::Source::Buffer.new(filename || '(string)')
    buffer.source = source
    
    begin
      ast = @parser.parse(buffer)
      method_calls = []
      extract_calls_recursive(ast, method_calls)
      method_calls
    rescue Parser::SyntaxError => e
      { error: e.message, line: e.diagnostic.location.line }
    end
  end
  
  private
  
  def extract_calls_recursive(node, calls)
    return unless node.is_a?(Parser::AST::Node)
    calls << node if node.type == :send
    node.children.each { |child| extract_calls_recursive(child, calls) }
  end
end

# After: Prism compatibility layer
require 'prism/translation/parser'

class MigratedCodeAnalysis
  def initialize
    @parser = Prism::Translation::Parser.new
  end
  
  def extract_method_calls(source, filename = nil)
    buffer = Parser::Source::Buffer.new(filename || '(string)')
    buffer.source = source
    
    begin
      # Same API, Prism backend
      ast = @parser.parse(buffer)
      method_calls = []
      extract_calls_recursive(ast, method_calls)
      method_calls
    rescue Parser::SyntaxError => e
      { error: e.message, line: e.diagnostic.location.line }
    end
  end
  
  private
  
  # Same extraction logic works unchanged
  def extract_calls_recursive(node, calls)
    return unless node.is_a?(Parser::AST::Node)
    calls << node if node.type == :send
    node.children.each { |child| extract_calls_recursive(child, calls) }
  end
end

Direct Prism Migration

Direct Prism usage provides access to enhanced error tolerance, better performance, and richer AST information. Error tolerance allows parsers to continue parsing programs even with syntax errors, generating syntax trees that editors and language servers can use for analysis.

# Migrating to direct Prism usage
class ModernCodeAnalysis  
  def extract_method_calls(source, filename = nil)
    result = Prism.parse(source, filepath: filename)
    
    if result.success?
      method_calls = []
      visitor = MethodCallVisitor.new(method_calls)
      result.value.accept(visitor)
      {
        calls: method_calls,
        comments: result.comments.map(&:location),
        warnings: result.warnings.map(&:message)
      }
    else
      # Error-tolerant parsing still provides partial AST
      method_calls = []
      if result.value
        visitor = MethodCallVisitor.new(method_calls)  
        result.value.accept(visitor)
      end
      
      {
        calls: method_calls,
        errors: result.errors.map { |e| 
          { message: e.message, line: e.location.start_line }
        },
        partial: true
      }
    end
  end
  
  class MethodCallVisitor < Prism::Visitor
    def initialize(method_calls)
      @method_calls = method_calls
    end
    
    def visit_call_node(node)
      @method_calls << {
        name: node.name,
        receiver: node.receiver&.slice,
        location: {
          line: node.location.start_line,
          column: node.location.start_column
        },
        arguments: node.arguments&.arguments&.length || 0
      }
      super
    end
  end
end

Tool-Specific Migration Patterns

Different tools require specialized migration approaches. RuboCop's migration to Prism involved extensive collaboration to ensure compatibility while achieving significant performance improvements.

# Linting tool migration pattern
class MigratedLinter
  def initialize(parser_type: :prism)
    @parser_type = parser_type
    setup_parser
  end
  
  def lint_file(filepath)
    source = File.read(filepath)
    
    case @parser_type
    when :prism
      lint_with_prism(source, filepath)
    when :parser_gem
      lint_with_parser_gem(source, filepath)  
    when :hybrid
      lint_with_hybrid(source, filepath)
    end
  end
  
  private
  
  def setup_parser
    case @parser_type
    when :prism
      require 'prism'
    when :parser_gem
      require 'parser/current'
    when :hybrid
      require 'prism'
      require 'parser/current'
    end
  end
  
  def lint_with_prism(source, filepath)
    result = Prism.parse(source, filepath: filepath)
    violations = []
    
    # Prism provides error-tolerant parsing
    if result.value
      visitor = LintingVisitor.new(violations, source)
      result.value.accept(visitor)
    end
    
    # Include parse errors as violations
    result.errors.each do |error|
      violations << create_violation(:syntax_error, error.location, error.message)
    end
    
    violations
  end
  
  def lint_with_parser_gem(source, filepath)
    buffer = Parser::Source::Buffer.new(filepath)
    buffer.source = source
    violations = []
    
    begin
      ast = Parser::CurrentRuby.new.parse(buffer)
      # Traditional AST processing
      process_parser_ast(ast, violations, source)
    rescue Parser::SyntaxError => e
      violations << create_violation(:syntax_error, e.diagnostic.location, e.message)
    end
    
    violations
  end
  
  def lint_with_hybrid(source, filepath)
    # Try Prism first for better error handling
    prism_result = lint_with_prism(source, filepath)
    return prism_result if prism_result.any?
    
    # Fallback to parser gem for edge cases
    lint_with_parser_gem(source, filepath)
  end
end

Version Support Matrix

Managing compatibility across Ruby versions requires careful version detection and feature support mapping.

# Version compatibility management
class ParserCompatibilityManager
  PARSER_SUPPORT_MATRIX = {
    '2.7' => { parser_gem: true, prism: false, ripper: true },
    '3.0' => { parser_gem: true, prism: false, ripper: true },
    '3.1' => { parser_gem: true, prism: false, ripper: true }, 
    '3.2' => { parser_gem: true, prism: false, ripper: true },
    '3.3' => { parser_gem: true, prism: true, ripper: true },
    '3.4' => { parser_gem: false, prism: true, ripper: true }
  }.freeze
  
  def self.recommended_parser(ruby_version = RUBY_VERSION)
    version_key = ruby_version[0..2] # "3.3"
    support = PARSER_SUPPORT_MATRIX[version_key]
    
    return :unknown unless support
    
    if support[:prism]
      :prism
    elsif support[:parser_gem]
      :parser_gem
    else
      :ripper
    end
  end
  
  def self.create_compatible_parser(ruby_version = RUBY_VERSION)
    case recommended_parser(ruby_version)
    when :prism
      PrismWrapper.new
    when :parser_gem
      ParserGemWrapper.new
    when :ripper
      RipperWrapper.new
    else
      raise "Unsupported Ruby version: #{ruby_version}"
    end
  end
  
  class PrismWrapper
    def parse(source, filepath: nil)
      result = Prism.parse(source, filepath: filepath)
      StandardizedResult.new(result.value, result.errors, result.comments)
    end
  end
  
  class ParserGemWrapper  
    def parse(source, filepath: nil)
      buffer = Parser::Source::Buffer.new(filepath || '(string)')
      buffer.source = source
      
      begin
        ast = Parser::CurrentRuby.new.parse(buffer)
        StandardizedResult.new(ast, [], [])
      rescue Parser::SyntaxError => e
        StandardizedResult.new(nil, [e], [])
      end
    end
  end
end

Serialization and Caching Migration

Prism deserialization is around 10 times faster than parsing, enabling strategies like shipping serialized versions of the standard library for faster boot speeds.

# Migrating caching strategies to leverage Prism serialization
class CachingMigration
  def initialize(cache_dir: './tmp/ast_cache')
    @cache_dir = cache_dir
    FileUtils.mkdir_p(@cache_dir)
  end
  
  def cached_parse(filepath)
    cache_key = cache_key_for(filepath)
    cached_path = File.join(@cache_dir, "#{cache_key}.prism")
    
    if cache_valid?(cached_path, filepath)
      load_from_cache(cached_path)
    else
      result = parse_and_cache(filepath, cached_path)
      result
    end
  end
  
  private
  
  def cache_key_for(filepath)
    Digest::SHA256.hexdigest("#{filepath}:#{File.mtime(filepath).to_i}")
  end
  
  def cache_valid?(cached_path, filepath)
    File.exist?(cached_path) && 
      File.mtime(cached_path) > File.mtime(filepath)
  end
  
  def load_from_cache(cached_path)
    # Prism's serialization format enables fast deserialization
    serialized = File.read(cached_path)
    Prism.load(serialized, serialized)
  end
  
  def parse_and_cache(filepath, cached_path)
    source = File.read(filepath)
    result = Prism.parse(source, filepath: filepath)
    
    # Cache successful parses using Prism serialization
    if result.success?
      serialized = Prism.dump(source, result.value)
      File.write(cached_path, serialized)
    end
    
    result
  end
end

Common Pitfalls

Parser migration introduces subtle compatibility issues, performance traps, and behavioral differences that can cause unexpected failures in production systems. Understanding these pitfalls prevents migration regressions and enables smooth transitions.

Node Structure Differences

Prism deals with offsets in bytes while the parser gem deals with offsets in characters, requiring conversion handling when building compatible ASTs. This fundamental difference affects location calculations, especially in files containing multi-byte characters.

# Pitfall: Incorrect location handling with multi-byte characters
class LocationHandlingPitfall
  def extract_method_locations_incorrect(source)
    result = Prism.parse(source)
    methods = []
    
    result.value.accept(LocationVisitor.new do |node|
      if node.is_a?(Prism::DefNode)
        # PITFALL: Using byte offsets directly
        methods << {
          name: node.name,
          byte_offset: node.location.start_offset, # Wrong for multi-byte
          line: node.location.start_line
        }
      end
    end)
    
    methods
  end
  
  def extract_method_locations_correct(source)
    result = Prism.parse(source)
    offset_cache = build_offset_cache(source)
    methods = []
    
    result.value.accept(LocationVisitor.new do |node|
      if node.is_a?(Prism::DefNode)
        # Correct: Convert byte offset to character offset
        char_offset = offset_cache.call(node.location.start_offset)
        methods << {
          name: node.name,
          char_offset: char_offset,
          line: node.location.start_line
        }
      end
    end)
    
    methods
  end
  
  private
  
  def build_offset_cache(source)
    if source.bytesize == source.length
      # ASCII-only optimization
      ->(offset) { offset }
    else
      # Multi-byte character handling
      offset_cache = []
      offset = 0
      source.each_char do |char|
        char.bytesize.times { offset_cache << offset }
        offset += 1
      end
      offset_cache << offset
      ->(byte_offset) { offset_cache[byte_offset] || offset_cache.last }
    end
  end
end

Error Handling Behavior Changes

Error tolerance differences between parsers create subtle migration issues. Prism continues parsing after syntax errors while traditional parsers halt immediately, affecting downstream error processing logic.

# Pitfall: Assuming parse failures halt processing entirely
class ErrorHandlingPitfall
  def process_files_incorrect(files)
    results = []
    
    files.each do |filepath|
      source = File.read(filepath)
      result = Prism.parse(source, filepath: filepath)
      
      # PITFALL: Assuming errors mean no AST
      if result.errors.empty?
        results << process_ast(result.value)
      else
        results << { error: "Parse failed", file: filepath }
      end
    end
    
    results
  end
  
  def process_files_correct(files)
    results = []
    
    files.each do |filepath|
      source = File.read(filepath)  
      result = Prism.parse(source, filepath: filepath)
      
      # Correct: Check for fatal errors vs warnings
      fatal_errors = result.errors.select(&:level).select { |e| e.level == :error }
      
      if result.value && fatal_errors.empty?
        # Process AST even with warnings
        ast_result = process_ast(result.value)
        results << {
          ast: ast_result,
          warnings: result.warnings.map(&:message),
          minor_errors: result.errors.reject { |e| e.level == :error }
        }
      else
        # Handle true parse failures
        results << { 
          error: "Fatal parse errors", 
          file: filepath,
          details: fatal_errors.map(&:message)
        }
      end
    end
    
    results
  end
end

Memory Management and Performance Traps

Prism provides 4-6 times performance improvement, but incorrect usage patterns can negate these benefits. Common performance traps include redundant parsing, inefficient visitor patterns, and memory leaks from retaining parse results.

# Pitfall: Redundant parsing and inefficient visitor usage
class PerformancePitfallExample
  def analyze_methods_inefficient(source)
    # PITFALL: Parsing multiple times for different analyses
    method_names = []
    method_lengths = []
    method_params = []
    
    # First parse for method names
    result1 = Prism.parse(source)
    result1.value.accept(MethodNameVisitor.new(method_names))
    
    # Second parse for method lengths - wasteful!
    result2 = Prism.parse(source)
    result2.value.accept(MethodLengthVisitor.new(method_lengths))
    
    # Third parse for parameters - very wasteful!
    result3 = Prism.parse(source)
    result3.value.accept(MethodParamVisitor.new(method_params))
    
    { names: method_names, lengths: method_lengths, params: method_params }
  end
  
  def analyze_methods_efficient(source)
    # Correct: Single parse with comprehensive visitor
    result = Prism.parse(source)
    analysis = MethodAnalysis.new
    
    result.value.accept(ComprehensiveVisitor.new(analysis))
    
    {
      names: analysis.method_names,
      lengths: analysis.method_lengths, 
      params: analysis.method_params
    }
  end
  
  class ComprehensiveVisitor < Prism::Visitor
    def initialize(analysis)
      @analysis = analysis
    end
    
    def visit_def_node(node)
      @analysis.add_method(
        name: node.name,
        length: node.location.end_line - node.location.start_line,
        param_count: node.parameters&.parameters&.length || 0
      )
      super
    end
  end
  
  class MethodAnalysis
    attr_reader :method_names, :method_lengths, :method_params
    
    def initialize
      @method_names = []
      @method_lengths = []
      @method_params = []
    end
    
    def add_method(name:, length:, param_count:)
      @method_names << name
      @method_lengths << length  
      @method_params << param_count
    end
  end
end

Compatibility Layer Limitations

The Prism::Translation::Parser compatibility layer provides seamless migration for most use cases but has limitations with advanced parser gem features. Direct manipulation of parser internals or custom processors may not translate correctly.

# Pitfall: Assuming complete API compatibility with advanced features
class CompatibilityLimitationPitfall
  def advanced_processing_incorrect
    parser = Prism::Translation::Parser.new
    
    # PITFALL: Assuming all parser gem methods work identically
    begin
      # Some advanced parser gem features may not be fully compatible
      parser.current_arg_stack  # May not exist in compatibility layer
      parser.static_env         # May behave differently
      parser.max_numparam_stack # May not be implemented
    rescue NoMethodError => e
      puts "Compatibility issue: #{e.message}"
    end
  end
  
  def advanced_processing_correct
    # Correct: Check compatibility or use direct Prism features
    if defined?(Prism::Translation::Parser)
      parser = Prism::Translation::Parser.new
      
      # Use only documented compatibility layer features
      source = File.read('example.rb')
      buffer = Parser::Source::Buffer.new('example.rb')
      buffer.source = source
      
      ast = parser.parse(buffer)
      process_standard_ast(ast)
    else
      # Fallback to direct Prism usage
      source = File.read('example.rb')
      result = Prism.parse(source, filepath: 'example.rb')
      process_prism_ast(result.value)
    end
  end
  
  private
  
  def process_standard_ast(ast)
    # Standard AST processing that works with both parsers
    collect_nodes(ast, :send)
  end
  
  def process_prism_ast(ast)
    # Direct Prism AST processing
    calls = []
    ast.accept(CallCollector.new(calls))
    calls
  end
  
  def collect_nodes(node, type)
    return [] unless node.respond_to?(:type)
    
    results = []
    results << node if node.type == type
    
    if node.respond_to?(:children)
      node.children.each do |child|
        results.concat(collect_nodes(child, type))
      end
    end
    
    results
  end
end

Version Detection and Feature Support

Assuming Prism availability or parser gem support across Ruby versions creates brittle migration code. Version detection must be robust and handle edge cases in deployment environments.

# Pitfall: Naive version detection and feature assumptions
class VersionDetectionPitfall
  def self.create_parser_incorrect
    # PITFALL: Assuming Prism is available in Ruby 3.3+
    if RUBY_VERSION >= '3.3'
      Prism::Translation::Parser.new
    else
      Parser::CurrentRuby.new
    end
  end
  
  def self.create_parser_correct
    # Correct: Robust feature detection with fallbacks
    if prism_available?
      create_prism_parser
    elsif parser_gem_available?
      create_parser_gem_parser
    elsif ripper_available?
      create_ripper_wrapper  
    else
      raise "No compatible Ruby parser available"
    end
  end
  
  def self.prism_available?
    begin
      require 'prism'
      require 'prism/translation/parser'
      true
    rescue LoadError
      false
    end
  end
  
  def self.parser_gem_available?
    begin
      require 'parser/current'
      
      # Check if parser gem supports current Ruby version
      ruby_version = RUBY_VERSION.gsub('.', '')[0..1] # "33" for 3.3
      parser_class_name = "Parser::Ruby#{ruby_version}"
      
      if Object.const_defined?(parser_class_name)
        true
      else
        # Try CurrentRuby as fallback
        defined?(Parser::CurrentRuby)
      end
    rescue LoadError
      false
    end
  end
  
  def self.ripper_available?
    begin
      require 'ripper'
      true
    rescue LoadError
      false
    end
  end
  
  private_class_method :prism_available?, :parser_gem_available?, :ripper_available?
  
  def self.create_prism_parser
    PrismParserAdapter.new
  end
  
  def self.create_parser_gem_parser
    ParserGemAdapter.new
  end
  
  def self.create_ripper_wrapper
    RipperAdapter.new
  end
end

Test Migration Challenges

Test suites often contain parser-specific assumptions about error messages, node structures, or timing that break during migration. Comprehensive test adaptation strategies prevent regression during parser migration.

# Pitfall: Parser-specific test assumptions
class TestMigrationPitfall
  # PITFALL: Hard-coded error messages and locations
  def test_syntax_error_incorrect
    source = "def incomplete_method"
    
    assert_raises(Parser::SyntaxError) do
      parser = Parser::CurrentRuby.new
      buffer = Parser::Source::Buffer.new('test.rb')
      buffer.source = source
      parser.parse(buffer)
    end
  end
  
  # Correct: Parser-agnostic error testing
  def test_syntax_error_correct
    source = "def incomplete_method"
    
    parser_adapter = ParserAdapter.create
    result = parser_adapter.parse(source, 'test.rb')
    
    assert result.has_errors?, "Expected parse errors for incomplete method"
    assert result.errors.any? { |e| e.message.include?('unexpected end') },
           "Expected error about unexpected end of input"
  end
  
  class ParserAdapter
    def self.create
      if defined?(Prism)
        PrismAdapter.new
      else
        ParserGemAdapter.new
      end
    end
  end
  
  class PrismAdapter
    def parse(source, filepath)
      result = Prism.parse(source, filepath: filepath)
      ParseResult.new(
        success: result.success?,
        ast: result.value,
        errors: result.errors,
        warnings: result.warnings
      )
    end
  end
  
  class ParserGemAdapter
    def parse(source, filepath)
      buffer = Parser::Source::Buffer.new(filepath)
      buffer.source = source
      
      begin
        ast = Parser::CurrentRuby.new.parse(buffer)
        ParseResult.new(success: true, ast: ast, errors: [], warnings: [])
      rescue Parser::SyntaxError => e
        ParseResult.new(success: false, ast: nil, errors: [e], warnings: [])
      end
    end
  end
end

Production Patterns

Production deployments require robust migration strategies, monitoring approaches, and rollback mechanisms to ensure system stability during parser transitions. These patterns address real-world concerns including performance monitoring, error handling, and gradual rollout strategies.

Gradual Rollout with Feature Flags

RuboCop's adoption of Prism involved extensive collaboration and testing to ensure compatibility while achieving performance improvements. Production rollouts benefit from similar phased approaches with comprehensive monitoring and rollback capabilities.

# Production-ready gradual rollout system
class ParserRolloutManager
  def initialize(config = {})
    @rollout_percentage = config[:rollout_percentage] || 0
    @force_parser = config[:force_parser]
    @monitoring = config[:monitoring] || ProductionMonitoring.new
    @fallback_enabled = config[:fallback_enabled] != false
  end
  
  def parse(source, filepath: nil, user_id: nil)
    parser_choice = determine_parser(user_id)
    start_time = Time.now
    
    begin
      result = case parser_choice
               when :prism
                 parse_with_prism(source, filepath)
               when :parser_gem
                 parse_with_parser_gem(source, filepath)
               when :hybrid
                 parse_with_hybrid_validation(source, filepath)
               end
      
      @monitoring.track_success(parser_choice, Time.now - start_time)
      result
      
    rescue => e
      @monitoring.track_error(parser_choice, e, {
        source_length: source.length,
        filepath: filepath,
        user_id: user_id
      })
      
      if @fallback_enabled && parser_choice != :parser_gem
        @monitoring.track_fallback(parser_choice, :parser_gem)
        parse_with_parser_gem(source, filepath)
      else
        raise
      end
    end
  end
  
  private
  
  def determine_parser(user_id)
    return @force_parser if @force_parser
    
    # Consistent hashing for gradual rollout
    if user_id
      hash = Digest::SHA256.hexdigest(user_id.to_s).to_i(16)
      percentage = hash % 100
      
      if percentage < @rollout_percentage
        :prism
      else
        :parser_gem
      end
    else
      # Random sampling for non-user requests
      Random.rand(100) < @rollout_percentage ? :prism : :parser_gem
    end
  end
  
  def parse_with_prism(source, filepath)
    result = Prism.parse(source, filepath: filepath)
    
    ParsedResult.new(
      ast: result.value,
      success: result.success?,
      errors: result.errors.map { |e| format_error(e) },
      warnings: result.warnings.map { |w| format_warning(w) },
      parser_used: :prism
    )
  end
  
  def parse_with_parser_gem(source, filepath)
    buffer = Parser::Source::Buffer.new(filepath || '(string)')
    buffer.source = source
    
    begin
      ast = Parser::CurrentRuby.new.parse(buffer)
      ParsedResult.new(
        ast: ast,
        success: true,
        errors: [],
        warnings: [],
        parser_used: :parser_gem
      )
    rescue Parser::SyntaxError => e
      ParsedResult.new(
        ast: nil,
        success: false,
        errors: [format_parser_error(e)],
        warnings: [],
        parser_used: :parser_gem
      )
    end
  end
  
  def parse_with_hybrid_validation(source, filepath)
    # Parse with both parsers for validation
    prism_result = parse_with_prism(source, filepath)
    parser_gem_result = parse_with_parser_gem(source, filepath)
    
    # Compare results and report differences
    differences = compare_results(prism_result, parser_gem_result)
    if differences.any?
      @monitoring.track_parser_difference(differences, {
        filepath: filepath,
        source_length: source.length
      })
    end
    
    # Return Prism result by default, fallback to parser gem on critical differences
    if critical_differences?(differences)
      parser_gem_result
    else
      prism_result
    end
  end
end

Performance Monitoring and Optimization

Performance benchmarks show significant improvements with Prism, but production monitoring ensures these benefits are realized in real deployments. Comprehensive metrics collection identifies performance regressions and optimization opportunities.

# Production performance monitoring system
class ParserPerformanceMonitor
  def initialize(metrics_backend = nil)
    @metrics = metrics_backend || DefaultMetrics.new
    @performance_thresholds = {
      parse_time_warning: 1.0,   # seconds
      parse_time_error: 5.0,     # seconds
      memory_growth_warning: 50, # MB
      memory_growth_error: 100   # MB
    }
  end
  
  def monitored_parse(source, filepath: nil, parser: :auto)
    gc_start = GC.stat[:total_allocated_objects]
    memory_start = get_memory_usage
    start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    
    begin
      result = perform_parse(source, filepath, parser)
      
      duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time
      memory_end = get_memory_usage
      gc_end = GC.stat[:total_allocated_objects]
      
      record_performance_metrics(
        parser: parser,
        duration: duration,
        memory_delta: memory_end - memory_start,
        gc_allocations: gc_end - gc_start,
        source_size: source.bytesize,
        success: result.success?,
        filepath: filepath
      )
      
      check_performance_thresholds(duration, memory_end - memory_start)
      
      result
      
    rescue => e
      duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time
      record_error_metrics(parser, duration, e, source.bytesize)
      raise
    end
  end
  
  def generate_performance_report(timeframe: 1.hour)
    since = Time.now - timeframe
    
    {
      parse_times: {
        prism: @metrics.percentiles('parse_duration.prism', since: since),
        parser_gem: @metrics.percentiles('parse_duration.parser_gem', since: since),
        improvement: calculate_performance_improvement(since)
      },
      memory_usage: {
        prism: @metrics.average('memory_delta.prism', since: since),
        parser_gem: @metrics.average('memory_delta.parser_gem', since: since)
      },
      error_rates: {
        prism: @metrics.error_rate('parse_errors.prism', since: since),
        parser_gem: @metrics.error_rate('parse_errors.parser_gem', since: since)
      },
      throughput: {
        prism: @metrics.count('parse_success.prism', since: since),
        parser_gem: @metrics.count('parse_success.parser_gem', since: since)
      }
    }
  end
  
  private
  
  def perform_parse(source, filepath, parser)
    case parser
    when :prism
      result = Prism.parse(source, filepath: filepath)
      StandardResult.from_prism(result)
    when :parser_gem
      buffer = Parser::Source::Buffer.new(filepath || '(string)')
      buffer.source = source
      ast = Parser::CurrentRuby.new.parse(buffer)
      StandardResult.from_parser_gem(ast)
    when :auto
      # Auto-select based on Ruby version and availability
      if prism_recommended?
        perform_parse(source, filepath, :prism)
      else
        perform_parse(source, filepath, :parser_gem)
      end
    end
  end
  
  def record_performance_metrics(parser:, duration:, memory_delta:, gc_allocations:, 
                                source_size:, success:, filepath:)
    @metrics.histogram("parse_duration.#{parser}", duration)
    @metrics.histogram("memory_delta.#{parser}", memory_delta)
    @metrics.histogram("gc_allocations.#{parser}", gc_allocations)
    @metrics.histogram("source_size_parsed.#{parser}", source_size)
    
    if success
      @metrics.increment("parse_success.#{parser}")
    else
      @metrics.increment("parse_failure.#{parser}")
    end
    
    # Track parsing performance per file type
    if filepath
      extension = File.extname(filepath)
      @metrics.histogram("parse_duration_by_ext.#{extension}.#{parser}", duration)
    end
  end
  
  def check_performance_thresholds(duration, memory_delta)
    if duration > @performance_thresholds[:parse_time_error]
      @metrics.alert("Parse time exceeded error threshold: #{duration}s")
    elsif duration > @performance_thresholds[:parse_time_warning]
      @metrics.warn("Parse time exceeded warning threshold: #{duration}s")
    end
    
    memory_delta_mb = memory_delta / (1024 * 1024)
    if memory_delta_mb > @performance_thresholds[:memory_growth_error]
      @metrics.alert("Memory growth exceeded error threshold: #{memory_delta_mb}MB")
    elsif memory_delta_mb > @performance_thresholds[:memory_growth_warning]
      @metrics.warn("Memory growth exceeded warning threshold: #{memory_delta_mb}MB")
    end
  end
  
  def get_memory_usage
    # Platform-specific memory measurement
    if defined?(GC.stat)
      GC.stat[:heap_allocated_pages] * GC::INTERNAL_CONSTANTS[:HEAP_PAGE_SIZE]
    else
      0 # Fallback for unsupported platforms
    end
  end
  
  def calculate_performance_improvement(since)
    prism_avg = @metrics.average('parse_duration.prism', since: since)
    parser_gem_avg = @metrics.average('parse_duration.parser_gem', since: since)
    
    if prism_avg && parser_gem_avg && parser_gem_avg > 0
      ((parser_gem_avg - prism_avg) / parser_gem_avg * 100).round(2)
    else
      nil
    end
  end
end

Error Recovery and Fallback Strategies

Production systems require robust error handling that maintains service availability during parser failures or compatibility issues. Error recovery patterns ensure graceful degradation while preserving debugging information.

# Production error recovery and fallback system
class ProductionParserWrapper
  def initialize(primary_parser: :prism, fallback_chain: [:parser_gem, :ripper])
    @primary_parser = primary_parser
    @fallback_chain = fallback_chain
    @error_tracker = ProductionErrorTracker.new
    @circuit_breaker = CircuitBreaker.new(
      failure_threshold: 10,
      recovery_timeout: 300,
      recovery_threshold: 3
    )
  end
  
  def safe_parse(source, filepath: nil, context: {})
    parsers_to_try = [@primary_parser] + @fallback_chain
    last_error = nil
    
    parsers_to_try.each do |parser_type|
      next if @circuit_breaker.open?(parser_type)
      
      begin
        result = @circuit_breaker.call(parser_type) do
          parse_with_parser(source, filepath, parser_type)
        end
        
        # Success - record and return
        @error_tracker.record_success(parser_type, context)
        return enhance_result(result, parser_type, fallback_used: parser_type != @primary_parser)
        
      rescue => e
        last_error = e
        @error_tracker.record_failure(parser_type, e, {
          source_preview: source[0..200],
          filepath: filepath,
          **context
        })
        
        # Continue to next parser in fallback chain
        next
      end
    end
    
    # All parsers failed - return error result
    @error_tracker.record_total_failure(last_error, {
      attempted_parsers: parsers_to_try,
      source_preview: source[0..200],
      filepath: filepath,
      **context
    })
    
    ErrorResult.new(last_error, attempted_parsers: parsers_to_try)
  end
  
  def health_check
    {
      primary_parser: @primary_parser,
      circuit_breakers: @circuit_breaker.status,
      error_rates: @error_tracker.recent_error_rates,
      fallback_usage: @error_tracker.fallback_statistics,
      recommendations: generate_health_recommendations
    }
  end
  
  private
  
  def parse_with_parser(source, filepath, parser_type)
    case parser_type
    when :prism
      parse_with_prism(source, filepath)
    when :parser_gem
      parse_with_parser_gem(source, filepath)
    when :ripper
      parse_with_ripper(source, filepath)
    else
      raise ArgumentError, "Unknown parser type: #{parser_type}"
    end
  end
  
  def parse_with_prism(source, filepath)
    result = Prism.parse(source, filepath: filepath)
    
    # Convert to standardized result format
    ProductionResult.new(
      success: result.success?,
      ast: result.value,
      errors: result.errors.map { |e| format_prism_error(e) },
      warnings: result.warnings.map { |w| format_prism_warning(w) },
      metadata: {
        parser: :prism,
        version: Prism::VERSION,
        error_tolerant: true
      }
    )
  end
  
  def parse_with_parser_gem(source, filepath)
    buffer = Parser::Source::Buffer.new(filepath || '(string)')
    buffer.source = source
    
    begin
      ast = Parser::CurrentRuby.new.parse(buffer)
      
      ProductionResult.new(
        success: true,
        ast: ast,
        errors: [],
        warnings: [],
        metadata: {
          parser: :parser_gem,
          version: Parser::VERSION,
          error_tolerant: false
        }
      )
    rescue Parser::SyntaxError => e
      ProductionResult.new(
        success: false,
        ast: nil,
        errors: [format_parser_gem_error(e)],
        warnings: [],
        metadata: {
          parser: :parser_gem,
          version: Parser::VERSION,
          error_tolerant: false
        }
      )
    end
  end
  
  def parse_with_ripper(source, filepath)
    begin
      ast = Ripper.sexp(source, filepath)
      
      ProductionResult.new(
        success: ast != nil,
        ast: ast,
        errors: ast ? [] : [{ message: "Ripper parsing failed", level: :error }],
        warnings: [],
        metadata: {
          parser: :ripper,
          version: RUBY_VERSION,
          error_tolerant: false
        }
      )
    rescue => e
      ProductionResult.new(
        success: false,
        ast: nil,
        errors: [{ message: e.message, level: :error, exception: e.class.name }],
        warnings: [],
        metadata: {
          parser: :ripper,
          version: RUBY_VERSION,
          error_tolerant: false
        }
      )
    end
  end
  
  def enhance_result(result, parser_used, fallback_used:)
    result.metadata[:actual_parser] = parser_used
    result.metadata[:fallback_used] = fallback_used
    result.metadata[:timestamp] = Time.now.utc
    result
  end
  
  def generate_health_recommendations
    error_stats = @error_tracker.recent_error_rates
    recommendations = []
    
    if error_stats[:prism] && error_stats[:prism] > 0.05
      recommendations << "High Prism error rate (#{(error_stats[:prism] * 100).round(1)}%) - consider investigating"
    end
    
    if @error_tracker.fallback_usage[:parser_gem] > 0.1
      recommendations << "High fallback usage (#{(@error_tracker.fallback_usage[:parser_gem] * 100).round(1)}%) - primary parser may have issues"
    end
    
    if @circuit_breaker.open?(:prism)
      recommendations << "Prism circuit breaker is open - service is degraded"
    end
    
    recommendations
  end
end

Deployment and Rollback Strategies

Safe production deployment requires comprehensive rollback mechanisms, configuration management, and monitoring integration to ensure service reliability during parser migrations.

# Production deployment management for parser migration
class ParserDeploymentManager
  def initialize(config_backend = nil)
    @config = config_backend || ConfigurationBackend.new
    @deployment_tracker = DeploymentTracker.new
    @health_monitor = HealthMonitor.new
  end
  
  def deploy_parser_configuration(config)
    deployment_id = SecureRandom.uuid
    
    begin
      @deployment_tracker.start_deployment(deployment_id, config)
      
      # Pre-deployment validation
      validate_configuration(config)
      
      # Gradual rollout with monitoring
      rollout_phases = [
        { name: 'canary', percentage: 1, duration: 300 },
        { name: 'early_adopters', percentage: 10, duration: 600 },
        { name: 'general_rollout', percentage: 50, duration: 1800 },
        { name: 'full_deployment', percentage: 100, duration: 0 }
      ]
      
      rollout_phases.each do |phase|
        deploy_phase(deployment_id, config, phase)
        
        unless phase[:duration] == 0
          monitor_health_during_phase(deployment_id, phase)
        end
      end
      
      @deployment_tracker.complete_deployment(deployment_id, :success)
      
    rescue => e
      @deployment_tracker.complete_deployment(deployment_id, :failed, error: e)
      rollback_deployment(deployment_id)
      raise
    end
  end
  
  def rollback_to_previous_configuration
    previous_config = @deployment_tracker.get_previous_stable_config
    
    if previous_config
      emergency_deployment = {
        parser_type: previous_config[:parser_type],
        rollout_percentage: 100,
        fallback_enabled: true,
        reason: 'Emergency rollback'
      }
      
      deploy_parser_configuration(emergency_deployment)
    else
      raise "No previous stable configuration found for rollback"
    end
  end
  
  private
  
  def validate_configuration(config)
    required_fields = [:parser_type, :rollout_percentage]
    missing_fields = required_fields - config.keys
    
    if missing_fields.any?
      raise ArgumentError, "Missing required configuration fields: #{missing_fields}"
    end
    
    unless (0..100).include?(config[:rollout_percentage])
      raise ArgumentError, "Rollout percentage must be between 0 and 100"
    end
    
    unless [:prism, :parser_gem, :hybrid].include?(config[:parser_type])
      raise ArgumentError, "Invalid parser type: #{config[:parser_type]}"
    end
    
    # Validate parser availability
    case config[:parser_type]
    when :prism
      validate_prism_availability
    when :parser_gem
      validate_parser_gem_availability
    end
  end
  
  def deploy_phase(deployment_id, config, phase)
    phase_config = config.merge(
      rollout_percentage: phase[:percentage],
      deployment_phase: phase[:name]
    )
    
    @config.update_configuration(phase_config)
    @deployment_tracker.record_phase(deployment_id, phase[:name], phase_config)
    
    puts "Deployed phase '#{phase[:name]}' with #{phase[:percentage]}% rollout"
  end
  
  def monitor_health_during_phase(deployment_id, phase)
    monitoring_start = Time.now
    monitoring_end = monitoring_start + phase[:duration]
    
    while Time.now < monitoring_end
      health_status = @health_monitor.check_parser_health
      
      if health_status[:critical_issues].any?
        raise DeploymentError, "Critical health issues detected: #{health_status[:critical_issues]}"
      end
      
      if health_status[:warning_threshold_exceeded]
        @deployment_tracker.record_warning(deployment_id, phase[:name], health_status[:warnings])
      end
      
      sleep(30) # Check every 30 seconds
    end
    
    final_health = @health_monitor.check_parser_health
    @deployment_tracker.record_phase_completion(deployment_id, phase[:name], final_health)
  end
  
  def rollback_deployment(deployment_id)
    puts "Rolling back deployment #{deployment_id}"
    
    # Get the last stable configuration
    stable_config = @deployment_tracker.get_last_stable_config
    
    if stable_config
      @config.update_configuration(stable_config)
      @deployment_tracker.record_rollback(deployment_id, stable_config)
      puts "Rolled back to stable configuration"
    else
      # Emergency fallback to safest configuration
      emergency_config = {
        parser_type: :parser_gem,
        rollout_percentage: 100,
        fallback_enabled: true
      }
      
      @config.update_configuration(emergency_config)
      @deployment_tracker.record_emergency_rollback(deployment_id, emergency_config)
      puts "Applied emergency rollback configuration"
    end
  end
  
  def validate_prism_availability
    require 'prism'
    
    # Test parse a simple expression
    result = Prism.parse("1 + 2")
    unless result.success?
      raise "Prism validation failed: parser not working correctly"
    end
  rescue LoadError
    raise "Prism is not available in this environment"
  end
  
  def validate_parser_gem_availability
    require 'parser/current'
    
    # Test parse a simple expression  
    buffer = Parser::Source::Buffer.new('test')
    buffer.source = "1 + 2"
    ast = Parser::CurrentRuby.new.parse(buffer)
    
    unless ast
      raise "Parser gem validation failed: parser not working correctly"
    end
  rescue LoadError
    raise "Parser gem is not available in this environment"
  end
end

Reference

Core Migration Classes

Migration Methods

Common Parse Options

Error Types and Levels

Node Type Mappings

Performance Characteristics

Version Compatibility Matrix

Environment Detection Utilities

# Version and feature detection reference
PARSER_CAPABILITY_MATRIX = {
  prism_available: -> { 
    require 'prism'
    true
  rescue LoadError
    false
  },
  parser_gem_available: -> {
    require 'parser/current'
    true
  rescue LoadError
    false
  },
  prism_translation_available: -> {
    require 'prism/translation/parser'
    true
  rescue LoadError
    false
  },
  ruby_version_supports_prism: -> {
    Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('3.3.0')
  }
}.freeze

Error Handling Patterns

# Standard error handling reference patterns
def handle_parse_result(result)
  case result
  when successful_parse?(result)
    process_successful_result(result)
  when recoverable_errors?(result)
    process_with_warnings(result)
  when fatal_errors?(result)
    handle_parse_failure(result)
  end
end

def successful_parse?(result)
  result.respond_to?(:success?) ? result.success? : !result.nil?
end

def recoverable_errors?(result)
  result.respond_to?(:errors) && 
    result.errors.any? && 
    result.respond_to?(:value) && 
    result.value
end

def fatal_errors?(result)
  result.respond_to?(:errors) && 
    result.errors.any? && 
    (!result.respond_to?(:value) || result.value.nil?)
end