Skip to content

Allow using prism as the parser engine#156

Open
kddnewton wants to merge 1 commit into
judofyr:masterfrom
kddnewton:prism
Open

Allow using prism as the parser engine#156
kddnewton wants to merge 1 commit into
judofyr:masterfrom
kddnewton:prism

Conversation

@kddnewton
Copy link
Copy Markdown

Allow consumers to use prism as the parser engine, and set it to the default if it is available to be required.

Prism ends up being quite a bit faster here, and there are a couple of nice correctness things (rationals like 1r are considered static now, for example).

I wrote a benchmark, the script is here:

benchmark.rb
# frozen_string_literal: true

# Run benchmarks for both engines by spawning subprocesses:
#   ruby benchmark.rb
#
# Run a single engine directly:
#   TEMPLE_PARSER_ENGINE=prism ruby benchmark.rb
#   TEMPLE_PARSER_ENGINE=ripper ruby benchmark.rb

if !ENV.key?("TEMPLE_PARSER_ENGINE")
  require "tempfile"

  results = {}
  %w[prism ripper].each do |engine|
    file = Tempfile.new(["benchmark_#{engine}_", ".txt"])
    file.close
    system({ "TEMPLE_PARSER_ENGINE" => engine }, RbConfig.ruby, __FILE__, out: file.path)
    results[engine] = File.read(file.path)
    file.unlink
  end

  results.each do |engine, output|
    puts "=" * 60
    puts "  #{engine.upcase}"
    puts "=" * 60
    puts output
    puts
  end

  exit
end

$LOAD_PATH.unshift(File.join(__dir__, "lib"))
require "benchmark/ips"
require "temple"
require "temple/static_analyzer"
require "temple/filters/string_splitter"
require "temple/filters/static_analyzer"

engine = ENV.fetch("TEMPLE_PARSER_ENGINE") { defined?(Prism) ? "prism" : "ripper" }
puts "Engine: #{engine}"
puts

static_expressions = ['true', 'false', '"hello world"', '[1, { 2 => 3 }]', "[\n1,\n]"]
dynamic_expressions = ['1 + 2', 'variable', 'method_call(a)', 'CONSTANT']
valid_code = ['Foo.bar.baz { |c| c.d! }', '{ foo: bar }']
invalid_code = ['Foo.bar.baz { |c| c.d! ', ' foo: bar ']
string_literals = ['"hello"', '"static#{dynamic}"', '%Q[a#{b}c#{d}e]', '"hello #{}world"', '"\#{}#{123}"', '%Q(href("#{1 + 1}");)']
filter_inputs = string_literals.map { |code| [:dynamic, code] }
analyzer_inputs = [[:dynamic, '"#{"hello"}#{100}"'], [:dynamic, '"#{hello}#{100}"'], [:dynamic, "[100,\n200,\n]"], [:dynamic, 'true'], [:dynamic, 'variable']]

Benchmark.ips do |x|
  x.report("StaticAnalyzer.static? static")  { static_expressions.each { |c| Temple::StaticAnalyzer.static?(c) } }
  x.report("StaticAnalyzer.static? dynamic") { dynamic_expressions.each { |c| Temple::StaticAnalyzer.static?(c) } }
  x.report("syntax_error? valid")            { valid_code.each { |c| Temple::StaticAnalyzer.syntax_error?(c) } }
  x.report("syntax_error? invalid")          { invalid_code.each { |c| Temple::StaticAnalyzer.syntax_error?(c) } }
  x.report("StringSplitter.compile")         { string_literals.each { |c| Temple::Filters::StringSplitter.compile(c) } }
  x.report("StringSplitter filter") do
    filter = Temple::Filters::StringSplitter.new
    filter_inputs.each { |exp| filter.call(exp) }
  end
  x.report("StaticAnalyzer filter") do
    filter = Temple::Filters::StaticAnalyzer.new
    analyzer_inputs.each { |exp| filter.call(exp) }
  end
end

On my machine it ends up being quite a bit faster.

Results
============================================================
  PRISM
============================================================
Engine: prism

ruby 4.1.0dev (2026-02-16T15:49:22Z master 38c602ea00) +PRISM [arm64-darwin23]
Warming up --------------------------------------
StaticAnalyzer.static? static
                         5.120k i/100ms
StaticAnalyzer.static? dynamic
                         6.338k i/100ms
 syntax_error? valid    65.753k i/100ms
syntax_error? invalid
                        50.021k i/100ms
StringSplitter.compile
                         2.382k i/100ms
StringSplitter filter
                         1.459k i/100ms
StaticAnalyzer filter
                         2.458k i/100ms
Calculating -------------------------------------
StaticAnalyzer.static? static
                         51.339k (± 3.5%) i/s   (19.48 μs/i) -    261.120k in   5.093414s
StaticAnalyzer.static? dynamic
                         61.660k (± 4.4%) i/s   (16.22 μs/i) -    310.562k in   5.047246s
 syntax_error? valid    662.991k (± 3.5%) i/s    (1.51 μs/i) -      3.353M in   5.065338s
syntax_error? invalid
                        504.193k (± 3.2%) i/s    (1.98 μs/i) -      2.551M in   5.065447s
StringSplitter.compile
                         23.359k (± 3.5%) i/s   (42.81 μs/i) -    119.100k in   5.105032s
StringSplitter filter
                         14.715k (± 2.4%) i/s   (67.96 μs/i) -     74.409k in   5.059881s
StaticAnalyzer filter
                         24.566k (± 1.8%) i/s   (40.71 μs/i) -    125.358k in   5.104575s

============================================================
  RIPPER
============================================================
Engine: ripper

ruby 4.1.0dev (2026-02-16T15:49:22Z master 38c602ea00) +PRISM [arm64-darwin23]
Warming up --------------------------------------
StaticAnalyzer.static? static
                         1.060k i/100ms
StaticAnalyzer.static? dynamic
                         2.126k i/100ms
 syntax_error? valid     9.598k i/100ms
syntax_error? invalid
                         9.273k i/100ms
StringSplitter.compile
                       652.000 i/100ms
StringSplitter filter
                       378.000 i/100ms
StaticAnalyzer filter
                       695.000 i/100ms
Calculating -------------------------------------
StaticAnalyzer.static? static
                         11.190k (± 4.0%) i/s   (89.36 μs/i) -     56.180k in   5.030278s
StaticAnalyzer.static? dynamic
                         20.688k (± 2.9%) i/s   (48.34 μs/i) -    104.174k in   5.039785s
 syntax_error? valid     91.207k (± 8.7%) i/s   (10.96 μs/i) -    451.106k in   5.004029s
syntax_error? invalid
                         87.513k (± 7.1%) i/s   (11.43 μs/i) -    435.831k in   5.010403s
StringSplitter.compile
                          6.163k (± 9.5%) i/s  (162.27 μs/i) -     30.644k in   5.042302s
StringSplitter filter
                          3.534k (± 9.5%) i/s  (282.99 μs/i) -     17.766k in   5.083927s
StaticAnalyzer filter
                          8.389k (± 4.5%) i/s  (119.20 μs/i) -     42.395k in   5.065375s

Allow consumers to use prism as the parser engine, and set it to
the default if it is available to be required.

Prism ends up being quite a bit faster here, and there are a couple
of nice correctness things (rationals like 1r are considered static
now, for example).
@kddnewton kddnewton mentioned this pull request Mar 16, 2026
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant