Skip to content

Optimize URI.encode_www_form/1 and RFC 3986 query encoding - #15736

Merged
josevalim merged 1 commit into
elixir-lang:mainfrom
NelsonVides:uri-encode-loop
Aug 12, 2026
Merged

Optimize URI.encode_www_form/1 and RFC 3986 query encoding#15736
josevalim merged 1 commit into
elixir-lang:mainfrom
NelsonVides:uri-encode-loop

Conversation

@NelsonVides

@NelsonVides NelsonVides commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Split out of #15733 — the loop rewrite only, without the SWAR fast path, details in the commit message.

Benchmark results, OTP 29 / MacBook M4:

Input encode_www_form/1 rfc3986 encoding
empty 63.9 → 6.3 · 10.15× 66.2 → 6.6 · 10.07×
safe, 6 B 191.4 → 96.4 · 1.98× 169.3 → 95.4 · 1.77×
safe, 70 B 1422 → 376 · 3.78× 1126 → 372 · 3.02×
mixed, 70 B 1866 → 556 · 3.36× 1594 → 622 · 2.56×
safe, 700 B 16187 → 3268 · 4.95× 11521 → 3275 · 3.52×
mixed, 700 B 19337 → 4938 · 3.92× 16170 → 5933 · 2.73×

Allocation is now flat regardless of input size: 1.74 KB -> 104 B at 70 bytes, 16.51 KB -> 104 B at 700 bytes, and 104 B -> 0 B for empty. Reductions 3.51 K -> 0.70 K (safe 700 B).

Benchmark, Claude generated:

Mix.install([{:benchee, "~> 1.5"}])

defmodule Old do
  ...
end

defmodule New do
  ...
end

safe_70 = String.duplicate("abcdefg", 10)
mixed_70 = String.duplicate("abc /?~", 10)

inputs = %{
  "empty" => "",
  "safe, 6 bytes" => "abcdef",
  "safe, 7 bytes" => "abcdefg",
  "safe, 70 bytes" => safe_70,
  "mixed, 70 bytes" => mixed_70,
  "safe, 700 bytes" => String.duplicate(safe_70, 10),
  "mixed, 700 bytes" => String.duplicate(mixed_70, 10)
}

options = [
  inputs: inputs,
  time: 3,
  warmup: 1,
  memory_time: 1,
  reduction_time: 1,
  print: [fast_warning: false]
]

IO.puts("\n=== encode_www_form/1 ===\n")

Benchee.run(
  %{
    "old (comprehension)" => &Old.encode_www_form/1,
    "new (tail-recursive loop)" => &New.encode_www_form/1
  },
  options
)

IO.puts("\n=== rfc3986 unreserved encoding (per key/value in encode_query/2) ===\n")

Benchee.run(
  %{
    "old (comprehension)" => &Old.encode_rfc3986/1,
    "new (tail-recursive loop)" => &New.encode_rfc3986/1
  },
  options
)

Both encoders were built on a binary comprehension whose body calls
`percent/2` through a captured predicate:

    for <<byte <- string>>, into: "" do
      case percent(byte, &char_unreserved?/1) do
        "%20" -> "+"
        percent -> percent
      end
    end

`for ... into: ""` over a binary generator compiles to a native Erlang
bitstring comprehension, so the comprehension itself is fine, but in the
body it pays per byte a `make_fun3` to build the captured predicate, a
call to `percent/2` plus an indirect call through it, a freshly
allocated one- or three-byte binary that is discarded immediately, an
`is_eq_exact` against the literal "%20", and a runtime-sized
`bs_create_bin` to append the result.

Rewrite it as a tail-recursive loop that writes the escape straight into
the accumulator. The unreserved test moves into a `defguardp` so it
inlines instead of going through a closure, `+` is handled by its own
clause rather than by comparing the produced binary against "%20", and
the appended segments have compile-time-known sizes.
@josevalim
josevalim merged commit f316902 into elixir-lang:main Aug 12, 2026
14 of 15 checks passed
@josevalim

Copy link
Copy Markdown
Member

💚 💙 💜 💛 ❤️

@NelsonVides
NelsonVides deleted the uri-encode-loop branch August 12, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants