Skip to content

[Java] Avoid RowCoder bytecode generation - #39619

Open
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/replace-rowcoder-bytecode-generation
Open

[Java] Avoid RowCoder bytecode generation#39619
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/replace-rowcoder-bytecode-generation

Conversation

@bvolpato

@bvolpato bvolpato commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

RowCoderGenerator currently creates and loads one ByteBuddy class for every uncached schema. Generated encode and decode methods only load instance fields and call existing Java delegates, so class generation adds startup time, allocation, metaspace pressure, and class-loader complexity without specializing field encoding.

This replaces generated classes with one immutable CustomCoder<Row> implementation while preserving schema UUID caching, encoding-position overrides, component coders, and existing encode/decode algorithms. It also adds JMH coverage for uncached coder creation and steady-state encoding/decoding.

Benchmarks

JDK 17, same host, three isolated JVM forks. Generation includes schema construction; buildSchema is a control.

Benchmark master This PR Change
Uncached coder creation 5,138.469 ± 422.881 µs/op 328.614 ± 30.207 µs/op 15.6x faster
Uncached coder allocation 554,862.4 ± 2,135.1 B/op 17,926.4 ± 33.0 B/op 31.0x lower
Schema construction control 147.588 ± 22.855 µs/op 145.541 ± 24.375 µs/op unchanged

Command:

./gradlew :sdks:java:core:jmh:jmh --args='org.apache.beam.sdk.jmh.coders.RowCoderGenerationBenchmark -prof gc -f 3 -foe=true'

Steady-state results use identical benchmarks and three forks:

Operation Static encoding master ns/op This PR ns/op
decode false 59.354 ± 0.892 60.901 ± 1.137
decode true 50.046 ± 1.455 50.931 ± 0.885
encode false 61.872 ± 1.607 59.405 ± 1.554
encode true 45.537 ± 0.958 45.286 ± 1.026

All old/new confidence intervals overlap, showing no material steady-state regression.

Historical context

RowCoder generation was introduced in 2018 to replace an implementation that inspected the schema and rebuilt component coders while processing every row. The combined change was reported as 30-40% faster. That comparison was against the former introspecting implementation, not against an ordinary coder with the same precomputed fields used here.

Over time, component-coder construction moved into regular Java and the resulting array was passed into the generated class. Its encode and decode methods remained forwarding wrappers around shared loops. Precomputing field coders is still valuable and remains unchanged in this PR; generating a class no longer provides that benefit. Current benchmarks show equivalent steady-state performance, while class generation adds substantial creation time and allocation.

Wire compatibility and testing

Encoding and decoding algorithms are unchanged. Java passes Beam's exact standard-coder vectors, including beam:coder:row:v1 vectors produced by Python.

  • ./gradlew :runners:java-fn-execution:test --tests org.apache.beam.runners.fnexecution.wire.CommonCoderTest (227 passed)
  • ./gradlew :sdks:java:core:test (full core suite passed; RowCoderTest 28/28 and SchemaCoderTest 19/19)
  • ./gradlew :sdks:java:core:spotlessApply :sdks:java:core:jmh:spotlessApply
  • Old/new RowCoderBenchmark JMH runs for dynamic and static encodings

  • No existing issue to link.
  • No CHANGES.md entry; internal performance change with unchanged API and wire format.
  • Small contribution; ICLA not applicable.

@github-actions github-actions Bot added the java label Aug 4, 2026
@bvolpato
bvolpato marked this pull request as ready for review August 5, 2026 00:04
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant