[FLINK-40168][table] Thread the EARLY_FIRE hint into the interval join - #8
Draft
weiqingy wants to merge 32 commits into
Draft
[FLINK-40168][table] Thread the EARLY_FIRE hint into the interval join#8weiqingy wants to merge 32 commits into
weiqingy wants to merge 32 commits into
Conversation
weiqingy
changed the title
…h-down source Wire the existing source.sleep-after-elements / source.sleep-time options into TestValuesScanTableSourceWithWatermarkPushDown Generated-By: Claude Opus 4.8 (1M context)
…st framework Lets restore tests take the stop-with-savepoint at a point defined by an input-side signal rather than sink output, so operators that emit nothing at the point of interest can be captured. * RestoreTestBase: extract the trigger into an overridable awaitSavepointReady (default unchanged: waits for sinks to reach their before-restore rows) and retry stop-with-savepoint while the job is not yet fully running. * TestValues watermark-push-down NewSource: add a per-table emission barrier (TestValuesTableFactory#awaitSourceEmitted) completed as rows are emitted. Generated-By: Claude Opus 4.8 (1M context)
Adds end-to-end coverage for the LATERAL SNAPSHOT processing-time temporal join: * LateralSnapshotJoinSemanticTests: Semantic tests LATERAL SNAPSHOT join * LateralSnapshotJoinITCase: non-deterministic result tests and tests over HEAP and ROCKSDB backends * LateralSnapshotJoinRestoreTest / LateralSnapshotJoinTestPrograms: savepoint restore tests * Configure UTC as local timezone for CommonSemanticTestBase and RestoreTestBase Generated-By: Claude Opus 4.8 (1M context)
…ode (apache#28763) In batch, all input is bounded and append-only, so the processing-time LATERAL SNAPSHOT join degenerates to a regular join of the probe side against the (final) build side; the SNAPSHOT-specific arguments are dropped. BatchPhysicalLateralSnapshotJoinRule converts the logical snapshot join into a shuffle hash join that builds the (smaller) SNAPSHOT side, mirroring StreamPhysicalLateralSnapshotJoinRule. Generated-By: Claude Opus 4.8 (1M context)
…led (apache#28675) * [FLINK-40079][table] Reject PTF calls with sys-args if they are disabled * Add a check in SqlValidator to reject PTF calls with system-args (on_time, uid) in SQL querys if the function disabled them. * Add a check in ResolveCallByArgumentsRule to reject system-args in functions that disabled them from Table API. Generated-By: Claude Opus 4.8 (1M context)
…oMultisetCastRule` should check for null values in runtime This closes apache#28777.
weiqingy
force-pushed
the
FLINK-36953-pr1b-thread
branch
2 times, most recently
from
July 21, 2026 04:43
7dc4d49 to
7502fc0
Compare
With unaligned checkpoints + interruptible timers, an operator's output watermark could stall for hours (surviving restarts) because it only advances once an entire due-timer backlog drains in one uninterrupted pass — a large backlog (e.g. after a rescale) can outlast every single attempt. InternalTimerServiceImpl/InternalTimeServiceManagerImpl now track the highest watermark known to be fully fired even when interrupted partway, and MailboxWatermarkProcessor emits that as an intermediate watermark instead of withholding all progress. This progress lives in a new field, not currentWatermark, since currentWatermark's eager semantics are relied on elsewhere (WindowOperator cleanup timers, user ProcessFunctions). Emission is paced by a configurable interval (default 5s, 0 disables) via an internal no-op processing-time nudge, avoiding per-timer clock checks.
* [FLINK-40131][docs] Document LATERAL SNAPSHOT join Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
weiqingy
force-pushed
the
FLINK-36953-pr1b-thread
branch
3 times, most recently
from
July 22, 2026 05:10
96ba48f to
647116f
Compare
…sts based on release-2.3.0 (apache#28785)
apache#28803) TimeIndicatorRelDataType.getOriginalType() does not reliably provide the correct nullability. The planner might have changed the nullability on the time indicator type without updating the nested original type. This change fixes the outputType computation to use the same rowtime type conversion as RelTimeIndicatorConverter. Generated-By: Claude Opus 4.8 (1M context)
…chable (apache#28775) If the current catalog is unreachable, any SQL query fails to parse - even queries that make fully-qualified accesses to catalogs that are reachable. This is because we make a call to databaseExists in the current catalog as part of parsing the statement. This commit wraps this in a try..catch so it doesn't block the remainder of the parsing. Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>
…nOperation This closes apache#28036.
…rsion PARSE_JSON accepted JSON numbers outside the double range, such as 1e400, and silently stored them as +/-Infinity. Variant.toJson() then emitted bare Infinity/-Infinity tokens, which are invalid JSON and cannot be parsed back by PARSE_JSON, so the round trip was broken. parseFloatingPoint now rejects a non-finite result from getDoubleValue() with a clear parse error. PARSE_JSON surfaces the failure and TRY_PARSE_JSON returns NULL, so a parsed Variant can never hold a non-finite value. As a defensive safeguard for the builder API, which can still inject non-finite values, toJson() now throws for non-finite DOUBLE and FLOAT values instead of emitting invalid tokens.
Cover the end-to-end SQL wiring for PARSE_JSON and TRY_PARSE_JSON: a JSON_STRING round trip, NULL handling, and the out-of-range number behavior. An overflowing number such as 1e400 makes PARSE_JSON fail with a TableRuntimeException, while TRY_PARSE_JSON returns NULL. The parsing semantics themselves stay covered by BinaryVariantInternalBuilderTest. This closes apache#28808.
weiqingy
force-pushed
the
FLINK-36953-pr1b-thread
branch
from
July 25, 2026 02:10
647116f to
a856401
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the FLIP-497 stack under umbrella FLINK-36953. Stacked on PR-1a. Landing order: PR-1a → PR-1b (this) → PR-2 → PR-3 → PR-4 → PR-5 → PR-6 → PR-7.
What is the purpose of the change
Threads the (already-registered)
EARLY_FIREhint through the planner to the ExecNode. The rule reads the hint, resolves the effective time mode from the join's time domain, validates the domain combinations, and threads the delay/time mode intoStreamExecIntervalJoinas NON_NULL JSON fields. The operator receives the parameters but ignores them — runtime behavior lands in PR-4.Brief change log
StreamPhysicalIntervalJoinRulereads the hint, resolves the effective time mode, rejects row-time triggering on a processing-time join, and rejects (for now) processing-time triggering on an event-time join.earlyFireDelay/earlyFireTimeModethroughStreamPhysicalIntervalJoinintoStreamExecIntervalJoinas NON_NULL JSON fields.Verifying this change
This change added tests and can be verified as follows:
IntervalJoinTest:earlyFireDelay/earlyFireTimeModereach the exec plan; row-time-on-proctime and processing-time-on-rowtime are rejected.Does this pull request potentially affect one of the following parts:
@Public(Evolving): noDocumentation
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Anthropic)