chore(flink): align the container runtime with the pinned 2.3.0 extra - #168
Merged
Conversation
pyproject pinned apache-flink==2.3.0 while every container ran 2.2.1 (flink_jobs image ARG + both flink:2.2.1-java17 cluster images), so local development could exercise behaviour absent from the deployed minor. Runtime moves to 2.3.0. The Kafka connector stays 5.0.0-2.2: the externalised connector is versioned <connector>-<flink minor> and no -2.3 artifact is published; @public API is stable within a Flink major, and flink-smoke submits the real job to a real cluster on every PR, so the pairing is asserted, not assumed. Documented at the ARG. docs/architecture.md Technology Choices reads Flink 2.3. Historical records keep the version they were measured on. Closes F2 of the 07.07 audit (plan_07_07_26.md, S2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DORA Metrics
MTTR note: No failed mainline CI runs in the selected window. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Audit
audit_07_07_26.mdF2:pyproject.tomlpinsapache-flink==2.3.0, but every container ran 2.2.1 (src/processing/flink_jobs/DockerfileARG +flink:2.2.1-java17indocker-compose.yml). The runtime was internally consistent, but a developer working against the 2.3.0 PyFlink could hit behaviour that does not exist on the deployed minor.What
FLINK_VERSIONARG2.2.1→2.3.0(drives the dist download, theflink-s3-fs-hadoopplugin symlink, and the in-imagepip install apache-flink==${FLINK_VERSION}).flink:2.3.0-java17.docs/architecture.mdTechnology Choices → Flink 2.3.[Unreleased](also records the S1 SDK lint gate that landed in chore(ci): put sdk/ under the ruff lint+format gate #167).The one judgement call: the Kafka connector
flink-sql-connector-kafkais versioned<connector>-<flink minor>, and its newest published artifact is5.0.0-2.2— there is no-2.3build on Maven Central (checked againstmaven-metadata.xml). Flink keeps@PublicAPI stable inside a major, so the 2.2 build runs on a 2.3 cluster. Rather than assume that, the pin stays at5.0.0-2.2, the reason is written at the ARG, andflink-smokesubmitsstream_processor.pyto a real Kafka + MinIO + Flink cluster on this PR — that job accepting the graph and assigning a JobID is the assertion.Verify
flink-smokeon this PR (built image, real cluster) — the gate for this change. Not verified locally: Docker is not run on this workstation.flink-2.3.0-bin-scala_2.12.tgz→ HTTP 200 on archive.apache.org;flink:2.3.0-java17exists on Docker Hub.Closes F2 of the 07.07 audit (plan
plan_07_07_26.md, S2).🤖 Generated with Claude Code