Skip to content

[SPARK-57546][K8S] Use eclipse-temurin to use Ubuntu 26.04 and Python 3.14 by default#56607

Closed
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-57546
Closed

[SPARK-57546][K8S] Use eclipse-temurin to use Ubuntu 26.04 and Python 3.14 by default#56607
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-57546

Conversation

@dongjoon-hyun

@dongjoon-hyun dongjoon-hyun commented Jun 19, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Like Apache Spark Docker repository, this PR aims to use eclipse-temurin to use Ubuntu 26.04 and Python 3.14 by default for Apache Spark 4.3.0.

https://github.com/apache/spark-docker/blob/75f9e755807c67794776276d8f5e3e2ceedfa82b/4.1.2/scala2.13-java17-ubuntu/Dockerfile#L17

Why are the changes needed?

Apache Spark 4.3.0 dropped Python 3.10.

However, azul/zulu-openjdk only supports Ubuntu 22.04 which doesn't support Python 3.11 officially. It only have Python 3.11.0rc1.

$ docker run -it --rm azul/zulu-openjdk:25 bash
root@3c66cb6be8be:/# cat /etc/os-release | grep VERSION_ID
VERSION_ID="22.04"
root@3c66cb6be8be:/# apt-get update
root@3c66cb6be8be:/# apt-get install -y python3.11
root@3c66cb6be8be:/# python3.11 --version
Python 3.11.0rc1

Does this PR introduce any user-facing change?

Note that previously we support this new combination by the following.

$ bin/docker-image-tool.sh -b java_image_name=eclipse-temurin build

A user can switch back manually in the same way.

$ bin/docker-image-tool.sh -b java_image_name=azul/zulu-openjdk build

Although Apache Spark behavior is not changed because it's orthogonal to the underlying JDK, OS and Python versions, yes, the underlying OS becomes Ubuntu 26.04 and Python becomes 3.14 and Java vendor becomes Eclipse Temurin. This is inevitable due to the lack of Python 3.11 support of Zulu JDK.

AFTER

$ docker run -it --rm eclipse-temurin:25-jre bash
root@de506cc01b8a:/# cat /etc/os-release | grep VERSION_ID
VERSION_ID="26.04"
root@de506cc01b8a:/# apt-get update
root@de506cc01b8a:/# apt-get install -y python3
root@de506cc01b8a:/# python3 --version
Python 3.14.4

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

cc @sarutak , @yaooqinn , @LuciferYang

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Thank you, @sarutak , @HyukjinKwon , @LuciferYang !

dongjoon-hyun added a commit that referenced this pull request Jun 19, 2026
…ython 3.14` by default

Like `Apache Spark Docker` repository, this PR aims to use `eclipse-temurin` to use `Ubuntu 26.04` and `Python 3.14` by default for Apache Spark 4.3.0.

https://github.com/apache/spark-docker/blob/75f9e755807c67794776276d8f5e3e2ceedfa82b/4.1.2/scala2.13-java17-ubuntu/Dockerfile#L17

Apache Spark 4.3.0 dropped Python 3.10.
- #55914

However, `azul/zulu-openjdk` only supports `Ubuntu 22.04` which doesn't support Python 3.11 officially. It only have `Python 3.11.0rc1`.

```
$ docker run -it --rm azul/zulu-openjdk:25 bash
root3c66cb6be8be:/# cat /etc/os-release | grep VERSION_ID
VERSION_ID="22.04"
root3c66cb6be8be:/# apt-get update
root3c66cb6be8be:/# apt-get install -y python3.11
root3c66cb6be8be:/# python3.11 --version
Python 3.11.0rc1
```

Note that previously we support this new combination by the following.
```
$ bin/docker-image-tool.sh -b java_image_name=eclipse-temurin build
```

A user can switch back manually in the same way.
```
$ bin/docker-image-tool.sh -b java_image_name=azul/zulu-openjdk build
```

Although Apache Spark behavior is not changed because it's orthogonal to the underlying JDK, OS and Python versions, yes, the underlying OS becomes `Ubuntu 26.04` and Python becomes `3.14` and Java vendor becomes `Eclipse Temurin`. This is inevitable due to the lack of Python 3.11 support of `Zulu JDK`.

**AFTER**
```
$ docker run -it --rm eclipse-temurin:25-jre bash
rootde506cc01b8a:/# cat /etc/os-release | grep VERSION_ID
VERSION_ID="26.04"
rootde506cc01b8a:/# apt-get update
rootde506cc01b8a:/# apt-get install -y python3
rootde506cc01b8a:/# python3 --version
Python 3.14.4
```

Pass the CIs.

Generated-by: Claude Code

Closes #56607 from dongjoon-hyun/SPARK-57546.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 880083f)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Merged to master/4.x.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-57546 branch June 19, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants