8369950: TLS connection to IPv6 address fails with BCJSSE due to IllegalArgumentException

[rm-gh-8]

Backporting JDK-8369950: TLS connection to IPv6 address fails with BCJSSE due to IllegalArgumentException.

This PR fixes HttpsURLConnection to comply with RFC 6066 by preventing SNI hostname from being set when connecting to literal IPv4 or IPv6 addresses, resolving failures with external JSSE providers like BCJSSE that reject non-LDH ASCII characters in SNI hostnames, and aligning behavior with the existing SSLSocketImpl implementation that already skips SNI construction for IP address literals.

This is a regression that last worked in Last worked in version 17.0.16.

This is a prerequisite for JDK-8376031 (dependent PR #349), which is to be added for parity with Oracle JDK.

Ran related tests on linux-x64, linux-aarch64, macos-aarch64 and windows-x64:

make test TEST=test/jdk/javax/net/ssl/HttpsURLConnection/SubjectAltNameIP.java
make test TEST=test/jdk/javax/net/ssl/HttpsURLConnection

Results attached:

windows-x64-specific-test.log
windows-x64-specific-2-test.log
macos-aarch64-specific-test.log
macos-aarch64-specific-2-test.log
linux-x64-specific-test.log
linux-x64-specific-2-test.log
linux-aarch64-specific-test.log
linux-aarch64-specific-2-test.log

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• JDK-8369950 needs maintainer approval

Issue

• JDK-8369950: TLS connection to IPv6 address fails with BCJSSE due to IllegalArgumentException (Bug - P4 - Requested)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk25u-dev.git pull/348/head:pull/348
$ git checkout pull/348

Update a local copy of the PR:
$ git checkout pull/348
$ git pull https://git.openjdk.org/jdk25u-dev.git pull/348/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 348

View PR using the GUI difftool:
$ git pr show -t 348

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk25u-dev/pull/348.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back rm-gh-8! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

This backport pull request has now been updated with issue from the original commit.

[openjdk]

⚠️ @rm-gh-8 This change is now ready for you to apply for maintainer approval. This can be done directly in each associated issue or by using the /approval command.

[mlbridge]

Webrevs

• 00: Full (2fde6300)

[rm-gh-8]

/approval request for backport of JDK-8369950: TLS connection to IPv6 address fails with BCJSSE due to IllegalArgumentException.

This PR fixes HttpsURLConnection to comply with RFC 6066 by preventing SNI hostname from being set when connecting to literal IPv4 or IPv6 addresses, resolving failures with external JSSE providers like BCJSSE that reject non-LDH ASCII characters in SNI hostnames, and aligning behavior with the existing SSLSocketImpl implementation that already skips SNI construction for IP address literals.

This is a regression that last worked in Last worked in version 17.0.16.

This is a prerequisite for JDK-8376031 (dependent PR #349), which is to be added for parity with Oracle JDK.

Low risk - this is a standards compliance fix that resolves actual failures with external JSSE providers (BCJSSE) by
preventing invalid SNI data from being sent during TLS handshake when using IP address literals; the change only affects the code path for non-SSLSocketImpl sockets and mirrors existing behavior in SSLSocketImpl, ensuring consistency across JSSE implementations while maintaining proper RFC 6066 compliance that requires SNI hostnames to be fully qualified DNS names, not IP addresses.

[openjdk]

@rm-gh-8
8369950: The approval request has been created successfully.

[GoeLin]

HI @rm-gh-8
can you please do a dependent PR for https://bugs.openjdk.org/browse/JDK-8373676 ?
Thanks!

[rm-gh-8]

@GoeLin Created #378