jfrog-cli is unable to gracefully recover from connection closed by peer on Apple platforms. #3406
Description
Describe the bug
jfrog-cli is unable to gracefully recover from connection closed by peer on Apple platforms. Despite any successive retries does succeed.
Current behavior
I have a situation where a large +100Mb upload on a 10Gb network between the build-servers and the artifactory server times up during the upload (nginx and Artifactory sets HTTP 408 - Request Timed Out), the connection is closed. We believe the root cause here is some performance issues with nginx under the hood of the artifactory deployment. However, it doesn't explain the behavior for jfrog-cli (or jenkins-jfrog-plugin).
jfrog-cli is configured to retry up to 5 times with a wait-retry of 30s. That is all working fine, and will work, but
the pipeline still fails as jfrog-cli is returning 1 (should return 0). Also, it spews out 100s of messages: 11:17:38 [Error] io: read/write on closed pipe
[apple] $ /Users/buildops/tools/io.jenkins.plugins.jfrog.JfrogInstallation/jfrog-cli/jf -v
[apple] $ /Users/buildops/tools/io.jenkins.plugins.jfrog.JfrogInstallation/jfrog-cli/jf rt upload --spec ../../build/jfrog/secure-access-apple-upload.json --retries 5 --retry-wait-time 30s --spec-vars=${SPEC_VARS}
11:16:38 [Info] [Thread 2] Uploading artifact: sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Mobility.xcarchive.zip
11:17:37 [Warn] [Thread 0] (Attempt 1) - Failure occurred while uploading to https://artifactory.failsafe.net/artifactory/sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Secure%20Access.ipa;build.branch=lynyrd;build.change=365976;build.config=%7B2%7D;build.id=15976.Mar30.lynyrd;build.platform=macos;build.stamp=NONE;build.version=14.60.15976;type=ipa;build.name=Secure+Access+-+Apple+-+%28IPHONE%2C+DEBUG%29+-+lynyrd;build.number=15976.Mar30.lynyrd;build.timestamp=1774894597112: server response: 408
<!doctype html><html lang="en"><head><title>HTTP Status 408 – Request Timeout</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 408 – Request Timeout</h1></body></html>
11:17:38 [Error] io: read/write on closed pipe
11:17:38 [Warn] [Thread 2] (Attempt 1) - Failure occurred while uploading to https://artifactory.failsafe.net/artifactory/sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Mobility.xcarchive.zip;build.branch=lynyrd;build.change=365976;build.config=%7B2%7D;build.id=15976.Mar30.lynyrd;build.platform=macos;build.stamp=NONE;build.version=14.60.15976;type=macos-app;build.name=Secure+Access+-+Apple+-+%28IPHONE%2C+DEBUG%29+-+lynyrd;build.number=15976.Mar30.lynyrd;build.timestamp=1774894597112: server response: 408
<!doctype html><html lang="en"><head><title>HTTP Status 408 – Request Timeout</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 408 – Request Timeout</h1></body></html>
11:17:38 [Error] io: read/write on closed pipe
...
11:17:38 [Error] io: read/write on closed pipe
11:18:09 [Error] io: read/write on closed pipe
{
"status": "failure",
"totals": {
"success": 2,
"failure": 0
}
}
11:18:09 [Info] Trace ID for JFrog Platform logs: 5005513f40df719b
11:18:09 [Error] upload finished with errors. Review the logs for more information
Found unhandled java.lang.RuntimeException exception:
Couldn't execute 'jf' command. RuntimeException: Running 'jf' command failed with exit code 1
PluginClassLoader for jfrog//io.jenkins.plugins.jfrog.JfStep$Execution.run(JfStep.java:134)
PluginClassLoader for jfrog//io.jenkins.plugins.jfrog.JfStep$Execution.run(JfStep.java:92)
PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:49)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
java.base/java.lang.Thread.run(Thread.java:1583)
Reproduction steps
Retry a bunch of large uploads and over a faulty nginx deployment that terminates the connection
Expected behavior
The jfrog-cli should recover gracefully and return 0 despite the read/write errors on a closed peer connection. That way, my build job can continue when a retry attempt was successful.
JFrog CLI version
2.97.0
Operating system type and version
MacOS 26.2 on macmini
JFrog Artifactory version
7.133.15
JFrog Xray version
N/A