Decouple K8s pod readiness from Ray Serve application state#8
Merged
oren-openteams merged 1 commit intoMay 1, 2026
Merged
Conversation
Override KubeRay's default worker readiness/liveness probes which chain a
raylet healthz check with `wget http://localhost:8000/-/healthz | grep
success`. The Serve check requires a deployed Serve application AND a
Serve HTTP proxy on the local pod — neither holds on a fresh cluster
install (default `serveApplications: []`, default Ray Serve
`proxy_location: HeadOnly` in Ray ≥2.10), so the worker stays 0/1 Ready
indefinitely and KubeRay can't complete its rollout.
Kubernetes pod readiness should reflect whether the Ray node itself is
healthy (raylet up, resources allocated). Serve application health is
tracked separately by the Serve controller and need not gate K8s
readiness. The chart's serve-svc Service already targets only the head
pod, so worker readiness state has no effect on user-visible HTTP
routing.
Changes:
- chart/values.yaml: add `serve.proxyLocation` (default `EveryNode`),
`worker.readinessProbe`, `worker.livenessProbe`, `head.readinessProbe`,
`head.livenessProbe`. All probe fields are tunable; setting either
worker probe to null (~) suppresses it and lets KubeRay's default
apply. Head probes default to {} (KubeRay default applies); users can
set explicit probes to override.
- chart/templates/rayservice.yaml: template proxy_location from values;
conditionally render head/worker probes from values via `with`.
- chart/Chart.yaml: bump version 0.2.0 → 0.3.0.
- .github/workflows/test.yaml: bump worker.replicas from 0 to 1 and add
a wait for worker Ready condition. The previous test only ran with
zero workers, so it didn't exercise the failure mode this PR fixes.
Closes #7
e89e1fa to
5ce6bdb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #7.
The chart's worker pods stay
0/1 Readyindefinitely on a fresh install, blocking KubeRay from completing its rollout. The cause is two layered defaults:wget http://localhost:8000/-/healthz | grep success. The Serve check requires both a deployed Serve application AND a Serve HTTP proxy on the local pod.serveApplications: [](no app deployed) and no explicitproxy_locationin serveConfigV2 (Ray Serve ≥2.10 defaults toHeadOnly, so workers don't get local proxies even when an app is deployed).The result is the worker probe fails forever with the confusing message
Readiness probe failed: success— the literal word "success" is the grep match from the first half of the chained command; the pipeline exits non-zero becausewgetagainst the dead port 8000 fails.What this PR does
Overrides the head and worker readiness/liveness probes to drop the Serve healthz check. The new probes check only the raylet healthz endpoint (port 52365), which reflects whether the Ray node itself is healthy (raylet up, resources allocated). Serve application health is tracked separately by the Serve controller and shouldn't gate Kubernetes pod readiness. The chart's serve-svc Service already targets only the head pod, so worker readiness state has no effect on user-visible HTTP routing.
Adds
proxy_location: EveryNodeto serveConfigV2. With KubeRay placing each Ray node in its own pod, having a local HTTP proxy on each node is typically what users want. Users with a different intent can override.Verification
I ran
helm templateagainst the modified chart to confirm the output is well-formed YAML and the probe overrides land in the right places in the rendered RayService.Reproduce the original failure
The full diagnosis lives in #7 — short version: install the chart on a clean cluster, observe worker stays
0/1indefinitely, manually exec into the head pod and runserve.shutdown(); serve.start(proxy_location='EveryNode'); serve.run(SomeApp.bind(), ...)to make the worker become Ready. The current PR removes the need for that manual step.Considered alternatives
import_pathproblem (no stub is universally importable from arbitrary Ray images), and adds opinionated state to the cluster that users will then have to remove.Notes
initialDelaySeconds,failureThreshold, etc.) match KubeRay's defaults for those fields, just with the Serve check removed from the command.serveApplicationsand configureproxy_location: EveryNode— for them, the probe was passing anyway. It only changes behavior for fresh installs and users without a Serve app deployed.