Skip to content

[ML] Fix analytics steps failing when not all platforms are built#3003

Open
edsavage wants to merge 1 commit intoelastic:mainfrom
edsavage:fix/dynamic-analytics-step-deps
Open

[ML] Fix analytics steps failing when not all platforms are built#3003
edsavage wants to merge 1 commit intoelastic:mainfrom
edsavage:fix/dynamic-analytics-step-deps

Conversation

@edsavage
Copy link
Contributor

@edsavage edsavage commented Mar 19, 2026

Summary

  • The check_build_regression, ingest_build_timings, and analyze_build_timings pipeline steps had hardcoded depends_on entries for all four platform build step keys. When a PR build only builds a subset of platforms (controlled by CI labels or comments), the missing step keys cause Buildkite to reject the pipeline upload with "Step dependencies not found".
  • Fix by having the pipeline generators (pipeline.json.py and branch.json.py) compute which step keys will actually exist and pass them as ML_BUILD_STEP_KEYS / ML_TEST_STEP_KEYS environment variables. The shell scripts then dynamically emit only the depends_on entries for steps that exist in the current build.

Test plan

  • Verified pipeline.json.py outputs correct ML_BUILD_STEP_KEYS for default (all platforms) config
  • Verified pipeline.json.py outputs only build_test_linux-x86_64-RelWithDebInfo when restricted to ci:build-linux,ci:build-x86_64
  • Verified check_build_regression.yml.sh emits correct YAML with single key
  • Verified check_build_regression.yml.sh emits correct YAML with all keys
  • Verified ingest_build_timings.yml.sh emits correct YAML
  • Verified analyze_build_timings.yml.sh emits correct YAML
  • Verified branch.json.py outputs both ML_BUILD_STEP_KEYS and ML_TEST_STEP_KEYS
  • CI passes on this PR

Made with Cursor

The check_build_regression, ingest_build_timings, and
analyze_build_timings pipeline steps had hardcoded depends_on entries
for all four platform build step keys.  When a PR build only builds a
subset of platforms (controlled by CI labels or comments), the missing
step keys cause Buildkite to reject the pipeline upload with "Step
dependencies not found".

Fix by having the pipeline generators (pipeline.json.py and
branch.json.py) compute which step keys will actually exist and pass
them as ML_BUILD_STEP_KEYS / ML_TEST_STEP_KEYS environment variables.
The shell scripts then emit depends_on entries dynamically from these
env vars.

Made-with: Cursor
@edsavage edsavage added the :ml label Mar 19, 2026
@prodsecmachine
Copy link

prodsecmachine commented Mar 19, 2026

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Buildkite pipeline uploads failing when PR/branch builds only include a subset of platform build/test steps by ensuring analytics steps only depend on step keys that actually exist for the current build configuration.

Changes:

  • Compute the set of platform build/test step keys in the pipeline generators and expose them via ML_BUILD_STEP_KEYS / ML_TEST_STEP_KEYS environment variables.
  • Update analytics pipeline upload scripts to dynamically emit depends_on entries from those environment variables instead of hardcoding all platforms.
  • Quote heredocs in the affected .yml.sh generators to prevent unintended variable expansion while generating YAML.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
.buildkite/pipelines/ingest_build_timings.yml.sh Dynamically generates depends_on from ML_BUILD_STEP_KEYS to avoid missing dependency keys.
.buildkite/pipelines/check_build_regression.yml.sh Dynamically generates depends_on from ML_BUILD_STEP_KEYS to avoid missing dependency keys.
.buildkite/pipelines/analyze_build_timings.yml.sh Dynamically generates depends_on from ML_TEST_STEP_KEYS to avoid missing dependency keys.
.buildkite/pipeline.json.py Computes ML_BUILD_STEP_KEYS for PR builds and sets it in pipeline env.
.buildkite/branch.json.py Computes ML_BUILD_STEP_KEYS and ML_TEST_STEP_KEYS for branch builds and sets them in pipeline env.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

edsavage added a commit to edsavage/ml-cpp that referenced this pull request Mar 20, 2026
Apply the same fix as PR elastic#3003 to the analyze_build_failure step:
compute which build step keys will exist based on the platform config
and pass them as ML_BUILD_STEP_KEYS for the shell script to use in
its depends_on section.  This prevents "Step dependencies not found"
errors when not all platforms are built.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants