Skip to content

Add payload-agent presubmit to openshift-eng/ai-helpers#81149

Open
not-stbenjam wants to merge 1 commit into
openshift:mainfrom
not-stbenjam:agent-presubmit
Open

Add payload-agent presubmit to openshift-eng/ai-helpers#81149
not-stbenjam wants to merge 1 commit into
openshift:mainfrom
not-stbenjam:agent-presubmit

Conversation

@not-stbenjam

@not-stbenjam not-stbenjam commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds an optional, always-run-false presubmit to openshift-eng/ai-helpers
  • Uses the same openshift-claude-payload workflow as the existing periodic
  • Triggerable via /test payload-agent on ai-helpers PRs

Test plan

  • Verify generated job has optional: true and always_run: false
  • Trigger with /test payload-agent on an ai-helpers PR to validate

🤖 Generated with Claude Code

Summary by CodeRabbit

This PR updates the OpenShift CI configuration for openshift-eng/ai-helpers to add a new optional presubmit job for the payload-agent variant. The job is wired to the existing openshift-claude-payload workflow, is configured with always_run: false, and can be manually triggered on openshift/release pull requests with /test payload-agent-analyze-presubmit.

In practical terms, this gives the payload-agent analysis workflow a dedicated, non-blocking presubmit path so it can be validated on demand without running on every change.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The CI config adds an optional payload-agent test step to the AI helpers job. The step runs the openshift-claude-payload workflow with PAYLOAD_TAG set to an empty string and does not run by default.

Changes

AI helpers CI test step

Layer / File(s) Summary
Optional payload-agent step
ci-operator/config/openshift-eng/ai-helpers/openshift-eng-ai-helpers-main.yaml
Adds a new optional payload-agent test step that invokes openshift-claude-payload with PAYLOAD_TAG: "" and always_run: false.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and correctly points to adding a payload-agent presubmit in openshift-eng/ai-helpers.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR only adds CI YAML; changed test files are ordinary Go tests and contain no Ginkgo titles or dynamic strings.
Test Structure And Quality ✅ Passed PR only changes ci-operator YAML; no Ginkgo test code was added or modified, so the test-quality check is not applicable.
Microshift Test Compatibility ✅ Passed Only ci-operator config and generated job YAML changed; no new Ginkgo e2e tests or OpenShift API usage were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: PR only adds a ci-operator presubmit/job using existing openshift-claude-payload workflow; no new Ginkgo e2e tests or node-topology assumptions were introduced.
Topology-Aware Scheduling Compatibility ✅ Passed This commit only changes ci-operator/payload job configs; no deployment manifests, operators, or controllers were modified, so no topology-sensitive scheduling was introduced.
Ote Binary Stdout Contract ✅ Passed PR only updates ci-operator/job YAML for ai-helpers; no OTE binary main/init/suite setup code changed, so no stdout contract violation.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR only changes ci-operator/job YAML; no new Ginkgo e2e test code or IPv4/networking assumptions were added, so this check is not applicable.
No-Weak-Crypto ✅ Passed PR only adds ci-operator YAML for a new presubmit and matching generated job; no crypto code, weak algorithms, or secret comparisons were introduced.
Container-Privileges ✅ Passed The added payload-agent job and workflow/ref files contain no privileged/root/hostNetwork/hostPID/allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed The PR only adds a presubmit config entry; it introduces no new log statements or secret-bearing output.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 26, 2026
@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Hi @not-stbenjam. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot requested review from smg247 and wking June 26, 2026 14:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
ci-operator/step-registry/openshift/claude/payload/agent/openshift-claude-payload-agent-commands.sh (1)

579-579: 🎯 Functional Correctness | 🔵 Trivial

Use line-robust jq parsing with fromjson? to avoid silent failures on malformed JSON

The current grep + strict jq pipeline fails to extract the summary if a line starting with { contains invalid JSON, causing the parser to abort. Switching to jq's raw input mode with optional JSON conversion (fromjson?) safely skips malformed lines and reliably retrieves the first valid result record.

-SUMMARY=$(grep -E '^\{' "${SLACK_LOG}" | jq -r 'select(.type == "result") | .result // empty' 2>/dev/null | head -1) || SUMMARY=""
+SUMMARY=$(jq -Rr 'fromjson? | select(.type == "result") | .result // empty' "${SLACK_LOG}" 2>/dev/null | head -1) || SUMMARY=""

Verification confirmed that with input containing a non-JSON brace line, the original approach returns empty while the proposed change correctly extracts "wanted".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/openshift/claude/payload/agent/openshift-claude-payload-agent-commands.sh`
at line 579, The summary extraction pipeline in the shell script is too brittle
because the current `grep` plus strict `jq` parsing stops on malformed
brace-prefixed lines, so update the parsing logic in the summary assignment to
use line-robust `jq` raw input with `fromjson?` and keep selecting the first
valid record with `.type == "result"` and `.result`. Make the change in the
summary extraction block identified by the `SUMMARY=` assignment so invalid JSON
lines are skipped instead of causing an empty result.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@ci-operator/step-registry/openshift/claude/payload/agent/openshift-claude-payload-agent-commands.sh`:
- Line 579: The summary extraction pipeline in the shell script is too brittle
because the current `grep` plus strict `jq` parsing stops on malformed
brace-prefixed lines, so update the parsing logic in the summary assignment to
use line-robust `jq` raw input with `fromjson?` and keep selecting the first
valid record with `.type == "result"` and `.result`. Make the change in the
summary extraction block identified by the `SUMMARY=` assignment so invalid JSON
lines are skipped instead of causing an empty result.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 31405c03-1f99-4a4c-b029-fe3586da6288

📥 Commits

Reviewing files that changed from the base of the PR and between 436ce01 and da4b36f.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/openshift/release/openshift-release-main-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (2)
  • ci-operator/config/openshift/release/openshift-release-main__payload-agent.yaml
  • ci-operator/step-registry/openshift/claude/payload/agent/openshift-claude-payload-agent-commands.sh

@stbenjam

Copy link
Copy Markdown
Member

/ok-to-test
/approve

(Automated: Extending self-approval rights and ok-to-test to my bot account.)

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 26, 2026
Add an optional, always-run-false presubmit that runs the same
payload analysis workflow as the existing periodic. Triggerable
via `/test payload-agent` on openshift-eng/ai-helpers PRs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@not-stbenjam not-stbenjam changed the title Add payload-agent-analyze presubmit job Add payload-agent presubmit to openshift-eng/ai-helpers Jun 26, 2026
@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: not-stbenjam, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@not-stbenjam: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-eng-ai-helpers-main-payload-agent openshift-eng/ai-helpers presubmit Presubmit changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift-eng/ai-helpers/openshift-eng-ai-helpers-main.yaml`:
- Around line 23-29: The new optional presubmit entry in the ci-operator config
needs its generated Prow job artifacts committed too. After updating the tests
entry in the openshift-eng-ai-helpers-main configuration, regenerate the
downstream ci-operator/jobs output using the standard CI config update flow (for
example, the make update / registry-metadata / ci-operator-config / jobs
sequence) and include those generated files in the same PR. Refer to the updated
tests stanza and the payload-agent job definition so the generated job config
stays in sync.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 07c845a6-b314-4c07-90b8-74747019f5e4

📥 Commits

Reviewing files that changed from the base of the PR and between da4b36f and 88eebfd.

📒 Files selected for processing (1)
  • ci-operator/config/openshift-eng/ai-helpers/openshift-eng-ai-helpers-main.yaml

@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

@not-stbenjam: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/generated-config 88eebfd link true /test generated-config

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants