Skip to content

Add OCP 4.22 to prow#81081

Closed
tbuskey wants to merge 14 commits into
openshift:mainfrom
tbuskey:260625
Closed

Add OCP 4.22 to prow#81081
tbuskey wants to merge 14 commits into
openshift:mainfrom
tbuskey:260625

Conversation

@tbuskey

@tbuskey tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Other changes:

  • MUST_GATHER_ON_FAILURE_ONLY: "false" # so prow always runs kata must-gather
  • INSTALL_KATA_RPM: true
  • KATA_RPM_VERSION: 3.31.0-1.rhaos4.19.el9 # 4.19 -> 4.21
  • KATA_RPM_VERSION: 3.31.0-1.rhaos4.22.el9 # 4.22

KATA-5459
Previous PR Rehersal success

Summary by CodeRabbit

Expanded ci-operator/config/openshift/sandboxed-containers-operator downstream Prow configuration to add OCP 4.22 and adjust kata e2e job runtime behavior:

  • For the OCP 4.19 downstream candidates (openshift-sandboxed-containers-operator-devel__downstream-candidate419.yaml and ...__downstream-candidate420.yaml), kata must-gather now runs on failures only is disabled (MUST_GATHER_ON_FAILURE_ONLY: "false") and kata RPM installation is enabled (INSTALL_KATA_RPM: "true") with KATA_RPM_VERSION: 3.31.0-1.rhaos4.19.el9 (while MUST_GATHER_IMAGE is unchanged).
  • For the OCP 4.22 downstream candidate (...__downstream-candidate422.yaml), the same must-gather behavior is applied (MUST_GATHER_ON_FAILURE_ONLY: "false", INSTALL_KATA_RPM: "true") with KATA_RPM_VERSION: 3.31.0-1.rhaos4.22.el9 (keeping MUST_GATHER_IMAGE unchanged).
  • For the *-coco variants, the job environment is updated to install Trustee (TRUSTEE_INSTALL: "true") and pin the Trustee catalog source image (TRUSTEE_CATALOG_SOURCE_IMAGE).

Also updated ci-operator/step-registry/sandboxed-containers-operator/README.md with guidance for Konflux prowjobs’ restrict_network_access behavior (including how it should be handled around /pj-rehearse).

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This PR updates sandboxed-containers-operator CI wiring, adds a new Trustee install step and docs, and changes several release, mirroring, scheduling, and Prow configuration files.

Changes

Sandboxed containers Trustee CI flow

Layer / File(s) Summary
Shared build-image wiring
ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate417.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate418.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate419.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate420.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-release.yaml
The sandboxed-containers-operator configs add the CLI base image, build-root builder tag, and the trustee-charts and tools-with-helm image build definitions.
Downstream candidate job envs
ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate417.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate418.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate419.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate420.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml, ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate422.yaml
Periodic Azure, ARO, and AWS job definitions update Kata RPM, must-gather, trustee catalog, and network access settings; candidate 422 adds the full downstream periodic test matrix and metadata.
Trustee operator step implementation
ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-commands.sh
The Trustee operator install step adds chart fetching, rendering, OLM waits, initdata generation, configmap patching, and KBS connectivity verification.
Trustee step registry and docs
ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/OWNERS, ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.yaml, ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.metadata.json, ci-operator/step-registry/sandboxed-containers-operator/pre/sandboxed-containers-operator-pre-chain.yaml, ci-operator/step-registry/sandboxed-containers-operator/README.md
The step registry, chain wiring, OWNERS, and README add the new Trustee install step and its configuration notes.

Release, mirroring, and scheduling updates

Layer / File(s) Summary
Hypershift release pinning
ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.19.yaml, ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.20.yaml, ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.21.yaml, ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.22.yaml, ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.19.yaml, ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.20.yaml, ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.21.yaml, ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22.yaml, ci-operator/config/openshift/hypershift/openshift-hypershift-release-5.0__periodics.yaml, ci-operator/config/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main__ocm-fvt-rosa-hcp-staging.yaml
Hypershift release configs pin hypershift-operator-init, update promotion targets, and adjust the periodic schedule; the ROSA staging jobs add cron schedules.
Loki release and image mirroring
ci-operator/config/openshift/loki/openshift-loki-release-6.4.yaml, ci-operator/config/openshift/loki/openshift-loki-upstream-v3.6.12.yaml, core-services/image-mirroring/openshift-logging/mapping_logging_loki_quay
The Loki release moves to Go 1.26, adds the v3.6.12 upstream pipeline, and updates the logging image mirror mappings.
Cluster resource whitelist
clusters/gitops/apps/appproject-core-ci.yaml
The core-ci AppProject whitelist adds three cluster-scoped resource kinds.

Prow policy and repo config updates

Layer / File(s) Summary
AI Gateway Prow config
core-services/prow/02_config/opendatahub-io/ai-gateway-operator/_pluginconfig.yaml, core-services/prow/02_config/opendatahub-io/ai-gateway-operator/_prowconfig.yaml
The ai-gateway-operator repo gets plugin endpoints, approval/LGTM settings, trusted app triggers, and Tide rules for the main branch.
Branch filters and Jira label gates
core-services/prow/02_config/opendatahub-io/ai-gateway-payload-processing/_prowconfig.yaml, core-services/prow/02_config/osac-project/bare-metal-fulfillment-operator/_prowconfig.yaml, core-services/prow/02_config/osac-project/docs/_prowconfig.yaml, core-services/prow/02_config/osac-project/enhancement-proposals/_prowconfig.yaml, core-services/prow/02_config/osac-project/fulfillment-service/_prowconfig.yaml, core-services/prow/02_config/osac-project/osac-aap/_prowconfig.yaml, core-services/prow/02_config/osac-project/osac-installer/_prowconfig.yaml, core-services/prow/02_config/osac-project/osac-operator/_prowconfig.yaml, core-services/prow/02_config/osac-project/osac-test-infra/_prowconfig.yaml, core-services/prow/02_config/osac-project/osac-ui/_prowconfig.yaml
The payload-processing Tide config adds a main-branch filter, and multiple osac-project repos add jira/valid-reference to Tide label rules.
OSAC workspace repo config
core-services/prow/02_config/osac-project/osac-workspace/OWNERS, core-services/prow/02_config/osac-project/osac-workspace/_pluginconfig.yaml, core-services/prow/02_config/osac-project/osac-workspace/_prowconfig.yaml
The osac-workspace repo adds generated OWNERS, plugin config, and Tide config files.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Suggested labels

lgtm, rehearsals-ack, jira/valid-reference


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error The Trustee install script logs TRUSTEE_URL/trustee_url in success and error paths, exposing internal KBS hostnames/IPs in CI logs. Redact or omit the full KBS URL/host from stdout/stderr; keep the endpoint only in shared files/artifacts or mask it before logging.
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning The new trustee step hardcodes 127.0.0.1, builds http://$host:$port without IPv6 brackets, and defaults to GitHub/quay.io lookups. Use IPv6-safe URL/localhost handling and mirror or pin charts/kbs-client so the step works without public internet; verify with the IPv6 disconnected payload job.
✅ Passed checks (13 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: adding OCP 4.22 CI/prow configuration and related 4.22 updates.
Docstring Coverage ✅ Passed Docstring coverage is 94.74% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No It/Describe/Context/When test titles were added in the changed files; the PR only touches CI YAML, docs, and shell scripts.
Test Structure And Quality ✅ Passed No Ginkgo test code was changed; the PR only updates CI YAML, docs, and a shell step, and the shell file contains no Ginkgo constructs.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; touched files are CI YAML/scripts only, and scans found no It()/Describe()/Context()/When() in them.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the PR only changes CI YAML and bash step scripts, and no changed file contains topology-sensitive test logic.
Topology-Aware Scheduling Compatibility ✅ Passed Only CI configs and test scripts changed; no deployment manifests/controller code or topology-sensitive scheduling fields (nodeSelector, affinity, spread, PDB, replicas) were added.
Ote Binary Stdout Contract ✅ Passed The patch only changes YAML/SH/JSON/docs; no Go binaries or process-level setup code was added that could write non-JSON stdout.
No-Weak-Crypto ✅ Passed No weak algorithms or custom crypto were added; exact searches found no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, and no secret/token comparisons.
Container-Privileges ✅ Passed No touched manifest sets privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN or allowPrivilegeEscalation:true; the trustee pod uses allowPrivilegeEscalation:false, runAsNonRoot:true, drop ALL.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tbuskey

tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/assign @wainersm
/assign @vvoronko
/pj-rehearse ack

@openshift-ci openshift-ci Bot requested review from snir911 and vvoronko June 25, 2026 15:18
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 25, 2026
@tbuskey tbuskey mentioned this pull request Jun 25, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 25, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate420.yaml`:
- Line 48: Update the KATA_RPM_VERSION value used by the 4.20 candidate config
so it points to the rhaos4.20 artifact tag instead of rhaos4.19. Locate the
KATA_RPM_VERSION setting in the downstream candidate YAML for the
sandboxed-containers operator and change the version string to the OCP
4.20-compatible RPM tag so get-kata-rpm resolves correctly for 4.20 builds.

In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml`:
- Line 48: The KATA_RPM_VERSION value is using the wrong OCP stream suffix for
the 4.21 downstream candidate config. Update the KATA_RPM_VERSION entries in
openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml to a
4.21-compatible build tag ending in rhaos4.21.el9, and make sure any replicated
occurrences of this version string are changed consistently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: f01acde8-e898-4b1b-a149-4d486483f148

📥 Commits

Reviewing files that changed from the base of the PR and between ea3c9f1 and f4d35ac.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (4)
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate419.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate420.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate422.yaml

Other changes:
- MUST_GATHER_ON_FAILURE_ONLY: "false" # so prow always runs kata must-gather
- INSTALL_KATA_RPM: true
- KATA_RPM_VERSION: 3.31.0-1.rhaos4.19.el9 # 4.19 -> 4.21
- KATA_RPM_VERSION: 3.31.0-1.rhaos4.22.el9 # 4.22

https://redhat.atlassian.net/browse/KATA-5459
Rehersal success in openshift#80932
Signed-off-by: Tom Buskey <tbuskey@redhat.com>

Add note to README.md about restrict_network_access
@openshift-merge-bot openshift-merge-bot Bot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 25, 2026
@wainersm

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added lgtm Indicates that a PR is ready to be merged. and removed lgtm Indicates that a PR is ready to be merged. labels Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 0f61b8079739bda0346da472cb5ca76c4ff19600 due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@tbuskey

tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse list

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@tbuskey

tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse list

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@tbuskey

tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-openshift-sandboxed-containers-operator-devel-downstream-candidate422-aws-ipi-peerpods

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@tbuskey

tbuskey commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse list

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable prepare a candidate for rehearsal; rehearsals will not be run. This could be due to a branch that needs to be rebased. ERROR:

couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

eranco74 and others added 13 commits June 25, 2026 17:14
…AC repos (openshift#81073)

* OSAC-1770: require jira/valid-reference label for merge across all OSAC repos
OSAC-1800: add Prow/Tide configuration for osac-workspace

Add jira/valid-reference to the required Tide labels for all 9 existing
OSAC repos, and add full Prow/Tide configuration for osac-workspace
(which was previously missing).

PRs must now have a valid Jira ticket (e.g. OSAC-1234: title) or
explicitly say NO-ISSUE in the title to be mergeable.

Assisted-by: Claude Code <noreply@anthropic.com>
Signed-off-by: Eran Cohen <eranco@redhat.com>

* OSAC-1770: determinize prow config for label ordering

Run `make prow-config` to sort labels alphabetically as
required by the prow-config CI check.

Assisted-by: Claude Code <noreply@anthropic.com>
Signed-off-by: Eran Cohen <eranco@redhat.com>

---------

Signed-off-by: Eran Cohen <eranco@redhat.com>
…nshift#81026)

Add run_if_changed path filtering to the 8 module package PR
image-mirror jobs so they only trigger when their respective
package directory is modified, instead of running on every PR.

Mapping:
- odh-mod-arch-model-registry-pr-image-mirror: ^packages/model-registry/
- odh-mod-arch-gen-ai-pr-image-mirror:         ^packages/gen-ai/
- odh-mod-arch-maas-pr-image-mirror:           ^packages/maas/
- odh-mod-arch-mlflow-pr-image-mirror:         ^packages/mlflow/
- odh-mod-arch-eval-hub-pr-image-mirror:       ^packages/eval-hub/
- odh-mod-arch-automl-pr-image-mirror:         ^packages/automl/
- odh-mod-arch-autorag-pr-image-mirror:        ^packages/autorag/
- odh-mod-arch-agent-ops-pr-image-mirror:      ^packages/agent-ops/

The main odh-dashboard-pr-image-mirror and all postsubmit jobs
remain always_run: true.
…1084)

add prow plugin and tide configuration for the new
ai-gateway-operator repository under opendatahub-io org.

- configure standard prow plugins (approve, lgtm, assign, trigger, etc.)
- configure external plugins (cherrypick, needs-rebase, jira-lifecycle, etc.)
- configure triggers with trusted_apps (openshift-merge-bot)
- set tide merge method to squash with standard label requirements
- restrict tide to main branch via includedBranches
- also add includedBranches to ai-gateway-payload-processing for consistency
- no ci-operator jobs at this stage, only prow wiring

Signed-off-by: Chaitanya Kulkarni <chkulkar@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…penshift#80864)

* CNF-25270: Export LCA git coordinates to image-based remote scripts

- Wire Prow PR metadata into ib-orchestrate-vm remote scripts so lifecycle-agent presubmits checkout the correct source during LCA deploy.

Assisted-by: Cursor/auto
AI-attribution: AIA,Primarily AI-generated,Human-initiated,Reviewed,Cursor/auto,v1.0
For more information on AI attribution statements, see: https://aiattribution.github.io/

* CNF-25270: Update ibu scripts to use the correct branch for rehearsals
- Previously, it was always checking out `main` branch for LCA, even for non-main rehearsals
- Where possible, try to preferentially use the matching release version for rehearsals
…se branches (openshift#80970)

The upgrade test on release-4.{19,20,21,22} installs the HO from
hypershift-operator-init before upgrading to the PR-built version.
With tag:latest this pulls the HO from main, causing a downgrade
instead of an upgrade. Pin the tag to the release version and add
a promotion target so each release branch publishes its HO image
with a version-specific tag to the hypershift namespace.

Signed-off-by: Ahmed Abdalla <aabdelre@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Returning TPNU job to weekly as verified job for feature promotion has
satisfied job runs. Keeping this as long-term with slack monitoring.
The two sanity jobs (cs-sanity-staging and cs-sanity-jira-staging)
were missing cron fields, making them always_run presubmits. These
jobs pull an external ocmci container and run ocmtest with zero
dependency on rosa-e2e repo code, so running them on every PR is
wasteful and blocks unrelated changes.

Add cron schedules to make them periodics like the other 13 FVT
jobs in this variant.
Signed-off-by: Steve Kuznetsov <stekuznetsov@microsoft.com>
…penshift#80601)

* Add Trustee operator installation step for Confidential Containers

Add automated Trustee operator installation for sandboxed-containers-operator
CoCo (Confidential Containers) tests using OLM and helm charts.

Key Features:
- OLM-based operator installation with comprehensive wait stages
- Helm chart integration with pre-built container image
- Works with restrict_network_access: true for rehearsals
- KBS connectivity verification and resource validation
- tools-with-helm custom image with oc, kubectl, helm, jq, skopeo, git

Step Registry:
- Created install-trustee-operator step in sandboxed-containers-operator
- Added to sandboxed-containers-operator-pre chain before OSC installation
- Runs for all *-ipi-coco test jobs when TRUSTEE_INSTALL="true"

CI Configuration:
- Added trustee-charts image (packages helm charts from confidential-devhub/charts)
- Added tools-with-helm image based on cli with helm and required tools
- Added cli to base_images for tools-with-helm dependency
- Enabled Trustee installation for azure-ipi-coco, aro-ipi-coco, aws-ipi-coco
- Set restrict_network_access: true for candidate421 aws-ipi-coco

Technical Implementation:
- FROM this-is-ignored with from: cli pattern for custom image builds
- Wait stages: CatalogSource → Subscription → InstallPlan → CSV → Deployment → Pods
- Generous 10-minute timeouts to avoid cluster rebuild waste
- Discovers KBS endpoint and persists TRUSTEE_URL for CoCo tests
- Generates INITDATA artifacts and patches osc-config ConfigMap
- Verifies connectivity with kbs-client pod testing resource retrieval

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix Trustee catalog configuration and remove unused env vars

  - Remove TRUSTEE_CATALOG_SOURCE_NAME (helm chart hardcodes catalog name)
  - Fix dev.enabled bug: explicitly control based on custom image presence
  - Remove invalid helm '--set catalogSource.name' parameter
  - Add tools-with-helm image to all downstream-release config
  - Document catalog source configuration and image tag strategies

  Implementation is tag-agnostic for future flexibility switching to :latest.

Signed-off-by: Tom Buskey <tbuskey@redhat.com>

---------

Signed-off-by: Tom Buskey <tbuskey@redhat.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
…penshift#81098)

Whitelist MutatingWebhookConfiguration, RuntimeClass, and PriorityClass
so ci-scheduling-webhook manifests can sync on core-ci via Argo CD.
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@tbuskey, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 6811336d759a76beffc8ece3dce3be7f7f82e04e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tbuskey, wainersm
Once this PR has been reviewed and has the lgtm label, please assign andrewballantyne, cblecker, danilo-gemoli, eliorerz, gdbranco, jcantrill for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@tbuskey: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate.yaml`:
- Around line 21-26: The helper image definition is still using moving targets,
so update the dockerfile_literal blocks to use immutable references instead of
registry.access.redhat.com/ubi9/ubi-minimal:latest and the
confidential-devhub/charts main branch. Pin the base image in the helper image
setup and replace the git clone revision in the chart fetch step with a fixed
commit, tag, or release reference in all duplicated variant blocks (for example,
the downstream candidate and any copied 4.17/4.18/4.19/default definitions).
Keep the change consistent across the shared helper image configuration so the
same inputs are used everywhere.

In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-release.yaml`:
- Around line 21-26: The image build in the dockerfile_literal block is cloning
confidential-devhub/charts from main, which makes the CI input non-reproducible.
Update the git clone step to use a reviewed tag or specific commit instead of
the moving branch, so future rebuilds always use the same charts revision. Keep
the change within the dockerfile_literal content used by this downstream-release
config.

In
`@ci-operator/step-registry/openshift/image-based/upgrade/seed/create/openshift-image-based-upgrade-seed-create-commands.sh`:
- Around line 103-118: The lifecycle-agent branch selection in the seed-create
script is defaulting release jobs to main, which can check out the wrong branch.
Update the LCA_GIT_BRANCH derivation in
openshift-image-based-upgrade-seed-create-commands.sh so it reads the
lifecycle-agent base ref from JOB_SPEC refs.extra_refs[].base_ref (or equivalent
refs.base_ref data) before falling back, and only use main as the last resort;
keep the existing lifecycle-agent-release-* and lifecycle-agent-main-* handling
in the same branch-selection block.

In
`@ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-commands.sh`:
- Around line 169-185: The git-clone fallback currently returns the repo root
path in charts_dir, but the render helpers expect the nested charts directory
inside the cloned repo. Update the fallback in the trustee charts fetch flow to
point to the cloned repo’s charts/ subdirectory before echoing the path, so the
later chart lookups for trustee-operator and trustee-operands resolve correctly.
Apply the same path adjustment anywhere this fallback result is consumed, using
the existing charts_dir handling and TRUSTEE_CHARTS_REPO/REF flow.
- Around line 334-377: The all-CatalogSources readiness loop in the trustee
install script is too broad and blocks on unrelated marketplace health. Update
the waiting logic in the sandboxed-containers-operator install-trustee flow to
check only the CatalogSources this install actually depends on, using the
existing readiness polling structure and the CatalogSource lookup/logging code.
Keep the wait, progress, and failure reporting behavior, but scope the `oc get
catalogsource` query and readiness count to the required sources instead of
every catalog in openshift-marketplace.

In
`@ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.yaml`:
- Around line 59-60: Update the step documentation text in the
sandboxed-containers-operator install-trustee-operator reference so it no longer
claims Helm is unnecessary; the current wording conflicts with the install logic
that checks for helm and constructs Helm args. Rephrase the note to say the step
avoids runtime chart fetches by using pre-rendered manifests and sed-based
substitution, while still relying on helm for installation, and keep the message
aligned with the install step’s behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: fad2d72d-8154-4353-9785-dbb04dbe7781

📥 Commits

Reviewing files that changed from the base of the PR and between fe6f4a9 and b534133.

⛔ Files ignored due to path filters (15)
  • ci-operator/jobs/opendatahub-io/odh-dashboard/opendatahub-io-odh-dashboard-main-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.19-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.19-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.20-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.20-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.21-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.21-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.22-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.22-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-5.0-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/loki/openshift-loki-upstream-v3.6.12-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/loki/openshift-loki-upstream-v3.6.12-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (47)
  • ci-operator/config/opendatahub-io/odh-dashboard/opendatahub-io-odh-dashboard-main.yaml
  • ci-operator/config/openshift-online/rosa-e2e/openshift-online-rosa-e2e-main__ocm-fvt-rosa-hcp-staging.yaml
  • ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.19.yaml
  • ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.20.yaml
  • ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.21.yaml
  • ci-operator/config/openshift-priv/hypershift/openshift-priv-hypershift-release-4.22.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.19.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.20.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.21.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-5.0__periodics.yaml
  • ci-operator/config/openshift/loki/openshift-loki-release-6.4.yaml
  • ci-operator/config/openshift/loki/openshift-loki-upstream-v3.6.12.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate417.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate418.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate419.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate420.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate421.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate422.yaml
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-release.yaml
  • ci-operator/step-registry/aro-hcp/gather/observability/aro-hcp-gather-observability-ref.yaml
  • ci-operator/step-registry/openshift/image-based/upgrade/seed/create/openshift-image-based-upgrade-seed-create-commands.sh
  • ci-operator/step-registry/openshift/image-based/upgrade/target/openshift-image-based-upgrade-target-commands.sh
  • ci-operator/step-registry/sandboxed-containers-operator/README.md
  • ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/OWNERS
  • ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-commands.sh
  • ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.metadata.json
  • ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.yaml
  • ci-operator/step-registry/sandboxed-containers-operator/pre/sandboxed-containers-operator-pre-chain.yaml
  • clusters/gitops/apps/appproject-core-ci.yaml
  • core-services/image-mirroring/openshift-logging/mapping_logging_loki_quay
  • core-services/prow/02_config/opendatahub-io/ai-gateway-operator/_pluginconfig.yaml
  • core-services/prow/02_config/opendatahub-io/ai-gateway-operator/_prowconfig.yaml
  • core-services/prow/02_config/opendatahub-io/ai-gateway-payload-processing/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/bare-metal-fulfillment-operator/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/docs/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/enhancement-proposals/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/fulfillment-service/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-aap/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-installer/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-operator/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-test-infra/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-ui/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-workspace/OWNERS
  • core-services/prow/02_config/osac-project/osac-workspace/_pluginconfig.yaml
  • core-services/prow/02_config/osac-project/osac-workspace/_prowconfig.yaml
✅ Files skipped from review due to trivial changes (9)
  • core-services/prow/02_config/osac-project/osac-test-infra/_prowconfig.yaml
  • ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/OWNERS
  • ci-operator/step-registry/aro-hcp/gather/observability/aro-hcp-gather-observability-ref.yaml
  • core-services/prow/02_config/osac-project/osac-workspace/OWNERS
  • core-services/prow/02_config/osac-project/osac-installer/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/osac-ui/_prowconfig.yaml
  • core-services/prow/02_config/osac-project/enhancement-proposals/_prowconfig.yaml
  • core-services/prow/02_config/opendatahub-io/ai-gateway-operator/_prowconfig.yaml
  • ci-operator/step-registry/sandboxed-containers-operator/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate422.yaml

Comment on lines +21 to +26
- dockerfile_literal: |
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
RUN microdnf install -y git tar gzip && microdnf clean all
RUN git clone --depth 1 --branch main \
https://github.com/confidential-devhub/charts /charts && \
rm -rf /charts/.git

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Pin the helper image inputs to immutable versions.

ubi-minimal:latest and cloning confidential-devhub/charts from main make these candidate jobs non-reproducible, so an upstream change can silently alter or break the 4.17/4.18/4.19/default variants without any corresponding change here. Please pin the base image and charts revision to immutable values in all copied blocks.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-candidate.yaml`
around lines 21 - 26, The helper image definition is still using moving targets,
so update the dockerfile_literal blocks to use immutable references instead of
registry.access.redhat.com/ubi9/ubi-minimal:latest and the
confidential-devhub/charts main branch. Pin the base image in the helper image
setup and replace the git clone revision in the chart fetch step with a fixed
commit, tag, or release reference in all duplicated variant blocks (for example,
the downstream candidate and any copied 4.17/4.18/4.19/default definitions).
Keep the change consistent across the shared helper image configuration so the
same inputs are used everywhere.

Comment on lines +21 to +26
- dockerfile_literal: |
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
RUN microdnf install -y git tar gzip && microdnf clean all
RUN git clone --depth 1 --branch main \
https://github.com/confidential-devhub/charts /charts && \
rm -rf /charts/.git

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Pin the Trustee charts revision.

Line 24 pulls confidential-devhub/charts from main, so any future rebuild can silently pick up unrelated upstream chart changes and start failing without a matching PR here. Please switch this image build to a reviewed tag or commit so the CI input stays reproducible.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/sandboxed-containers-operator/openshift-sandboxed-containers-operator-devel__downstream-release.yaml`
around lines 21 - 26, The image build in the dockerfile_literal block is cloning
confidential-devhub/charts from main, which makes the CI input non-reproducible.
Update the git clone step to use a reviewed tag or specific commit instead of
the moving branch, so future rebuilds always use the same charts revision. Keep
the change within the dockerfile_literal content used by this downstream-release
config.

Comment on lines +103 to +118
if [[ "${REPO_OWNER}/${REPO_NAME}" == "openshift-kni/lifecycle-agent" ]]; then
CI_LCA_GIT_REF="${PULL_PULL_SHA:-}"
CI_LCA_GIT_PULL="${PULL_NUMBER:-}"
LCA_GIT_BRANCH="${PULL_BASE_REF:-${LCA_GIT_BRANCH:-}}"
elif [[ "${JOB_NAME}" == rehearse-* ]] && [[ "${JOB_NAME}" == *lifecycle-agent* ]]; then
LCA_GIT_BRANCH="$(echo "${JOB_SPEC}" | jq -r '[.extra_refs[]? | select(.org == "openshift-kni" and .repo == "lifecycle-agent") | .base_ref][0] // empty')"
if [[ -z "${LCA_GIT_BRANCH}" ]]; then
if [[ "${JOB_NAME}" =~ lifecycle-agent-release-([0-9]+\.[0-9]+) ]]; then
LCA_GIT_BRANCH="release-${BASH_REMATCH[1]}"
elif [[ "${JOB_NAME}" == *lifecycle-agent-main-* ]]; then
LCA_GIT_BRANCH="main"
else
LCA_GIT_BRANCH="main"
fi
fi
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== step consumers =="
rg -n -C2 'openshift-image-based-upgrade-(seed-create|target)' ci-operator/config

echo
echo "== branch sources already used in related scripts =="
rg -n -C2 'PULL_BASE_REF|refs\.base_ref|JOB_SPEC' ci-operator/step-registry/openshift/image-based/upgrade

echo
echo "== lifecycle-agent jobs in CI config =="
rg -n -C2 'openshift-kni.*lifecycle-agent|lifecycle-agent.*release-[0-9]+\.[0-9]+' ci-operator/config

Repository: openshift/release

Length of output: 36238


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== relevant script context =="
sed -n '90,140p' ci-operator/step-registry/openshift/image-based/upgrade/seed/create/openshift-image-based-upgrade-seed-create-commands.sh

echo
echo "== matching logic in target step =="
sed -n '30,60p' ci-operator/step-registry/openshift/image-based/upgrade/target/openshift-image-based-upgrade-target-commands.sh

echo
echo "== where LCA_GIT_BRANCH is set in repo configs =="
rg -n 'LCA_GIT_BRANCH|CI_LCA_GIT_REF|CI_LCA_GIT_PULL|PULL_BASE_REF|base_ref' ci-operator/config/openshift-kni/lifecycle-agent

echo
echo "== references to JOB_SPEC.refs.base_ref in the repo =="
rg -n 'JOB_SPEC.*refs\.base_ref|refs\.base_ref' ci-operator/step-registry ci-operator/config

Repository: openshift/release

Length of output: 3706


Don't default lifecycle-agent release jobs to main.
LCA_GIT_BRANCH is only seeded from PULL_BASE_REF here, and release periodic jobs under ci-operator/config/openshift-kni/lifecycle-agent/* won't have that value, so the export at line 133 falls back to main and checks out the wrong branch. Use JOB_SPEC/refs.base_ref before the final fallback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/openshift/image-based/upgrade/seed/create/openshift-image-based-upgrade-seed-create-commands.sh`
around lines 103 - 118, The lifecycle-agent branch selection in the seed-create
script is defaulting release jobs to main, which can check out the wrong branch.
Update the LCA_GIT_BRANCH derivation in
openshift-image-based-upgrade-seed-create-commands.sh so it reads the
lifecycle-agent base ref from JOB_SPEC refs.extra_refs[].base_ref (or equivalent
refs.base_ref data) before falling back, and only use main as the last resort;
keep the existing lifecycle-agent-release-* and lifecycle-agent-main-* handling
in the same branch-selection block.

Source: Coding guidelines

Comment on lines +169 to +185
# Option 2: Fallback to git clone (requires restrict_network_access: false)
echo ">>> Fetching trustee charts from GitHub: ${TRUSTEE_CHARTS_REPO} (ref: ${TRUSTEE_CHARTS_REF})" >&2

if ! command -v git &> /dev/null; then
echo ">>> ERROR: git command not found" >&2
return 1
fi

git clone --depth 1 --branch "${TRUSTEE_CHARTS_REF}" "${TRUSTEE_CHARTS_REPO}" "${charts_dir}"

if [[ ! -d "${charts_dir}" ]]; then
echo ">>> ERROR: Failed to clone charts repository" >&2
return 1
fi

echo ">>> Charts cloned from GitHub" >&2
echo "${charts_dir}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Return the nested charts/ directory from the git-clone fallback.

The fallback clones the repo root and returns ${charts_dir}, but both render helpers expect ${charts_dir}/trustee-operator and ${charts_dir}/trustee-operands. With the documented repo layout (charts/<chart>), any run that misses IMAGE_TRUSTEE_CHARTS or falls back after a failed extract will fail later with “chart not found”.

Proposed fix
   git clone --depth 1 --branch "${TRUSTEE_CHARTS_REF}" "${TRUSTEE_CHARTS_REPO}" "${charts_dir}"
 
   if [[ ! -d "${charts_dir}" ]]; then
     echo ">>> ERROR: Failed to clone charts repository" >&2
     return 1
   fi
 
+  local repo_charts_dir="${charts_dir}/charts"
+  if [[ ! -d "${repo_charts_dir}/trustee-operator" ]] || [[ ! -d "${repo_charts_dir}/trustee-operands" ]]; then
+    echo ">>> ERROR: Expected charts under ${repo_charts_dir}" >&2
+    return 1
+  fi
+
   echo ">>> Charts cloned from GitHub" >&2
-  echo "${charts_dir}"
+  echo "${repo_charts_dir}"
 }

Also applies to: 221-223, 271-272

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-commands.sh`
around lines 169 - 185, The git-clone fallback currently returns the repo root
path in charts_dir, but the render helpers expect the nested charts directory
inside the cloned repo. Update the fallback in the trustee charts fetch flow to
point to the cloned repo’s charts/ subdirectory before echoing the path, so the
later chart lookups for trustee-operator and trustee-operands resolve correctly.
Apply the same path adjustment anywhere this fallback result is consumed, using
the existing charts_dir handling and TRUSTEE_CHARTS_REPO/REF flow.

Comment on lines +334 to +377
# Stage 0: Wait for ALL CatalogSources to be READY (600s / 10 minutes)
# This prevents Subscription failures due to missing/unavailable catalogs
echo ">>> Waiting for all CatalogSources to be READY..."
local all_catalogs_ready=false
for i in {1..120}; do
# Get all catalogs and their states
local catalog_states
catalog_states=$(oc get catalogsource -n openshift-marketplace -o jsonpath='{range .items[*]}{.metadata.name}={.status.connectionState.lastObservedState}{"\n"}{end}' 2>/dev/null || echo "")

if [[ -z "${catalog_states}" ]]; then
echo ">>> WARNING: Unable to get catalog states (attempt ${i}/120)"
[[ ${i} -lt 120 ]] && sleep 5
continue
fi

# Count total vs ready catalogs
local total_catalogs
total_catalogs=$(echo "${catalog_states}" | wc -l)
local ready_catalogs
ready_catalogs=$(echo "${catalog_states}" | grep -c "=READY" || echo "0")

if [[ ${ready_catalogs} -eq ${total_catalogs} && ${ready_catalogs} -gt 0 ]]; then
echo ">>> All CatalogSources are READY (${ready_catalogs}/${total_catalogs})"
all_catalogs_ready=true
break
fi

# Show progress every 6 iterations (30 seconds)
if [[ $((i % 6)) -eq 0 ]]; then
echo ">>> CatalogSources ready: ${ready_catalogs}/${total_catalogs} (checking ${i}/120, $((i*5))s elapsed)..."
echo "${catalog_states}" | grep -v "=READY" | head -5 || true
fi

[[ ${i} -lt 120 ]] && sleep 5
done

if [[ "${all_catalogs_ready}" != "true" ]]; then
echo ">>> ERROR: Not all CatalogSources are READY after 600s"
echo ">>> Current CatalogSource states:"
oc get catalogsource -n openshift-marketplace -o custom-columns=NAME:.metadata.name,STATE:.status.connectionState.lastObservedState || true
echo ">>> CatalogSource pods:"
oc get pods -n openshift-marketplace || true
return 1
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Wait only on the CatalogSources this install actually needs.

Failing the step until every CatalogSource in openshift-marketplace is READY makes Trustee installation depend on unrelated marketplace health. A single degraded third-party catalog will deadlock this step even when redhat-operators and the Trustee catalog are fine.

Suggested direction
-  # Stage 0: Wait for ALL CatalogSources to be READY (600s / 10 minutes)
-  # This prevents Subscription failures due to missing/unavailable catalogs
-  echo ">>> Waiting for all CatalogSources to be READY..."
+  # Stage 0: Wait only for the CatalogSources this install depends on.
+  echo ">>> Waiting for required CatalogSources to be READY..."
+  local required_catalogs=("redhat-operators")
+  [[ -n "${TRUSTEE_CATALOG_SOURCE_IMAGE}" ]] && required_catalogs+=("trustee-operator-dev-catalog")
+
+  for catalog in "${required_catalogs[@]}"; do
+    if ! wait_until "CatalogSource ${catalog} READY" 600 5 \
+      "[[ \"\$(oc get catalogsource -n openshift-marketplace '${catalog}' -o jsonpath='{.status.connectionState.lastObservedState}' 2>/dev/null)\" == \"READY\" ]]"; then
+      oc get catalogsource -n openshift-marketplace "${catalog}" -o yaml || true
+      return 1
+    fi
+  done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Stage 0: Wait for ALL CatalogSources to be READY (600s / 10 minutes)
# This prevents Subscription failures due to missing/unavailable catalogs
echo ">>> Waiting for all CatalogSources to be READY..."
local all_catalogs_ready=false
for i in {1..120}; do
# Get all catalogs and their states
local catalog_states
catalog_states=$(oc get catalogsource -n openshift-marketplace -o jsonpath='{range .items[*]}{.metadata.name}={.status.connectionState.lastObservedState}{"\n"}{end}' 2>/dev/null || echo "")
if [[ -z "${catalog_states}" ]]; then
echo ">>> WARNING: Unable to get catalog states (attempt ${i}/120)"
[[ ${i} -lt 120 ]] && sleep 5
continue
fi
# Count total vs ready catalogs
local total_catalogs
total_catalogs=$(echo "${catalog_states}" | wc -l)
local ready_catalogs
ready_catalogs=$(echo "${catalog_states}" | grep -c "=READY" || echo "0")
if [[ ${ready_catalogs} -eq ${total_catalogs} && ${ready_catalogs} -gt 0 ]]; then
echo ">>> All CatalogSources are READY (${ready_catalogs}/${total_catalogs})"
all_catalogs_ready=true
break
fi
# Show progress every 6 iterations (30 seconds)
if [[ $((i % 6)) -eq 0 ]]; then
echo ">>> CatalogSources ready: ${ready_catalogs}/${total_catalogs} (checking ${i}/120, $((i*5))s elapsed)..."
echo "${catalog_states}" | grep -v "=READY" | head -5 || true
fi
[[ ${i} -lt 120 ]] && sleep 5
done
if [[ "${all_catalogs_ready}" != "true" ]]; then
echo ">>> ERROR: Not all CatalogSources are READY after 600s"
echo ">>> Current CatalogSource states:"
oc get catalogsource -n openshift-marketplace -o custom-columns=NAME:.metadata.name,STATE:.status.connectionState.lastObservedState || true
echo ">>> CatalogSource pods:"
oc get pods -n openshift-marketplace || true
return 1
fi
# Stage 0: Wait only for the CatalogSources this install depends on.
echo ">>> Waiting for required CatalogSources to be READY..."
local required_catalogs=("redhat-operators")
[[ -n "${TRUSTEE_CATALOG_SOURCE_IMAGE}" ]] && required_catalogs+=("trustee-operator-dev-catalog")
for catalog in "${required_catalogs[@]}"; do
if ! wait_until "CatalogSource ${catalog} READY" 600 5 \
"[[ \"\$(oc get catalogsource -n openshift-marketplace '${catalog}' -o jsonpath='{.status.connectionState.lastObservedState}' 2>/dev/null)\" == \"READY\" ]]"; then
oc get catalogsource -n openshift-marketplace "${catalog}" -o yaml || true
return 1
fi
done
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-commands.sh`
around lines 334 - 377, The all-CatalogSources readiness loop in the trustee
install script is too broad and blocks on unrelated marketplace health. Update
the waiting logic in the sandboxed-containers-operator install-trustee flow to
check only the CatalogSources this install actually depends on, using the
existing readiness polling structure and the CatalogSource lookup/logging code.
Keep the wait, progress, and failure reporting behavior, but scope the `oc get
catalogsource` query and readiness count to the required sources instead of
every catalog in openshift-marketplace.

Comment on lines +59 to +60
NO NETWORK ACCESS REQUIRED: This step uses pre-rendered manifests with runtime variable
substitution via sed, eliminating the need for helm or git. Works with restrict_network_access: true.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the Helm requirement in the step docs.

These lines say the step eliminates the need for Helm, but the script explicitly exits when helm is missing and builds Helm arguments during install. Please reword this to say the step avoids runtime chart fetches, not that Helm is unnecessary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/sandboxed-containers-operator/install-trustee-operator/sandboxed-containers-operator-install-trustee-operator-ref.yaml`
around lines 59 - 60, Update the step documentation text in the
sandboxed-containers-operator install-trustee-operator reference so it no longer
claims Helm is unnecessary; the current wording conflicts with the install logic
that checks for helm and constructs Helm args. Rephrase the note to say the step
avoids runtime chart fetches by using pre-rendered manifests and sed-based
substitution, while still relying on helm for installation, and keep the message
aligned with the install step’s behavior.

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 26, 2026
@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tbuskey

tbuskey commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Moved to #81141

@tbuskey tbuskey closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.