ROSAENG-64876: Add weekly adversary security scan (GitHub Actions) - #745
ROSAENG-64876: Add weekly adversary security scan (GitHub Actions)#745jonseidman wants to merge 2 commits into
Conversation
Add a scheduled GitHub Actions workflow that runs the adversary security scanner weekly (Saturday 6am UTC) with groundwork mode. Supports on-demand runs via workflow_dispatch with configurable scan mode. Posts traffic-light Slack notifications (red for CRITICAL/HIGH, yellow for MEDIUM/LOW, green for clean). Requires ANTHROPIC_API_KEY and SLACK_WEBHOOK_URL secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
openshift-ci-robot
added
the
jira/valid-reference
|
@jonseidman: This pull request references ROSAENG-64876 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
needs-ok-to-test
|
Hi @jonseidman. Thanks for your PR. I'm waiting for a openshift-online member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
📝 WalkthroughWalkthroughThis change adds a GitHub Actions workflow for scheduled and manual adversary security scans. The workflow installs Claude Code and the ROSA security plugin, runs a bounded scan, publishes results, uploads artifacts, and sends conditional Slack notifications. ChangesAdversary scan automation
Estimated code review effort: 3 (Moderate) | ~20 minutes Mergeability Score: 🟠 High · up to The new weekly workflow can report a clean security scan even when the scanner fails, and it runs unpinned external code while an API key is available. These issues could hide incomplete security coverage or allow changed dependency code to access the secret, so merge should wait for failure propagation and approved dependency pinning. Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant ClaudeCode
participant ROSASecurityPlugin
participant Slack
GitHubActions->>ClaudeCode: Install Claude Code and ROSA security plugin
GitHubActions->>ClaudeCode: Run configured adversary scan
ClaudeCode->>ROSASecurityPlugin: Execute security scan
ROSASecurityPlugin-->>GitHubActions: Return scan findings and severity counts
GitHubActions->>Slack: Send notification with findings, duration, and run URL
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
.github/workflows/adversary-scan.yml (2)
14-38: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winConsider failing the run on CRITICAL or HIGH findings.
The workflow reports findings only through Slack. A missed Slack message hides critical findings. Add a final step that exits non-zero when
steps.scan.outputs.criticalorsteps.scan.outputs.highis greater than 0, so the run status also reflects the result. Aconcurrencygroup also prevents an overlapping manual run during the 3-hour scheduled scan.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/adversary-scan.yml around lines 14 - 38, Update the adversary-scan workflow by adding a concurrency group that prevents scheduled and manually dispatched runs from overlapping, and add a final job step after the scan step to fail the workflow when steps.scan.outputs.critical or steps.scan.outputs.high is greater than zero. Preserve the existing Slack reporting and ensure the final check references the scan step’s outputs.
46-47: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPin the Claude Code version.
npm install -g@anthropic-ai/claude-code`` resolves to the latest published version on every run. A weekly scheduled scan then changes tool behavior without a code change. Pin an exact version and update it deliberately.- run: npm install -g `@anthropic-ai/claude-code` + run: npm install -g `@anthropic-ai/claude-code`@<pinned-version>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/adversary-scan.yml around lines 46 - 47, Update the Install Claude Code step to install an exact pinned version of `@anthropic-ai/claude-code` instead of resolving the latest release; retain the existing package installation flow and choose a specific version that can be updated deliberately.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/adversary-scan.yml:
- Around line 41-44: Pin both third-party actions in
.github/workflows/adversary-scan.yml to immutable commit SHAs: replace
actions/checkout@v7 at lines 41-44 with the target release SHA and a matching #
v7.x.y comment, and replace actions/upload-artifact@v4 at lines 95-97 with its
target release SHA and # v4.x.y comment.
- Around line 79-93: Update the severity-count assignments in the scan step to
avoid appending a second zero when grep finds no matches: capture grep’s output
while defaulting the command substitution to a single 0 value. Ensure each of
CRITICAL, HIGH, MEDIUM, and LOW is always one-line numeric output for
GITHUB_OUTPUT and later arithmetic, and verify the stream-json log format
supports line-based counting before retaining grep -c.
- Around line 119-144: Update the Slack message construction in the
traffic-light logic and duration section to append actual newline characters
rather than the literal "\n" sequence, so Slack renders separate lines. Also
move the step-output expressions currently expanded in the script into the
step’s env block, then reference those environment variables in the shell code
to eliminate template-injection warnings.
---
Nitpick comments:
In @.github/workflows/adversary-scan.yml:
- Around line 14-38: Update the adversary-scan workflow by adding a concurrency
group that prevents scheduled and manually dispatched runs from overlapping, and
add a final job step after the scan step to fail the workflow when
steps.scan.outputs.critical or steps.scan.outputs.high is greater than zero.
Preserve the existing Slack reporting and ensure the final check references the
scan step’s outputs.
- Around line 46-47: Update the Install Claude Code step to install an exact
pinned version of `@anthropic-ai/claude-code` instead of resolving the latest
release; retain the existing package installation flow and choose a specific
version that can be updated deliberately.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: e53466dd-7ea5-4bf8-b1fa-af50813a8e14
📒 Files selected for processing (1)
.github/workflows/adversary-scan.yml
Fix grep severity count bug (|| echo 0 appends duplicate value),
pin actions to commit SHAs, fix Slack newline escaping (literal \n),
move ${{ }} expressions into env block to prevent template injection,
and add concurrency group to prevent overlapping runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.github/workflows/adversary-scan.yml (2)
74-97: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winPreserve scanner failures and report them as failures.
teeis the last command in this pipeline. Withoutset -o pipefail, aclaudeerror or the 10,200-second timeout is masked whenteewrites the log successfully. The step then publishesexit_code=0, completes successfully, and the Slack step can send a green “No findings” message for an incomplete scan.Enable
pipefail, capture the scanner status after the pipeline, and fail the scan step only after it writes outputs. Passexit_codeto the Slack step and send a scan-failed notification when it is nonzero.Proposed fix
- name: Run adversary scan id: scan run: | + set -o pipefail echo "=== Adversary Security Scan ===" ... - timeout 10200 claude \ + timeout 10200 claude \ --model "claude-opus-4-6" \ --output-format stream-json \ --max-turns 100 \ -p "${PROMPT}" \ - --verbose 2>&1 | tee adversary-scan.log || EXIT_CODE=$? + --verbose 2>&1 | tee adversary-scan.log + EXIT_CODE=$? ... echo "exit_code=${EXIT_CODE}" >> "$GITHUB_OUTPUT" + + exit "${EXIT_CODE}"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/adversary-scan.yml around lines 74 - 97, Update the adversary scan step around the claude/tee pipeline to enable pipefail and capture the scanner or timeout status rather than tee’s status. Preserve that status in the exit_code GitHub output, write all scan outputs before failing the step when it is nonzero, and update the Slack notification logic to receive exit_code and send a scan-failed notification instead of a green “No findings” message for failed scans.
50-57: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winPin Claude Code and the adversary plugin to approved revisions.
Pin Claude Code to an exact npm version. Add the marketplace with a fixed branch or tag. Claude Code does not support SHA pins for Git marketplace sources. Pin the
securityplugin source to its full commit SHA inmarketplace.json. These dependencies execute whileANTHROPIC_API_KEYis available.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/adversary-scan.yml around lines 50 - 57, Update the “Install Claude Code” step to install an explicitly approved exact npm version, and change the marketplace addition in the adversary skill setup to use an approved fixed branch or tag. In the referenced marketplace.json, pin the security plugin source to its full approved commit SHA while preserving the existing plugin installation flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/adversary-scan.yml:
- Around line 74-97: Update the adversary scan step around the claude/tee
pipeline to enable pipefail and capture the scanner or timeout status rather
than tee’s status. Preserve that status in the exit_code GitHub output, write
all scan outputs before failing the step when it is nonzero, and update the
Slack notification logic to receive exit_code and send a scan-failed
notification instead of a green “No findings” message for failed scans.
- Around line 50-57: Update the “Install Claude Code” step to install an
explicitly approved exact npm version, and change the marketplace addition in
the adversary skill setup to use an approved fixed branch or tag. In the
referenced marketplace.json, pin the security plugin source to its full approved
commit SHA while preserving the existing plugin installation flow.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: b239bc47-a0ec-4bfe-b6fe-99c0ade3bf7c
📒 Files selected for processing (1)
.github/workflows/adversary-scan.yml
2 participants
Summary
.github/workflows/adversary-scan.yml) that runs the adversary security scanner weeklyrosa-claude-pluginsmarketplace security skill to scan across 17 security domains (SAST, IaC, containers, secrets, supply chain, CI/CD, etc.)workflow_dispatchtriggerRequired Secrets
ANTHROPIC_API_KEY— Anthropic API key for Claude CodeSLACK_WEBHOOK_URL— Slack incoming webhook for notificationsTest plan
ANTHROPIC_API_KEYsecret in repo settingsSLACK_WEBHOOK_URLsecret in repo settingsJira: ROSAENG-64876
🤖 Generated with Claude Code
Summary by CodeRabbit