NO-JIRA: Add improve-ai-quality skill for dashboard#8688
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
|
@bryan-cox: This pull request explicitly references no jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
|
Please specify an area label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
📝 WalkthroughWalkthroughThis PR introduces a skill definition for the jira-agent-dashboard workflow that documents the "improve-ai-quality" Claude skill. The file specifies the skill's purpose, environment connectivity, target repositories, and a detailed 7-step operational flow: collecting classified comment summaries from the dashboard API; analyzing and prioritizing patterns; mapping to relevant code areas; cloning target repos; implementing scoped improvements; opening PRs with structured body requirements; and reporting findings. It also documents required CLI arguments and example command invocations. Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error)
✅ Passed checks (10 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bryan-cox The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md (2)
32-34: 💤 Low valueAdd language identifier to code block.
Markdown linters prefer explicit language identifiers on fenced code blocks for better syntax highlighting and tooling support.
📝 Proposed fix
-``` +```http http://dashboard.jira-agent-dashboard.svc.cluster.local:8080</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.mdaround lines
32 - 34, The fenced code block containing the URL in SKILL.md lacks a language
identifier; update the block delimiter to include a language (e.g., add "http"
after the opening backticks) so the snippet reads as a fenced code block with a
language identifier for the URL shown in the diff (the block currently
containing "http://dashboard.jira-agent-dashboard.svc.cluster.local:8080").</details> --- `252-254`: _💤 Low value_ **Add language identifiers to command example code blocks.** Markdown linters prefer explicit language identifiers on fenced code blocks for better syntax highlighting and tooling support. <details> <summary>📝 Proposed fix</summary> ```diff -``` +```bash /improve-ai-quality 2026-05-01 2026-06-01Or for the default last-7-days window:
-+bash
/improve-ai-qualityAlso applies to: 257-259
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md` around lines 252 - 254, The fenced code blocks showing the slash command examples (e.g., the block containing "/improve-ai-quality 2026-05-01 2026-06-01" and the other block for the default "/improve-ai-quality") lack language identifiers; update those fenced code blocks to include "bash" (i.e., change the opening ``` to ```bash) so markdown linters and syntax highlighters recognize them—apply this change for the example at the shown block and the other occurrence referenced around lines 257-259.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md`:
- Line 187: The heredoc is quoted as <<'EOF' which prevents shell variable
expansion; change the heredoc opener used with --body "$(cat <<'EOF'" to an
unquoted marker (e.g., <<EOF) so variables like ${FROM}, ${TO}, ${COUNT}, and
${ACTIONABLE} are expanded into the PR body; update the heredoc opener in the
command that constructs the --body (the cat <<'EOF' invocation) and leave the
closing EOF unchanged.
---
Nitpick comments:
In `@jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md`:
- Around line 32-34: The fenced code block containing the URL in SKILL.md lacks
a language identifier; update the block delimiter to include a language (e.g.,
add "http" after the opening backticks) so the snippet reads as a fenced code
block with a language identifier for the URL shown in the diff (the block
currently containing
"http://dashboard.jira-agent-dashboard.svc.cluster.local:8080").
- Around line 252-254: The fenced code blocks showing the slash command examples
(e.g., the block containing "/improve-ai-quality 2026-05-01 2026-06-01" and the
other block for the default "/improve-ai-quality") lack language identifiers;
update those fenced code blocks to include "bash" (i.e., change the opening ```
to ```bash) so markdown linters and syntax highlighters recognize them—apply
this change for the example at the shown block and the other occurrence
referenced around lines 257-259.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 374628df-fd3c-43d3-bbf8-b3f897cda24c
📒 Files selected for processing (1)
jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md
|
|
||
| gh pr create --base main \ | ||
| --title "NO-JIRA: Improve AI skill quality based on review comment analysis" \ | ||
| --body "$(cat <<'EOF' |
There was a problem hiding this comment.
Fix heredoc quoting to enable variable expansion.
The heredoc uses <<'EOF' with single quotes, which prevents shell variable expansion. Variables like ${FROM}, ${TO}, ${COUNT}, and ${ACTIONABLE} on lines 195, 196 will appear literally in the PR body instead of being substituted with their values.
🐛 Proposed fix
- --body "$(cat <<'EOF'
+ --body "$(cat <<EOF
## Summary📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| --body "$(cat <<'EOF' | |
| --body "$(cat <<EOF |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.md` at line 187,
The heredoc is quoted as <<'EOF' which prevents shell variable expansion; change
the heredoc opener used with --body "$(cat <<'EOF'" to an unquoted marker (e.g.,
<<EOF) so variables like ${FROM}, ${TO}, ${COUNT}, and ${ACTIONABLE} are
expanded into the PR body; update the heredoc opener in the command that
constructs the --body (the cat <<'EOF' invocation) and leave the closing EOF
unchanged.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8688 +/- ##
=======================================
Coverage 41.43% 41.43%
=======================================
Files 756 756
Lines 93647 93647
=======================================
Hits 38802 38802
Misses 52124 52124
Partials 2721 2721
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Adds a Claude Code skill that analyzes classified review comments from the JIRA Agent Dashboard, identifies recurring patterns in PR feedback, and opens PRs to improve the AI skills and repo configuration that generate those PRs. This closes the feedback loop between human reviewers and the jira-solve / address-review-comments automation. The skill fetches comments from the dashboard API, groups by topic and severity, maps patterns to specific files in hypershift and ai-helpers repos, and opens draft PRs with evidence-backed reasoning for each proposed change. Signed-off-by: Bryan Cox <brcox@redhat.com> Commit-Message-Assisted-by: Claude (via Claude Code)
bryan-cox
force-pushed
the
improve-ai-quality-skill
branch
from
a6073a4 to
0af4d80
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@jira-agent-dashboard/.claude/skills/improve-ai-quality/scripts/fetch-comments.sh`:
- Around line 8-13: The fallback public dashboard URL is hardcoded into the
BASE_URL assignment; replace the literal personal cluster URL with a
configurable value (e.g., read DASHBOARD_PUBLIC_URL or DASHBOARD_BASE_URL env
var) and fall back to a neutral default or fail loudly if not set. Update the
curl check/else branch that sets BASE_URL so it uses the environment variable
(and optionally a safe default like an empty string or a documented public host)
instead of the developer-specific URL, and ensure any downstream code handles an
unset/empty BASE_URL gracefully.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 2ffd9c7b-622e-4aea-be96-3fcd80caa51a
📒 Files selected for processing (2)
jira-agent-dashboard/.claude/skills/improve-ai-quality/SKILL.mdjira-agent-dashboard/.claude/skills/improve-ai-quality/scripts/fetch-comments.sh
| # In-cluster URL (CronJob) or public route (local dev) | ||
| if curl -s --connect-timeout 2 "http://dashboard.jira-agent-dashboard.svc.cluster.local:8080/healthz" >/dev/null 2>&1; then | ||
| BASE_URL="http://dashboard.jira-agent-dashboard.svc.cluster.local:8080" | ||
| else | ||
| BASE_URL="https://dashboard-public-jira-agent-dashboard.apps.jira-agent-scraper.brcox.hypershift.devcluster.openshift.com" | ||
| fi |
There was a problem hiding this comment.
Avoid hardcoding a developer-specific public dashboard URL.
At Line 12, the fallback host is tied to a personal/dev cluster domain. That makes this script non-portable and fragile outside that exact environment.
Proposed fix
# In-cluster URL (CronJob) or public route (local dev)
if curl -s --connect-timeout 2 "http://dashboard.jira-agent-dashboard.svc.cluster.local:8080/healthz" >/dev/null 2>&1; then
BASE_URL="http://dashboard.jira-agent-dashboard.svc.cluster.local:8080"
else
- BASE_URL="https://dashboard-public-jira-agent-dashboard.apps.jira-agent-scraper.brcox.hypershift.devcluster.openshift.com"
+ BASE_URL="${DASHBOARD_PUBLIC_BASE_URL:-https://dashboard-public-jira-agent-dashboard.apps.<cluster-domain>}"
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # In-cluster URL (CronJob) or public route (local dev) | |
| if curl -s --connect-timeout 2 "http://dashboard.jira-agent-dashboard.svc.cluster.local:8080/healthz" >/dev/null 2>&1; then | |
| BASE_URL="http://dashboard.jira-agent-dashboard.svc.cluster.local:8080" | |
| else | |
| BASE_URL="https://dashboard-public-jira-agent-dashboard.apps.jira-agent-scraper.brcox.hypershift.devcluster.openshift.com" | |
| fi | |
| # In-cluster URL (CronJob) or public route (local dev) | |
| if curl -s --connect-timeout 2 "http://dashboard.jira-agent-dashboard.svc.cluster.local:8080/healthz" >/dev/null 2>&1; then | |
| BASE_URL="http://dashboard.jira-agent-dashboard.svc.cluster.local:8080" | |
| else | |
| BASE_URL="${DASHBOARD_PUBLIC_BASE_URL}" | |
| if [ -z "$BASE_URL" ]; then | |
| echo "Error: DASHBOARD_PUBLIC_BASE_URL environment variable not set" >&2 | |
| exit 1 | |
| fi | |
| fi |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@jira-agent-dashboard/.claude/skills/improve-ai-quality/scripts/fetch-comments.sh`
around lines 8 - 13, The fallback public dashboard URL is hardcoded into the
BASE_URL assignment; replace the literal personal cluster URL with a
configurable value (e.g., read DASHBOARD_PUBLIC_URL or DASHBOARD_BASE_URL env
var) and fall back to a neutral default or fail loudly if not set. Update the
curl check/else branch that sets BASE_URL so it uses the environment variable
(and optionally a safe default like an empty string or a documented public host)
instead of the developer-specific URL, and ensure any downstream code handles an
unset/empty BASE_URL gracefully.
2 participants
What this PR does / why we need it:
Adds a Claude Code skill (
improve-ai-quality) to the jira-agent-dashboardthat closes the feedback loop between human reviewers and the AI automation
(jira-solve / address-review-comments).
The skill:
Each PR it opens must include verbatim reviewer quotes and clear reasoning
connecting the comment pattern to the proposed fix, so reviewers can verify
the changes make sense.
Success metric: fewer review comments per PR over time.
Which issue(s) this PR fixes:
N/A — new skill for the JIRA Agent Dashboard
Special notes for your reviewer:
This is a skill definition only (a markdown file). No Go code, no deployed
infrastructure changes. The skill will be invoked manually or via a future
CronJob.
Checklist:
Generated with Claude Code
Summary by CodeRabbit