[issues/249] Defer QA version naming with "Unreleased" placeholder#605
Merged
Conversation
…eholder Mid-cycle SemVer pivots (e.g., 1.1.0 → 2.0.0) force renaming the QA YAML, editing headers, and chasing references across consumer scripts. Extending the `[Unreleased]` convention from CHANGELOG to the QA tooling lets the version stay deferred until finalize-release locks it in. Both generators now branch on `nextTargetVersion == "Unreleased"`: filenames drop the `v` prefix and headers render without it, while SemVer values keep the existing `v`-prefixed form for forward compatibility with the future finalize step. The QA test plan generator also stops no-op-ing when the output file exists so a header refresh can be a separate, minimal-diff commit. Consumer auto-discovery in resolve-qa-labels.js and the verify-qa-scripts.sh glob are loosened so the new unreleased.yaml is picked up. New BATS coverage exercises both filename branches, the in-place refresh, the no-clobber early-exit on the testing-instructions side, and the existing error paths. Benefits: - No more mid-cycle file renames when a version target shifts - Symmetric behavior between generate-qa-test-plan and generate-release-testing-instructions - Forward-compatible: SemVer paths still work for finalize-release - BATS coverage prevents regressions on either branch
Activates the deferred-version pattern by flipping nextTargetVersion from a SemVer placeholder to the literal "Unreleased". The QA YAML header is regenerated in the same commit so the file accurately reflects the new state and the stale self-reference (qa-test-cases-v1.1.0-003.yaml) is corrected. Body content unchanged. Cleaned up the header template at the same time
… pattern CLAUDE.md QA002, TESTING.md, qa-suggest SKILL.md, and plan-integration-test agent all referenced the old version-based filename convention (qa-test-cases-v1.1.0.yaml). Updated to describe qa-test-cases-unreleased.yaml as the trunk-based development filename, with version locking deferred to finalize-release.
The old sed range expression preserved blank lines between the header and body, which combined with the separator echo to add one blank line per regeneration. Dropping the file-exists early-exit in commit 1 surfaced this latent bug. Replaced with a simpler sed -n range that skips directly from test_cases: to EOF. Also updated a stale usage example in generate-qa-issue.sh.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
WalkthroughThe PR converts QA workflows to a trunk-based "Unreleased" mode: package metadata and QA YAML use ChangesTrunk-based QA Test Case Workflow
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
tests/shell/generate-qa-test-plan.bats (1)
124-173: ⚡ Quick winAdd a mixed-file rerun test (versioned + unreleased both present).
Current idempotence coverage misses the case where both YAML variants exist; that’s the scenario most likely to regress source selection.
🧪 Suggested test shape
+@test "re-running prefers existing unreleased.yaml when both unreleased and versioned files exist" { + setup_fixture + write_package_json <<'EOF' +{ + "version": "1.0.0", + "nextTargetVersion": "Unreleased" +} +EOF + write_yaml "qa-test-cases-v1.0.0.yaml" <<'EOF' +test_cases: + - id: foo-001 + scenario: 'from versioned' + automated: true +EOF + write_yaml "qa-test-cases-unreleased.yaml" <<'EOF' +test_cases: + - id: foo-002 + scenario: 'from unreleased' + automated: true +EOF + + run "$SCRIPT" + [[ "$status" -eq 0 ]] + grep -q "scenario: 'from unreleased'" "$FIXTURE_ROOT/qa/qa-test-cases-unreleased.yaml" + ! grep -q "scenario: 'from versioned'" "$FIXTURE_ROOT/qa/qa-test-cases-unreleased.yaml" +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/shell/generate-qa-test-plan.bats` around lines 124 - 173, Add a new bats test that covers the mixed-file rerun case where both the versioned file and the unreleased file exist: create a `@test` (e.g. "re-run mixed-file idempotent: versioned + unreleased present") which calls setup_fixture, writes package.json with version 1.0.0/nextTargetVersion Unreleased, writes both "qa-test-cases-v1.0.0.yaml" (use minimal_previous_yaml) and a pre-existing "qa-test-cases-unreleased.yaml" (can be a small body or the stale header example), then run "$SCRIPT" twice and assert status 0 both times and that the shasum of "$FIXTURE_ROOT/qa/qa-test-cases-unreleased.yaml" (or the file expected to be overwritten) is identical after first and second runs; reuse helpers used in other tests (setup_fixture, write_yaml, minimal_previous_yaml, SCRIPT) to match existing test style.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/rangelink-vscode-extension/scripts/generate-qa-test-plan.sh`:
- Around line 35-45: The script's file-selection logic (in
generate-qa-test-plan.sh using NEXT_VERSION, BASE_NAME, OUTPUT_FILE and QA_DIR)
should prefer an existing target YAML to avoid clobbering in-progress
"qa-test-cases-unreleased.yaml": before performing discovery or selecting the
highest-versioned file, check if "$OUTPUT_FILE" exists and use it as the
source/output; only if it does not exist proceed with the current
discovery/selection logic (the code block around lines referencing
NEXT_VERSION/BASE_NAME and the later discovery section) so reruns will reuse the
existing unreleased file instead of overwriting it.
In `@packages/rangelink-vscode-extension/scripts/resolve-qa-labels.js`:
- Line 99: The auto-discovery currently filters for names starting with
"qa-test-cases-" then later sorts/picks by version, which causes
"qa-test-cases-unreleased.yaml" to lose to "v*" files; update the selection
logic so after applying the filter (.filter((f) =>
f.startsWith('qa-test-cases-') && f.endsWith('.yaml'))), check explicitly for
the exact filename "qa-test-cases-unreleased.yaml" and return it immediately if
present, otherwise fall back to the existing version-sorting logic; reference
the filter expression and the code path that performs the version sort/pick so
you can add the short-circuit check before the sort.
In `@tests/shell/resolve-qa-labels.bats`:
- Around line 212-224: Add a new Bats test that creates both filename styles and
asserts the versioned file wins; e.g., add a test like resolve-qa-labels:
auto-discovery prefers vX.Y.Z over unreleased that calls setup_fixture, writes
two YAML fixtures "qa-test-cases-unreleased.yaml" (with id unreleased-001) and
"qa-test-cases-v1.2.3.yaml" (with id released-001), runs node "$SCRIPT", checks
status is 0 and that output equals "released-001" to lock the auto-discovery
precedence; use the same structure as the existing test and the same run/assert
pattern.
---
Nitpick comments:
In `@tests/shell/generate-qa-test-plan.bats`:
- Around line 124-173: Add a new bats test that covers the mixed-file rerun case
where both the versioned file and the unreleased file exist: create a `@test`
(e.g. "re-run mixed-file idempotent: versioned + unreleased present") which
calls setup_fixture, writes package.json with version 1.0.0/nextTargetVersion
Unreleased, writes both "qa-test-cases-v1.0.0.yaml" (use minimal_previous_yaml)
and a pre-existing "qa-test-cases-unreleased.yaml" (can be a small body or the
stale header example), then run "$SCRIPT" twice and assert status 0 both times
and that the shasum of "$FIXTURE_ROOT/qa/qa-test-cases-unreleased.yaml" (or the
file expected to be overwritten) is identical after first and second runs; reuse
helpers used in other tests (setup_fixture, write_yaml, minimal_previous_yaml,
SCRIPT) to match existing test style.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 2f541a9f-a9da-482f-9336-cd9743c47b24
📒 Files selected for processing (15)
.claude/agents/plan-integration-test.md.claude/skills/qa-suggest/SKILL.mdCLAUDE.mdpackages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/package.jsonpackages/rangelink-vscode-extension/qa/qa-test-cases-unreleased.yamlpackages/rangelink-vscode-extension/scripts/generate-qa-issue.shpackages/rangelink-vscode-extension/scripts/generate-qa-test-plan.shpackages/rangelink-vscode-extension/scripts/generate-release-testing-instructions.shpackages/rangelink-vscode-extension/scripts/resolve-qa-labels.jspackages/rangelink-vscode-extension/scripts/verify-qa-scripts.shtests/shell/generate-qa-test-plan.batstests/shell/generate-release-testing-instructions.batstests/shell/resolve-qa-labels.batstests/shell/validate-qa-coverage.bats
Comment on lines
+212
to
+224
| @test "resolve-qa-labels: auto-discovery picks qa-test-cases-unreleased.yaml when it is the only file" { | ||
| setup_fixture | ||
| write_yaml "qa-test-cases-unreleased.yaml" <<'EOF' | ||
| test_cases: | ||
| - id: unreleased-001 | ||
| feature: Unreleased | ||
| scenario: Single file | ||
| automated: true | ||
| EOF | ||
| run node "$SCRIPT" | ||
| [[ "$status" -eq 0 ]] | ||
| [[ "$output" == "unreleased-001" ]] | ||
| } |
Contributor
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Add a mixed-file precedence test (unreleased + vX.Y.Z) to lock expected selection.
This new case only validates the single-file path. Please add a fixture with both filename styles and assert which one must win, so auto-discovery precedence is explicit and regression-safe.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/shell/resolve-qa-labels.bats` around lines 212 - 224, Add a new Bats
test that creates both filename styles and asserts the versioned file wins;
e.g., add a test like resolve-qa-labels: auto-discovery prefers vX.Y.Z over
unreleased that calls setup_fixture, writes two YAML fixtures
"qa-test-cases-unreleased.yaml" (with id unreleased-001) and
"qa-test-cases-v1.2.3.yaml" (with id released-001), runs node "$SCRIPT", checks
status is 0 and that output equals "released-001" to lock the auto-discovery
precedence; use the same structure as the existing test and the same run/assert
pattern.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…sorts Both generate-qa-test-plan.sh and resolve-qa-labels.js sort candidate files with ASCII ordering where 'unreleased' (u=117) sorts before 'v' (v=118), causing versioned files to win discovery when both coexist. This would clobber in-progress unreleased edits once a versioned file exists (post-finalize). Fixed by short-circuiting: prefer the existing target file in the shell script, and return the unreleased file immediately in the JS auto-discovery. Ref: #605 (review)
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Owner
Author
|
@coderabbitai full review -- I was rate-limited on earlier run |
Contributor
|
(ᴗ˳ᴗ) 🐇 ✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
tests/shell/generate-qa-test-plan.bats (1)
175-200: ⚡ Quick winAdd a regression test for numeric SemVer ordering in fallback selection.
Current tests don’t guard against
v1.10.0vsv1.9.0ordering mistakes when selecting the carry-forward source.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/shell/generate-qa-test-plan.bats` around lines 175 - 200, Add a regression test that verifies numeric SemVer ordering when selecting the carry-forward source: in the existing test "re-running prefers existing unreleased.yaml when both unreleased and versioned files exist" (or add a new test with a similar name), set package.json version to "1.10.0" and create two versioned YAML fixtures (e.g., "qa-test-cases-v1.9.0.yaml" and "qa-test-cases-v1.10.0.yaml") with distinct scenarios, plus "qa-test-cases-unreleased.yaml"; run "$SCRIPT" and assert the script prefers the correct numeric-latest version (v1.10.0) as the fallback (or prefers unreleased when intended) by grepping for the expected scenario and ensuring the other scenario is not present.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/qa-suggest/SKILL.md:
- Line 147: Update the scratchpad filename and header templates to handle a
locked SemVer by adding conditional logic around nextTargetVersion: when
nextTargetVersion === "Unreleased" keep the current header ("v<version> →
Unreleased") and filename pattern (NNNN-qa-suggest.txt); otherwise format the
header as "v<version> → v<nextTargetVersion>" and alter the filename to include
the target version suffix (e.g., NNNN-qa-suggest-v<nextTargetVersion>.txt).
Locate and change the templates that produce the scratchpad path
`.claude-work/issues/<ID>/scratchpads/NNNN-qa-suggest.txt` and the header that
currently renders "v<version> → Unreleased" to use this conditional logic using
the variables nextTargetVersion and version.
In `@packages/rangelink-vscode-extension/scripts/generate-qa-test-plan.sh`:
- Around line 52-66: The current PREVIOUS_YAML selection sorts the normalized
basename lexicographically and can pick the wrong semantic version (e.g.,
v1.10.0 vs v1.9.0); update the pipeline that emits and sorts the "base<TAB>file"
pairs (the block assigning PREVIOUS_YAML and the for-loop that prints normalized
names) to perform a semantic/version-aware sort (use the version sort option)
instead of plain lexicographic sort so the highest semver is chosen; keep the
existing normalization logic (adding -000 for unsuffixed names) but replace the
final sort invocation with a version-aware sort of the first field before tail
-1 and cut -f2.
---
Nitpick comments:
In `@tests/shell/generate-qa-test-plan.bats`:
- Around line 175-200: Add a regression test that verifies numeric SemVer
ordering when selecting the carry-forward source: in the existing test
"re-running prefers existing unreleased.yaml when both unreleased and versioned
files exist" (or add a new test with a similar name), set package.json version
to "1.10.0" and create two versioned YAML fixtures (e.g.,
"qa-test-cases-v1.9.0.yaml" and "qa-test-cases-v1.10.0.yaml") with distinct
scenarios, plus "qa-test-cases-unreleased.yaml"; run "$SCRIPT" and assert the
script prefers the correct numeric-latest version (v1.10.0) as the fallback (or
prefers unreleased when intended) by grepping for the expected scenario and
ensuring the other scenario is not present.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9550da10-2a19-40b5-9311-6ba73d32fed0
📒 Files selected for processing (15)
.claude/agents/plan-integration-test.md.claude/skills/qa-suggest/SKILL.mdCLAUDE.mdpackages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/package.jsonpackages/rangelink-vscode-extension/qa/qa-test-cases-unreleased.yamlpackages/rangelink-vscode-extension/scripts/generate-qa-issue.shpackages/rangelink-vscode-extension/scripts/generate-qa-test-plan.shpackages/rangelink-vscode-extension/scripts/generate-release-testing-instructions.shpackages/rangelink-vscode-extension/scripts/resolve-qa-labels.jspackages/rangelink-vscode-extension/scripts/verify-qa-scripts.shtests/shell/generate-qa-test-plan.batstests/shell/generate-release-testing-instructions.batstests/shell/resolve-qa-labels.batstests/shell/validate-qa-coverage.bats
…plates The fallback YAML selection used lexicographic sort, which ranked v1.9.0 above v1.10.0 because 9 > 1 at offset 3. Replaced with numeric sort by MAJOR, MINOR, PATCH. The qa-suggest SKILL.md now conditionally formats the scratchpad header and filename based on whether nextTargetVersion is "Unreleased" or a locked SemVer. Benefits: - Correct carry-forward source when version components cross the 10 boundary - Scratchpad naming reflects actual version state during locked-version QA cycles - Regression test prevents backsliding on numeric sort Ref: #605 (review)
✅ CI / Integration Tests (with extensions) — run summary
|
✅ CI / Test & Validate — run summary
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adopts the CHANGELOG's
[Unreleased]convention across QA tooling:nextTargetVersionnow stays"Unreleased"during trunk-based development instead of pinning a SemVer that may change (e.g., the 1.1.0 → 2.0.0 pivot). The version is locked in only at release time viafinalize-release(future PR). This eliminates mid-cycle file renames when version targets shift.Changes
generate-qa-test-plan.sh— conditional output filename (qa-test-cases-unreleased.yamlvsqa-test-cases-vX.Y.Z.yaml), conditional header rendering, dropped file-exists early-exit so in-place header refresh works, fixed blank-line accumulation bug in the body extraction sedgenerate-release-testing-instructions.sh— symmetric Unreleased support: conditional output filename and internal yaml referencesresolve-qa-labels.js— loosened auto-discovery filter fromqa-test-cases-v*toqa-test-cases-*so the unreleased file is selectableverify-qa-scripts.sh— loosened artifact globs to cover unreleased filenamesqa/qa-test-cases-unreleased.yaml— renamed fromqa-test-cases-v1.1.0.yamlviagit mv; header refreshed to readv1.0.0 → Unreleasedpackage.json—nextTargetVersionchanged from"1.1.0"to"Unreleased"generate-qa-test-plan(9 tests) andgenerate-release-testing-instructions(7 tests), plus targeted additions to existing suitesTest Plan
generate:qa-test-planis idempotent (consecutive runs produce zero diff)generate:qa-issue --dry-runproduces correct "QA Checklist — unreleased" outputvalidate:qa-coveragepasses against the unreleased yamlgenerate:release-testing-instructionsemitsrelease-testing-instructions-unreleased.mdwith correct internal referencesRelated
2.0.0release #249Summary by CodeRabbit
Documentation
Tests
Chores