diff --git a/.claude/00_START_HERE.md b/.claude/00_START_HERE.md new file mode 100644 index 0000000000..accee92b48 --- /dev/null +++ b/.claude/00_START_HERE.md @@ -0,0 +1,7 @@ +# MCO QE Tooling - Start Here + +**New to this tooling?** Read [README.md](README.md) - complete guide in one file. + +**Quick setup**: Run `./scripts/setup.sh --verify` and follow § Setup in README. + +That's it! diff --git a/.claude/JIRA_INTEGRATION.md b/.claude/JIRA_INTEGRATION.md new file mode 100644 index 0000000000..1c587291b6 --- /dev/null +++ b/.claude/JIRA_INTEGRATION.md @@ -0,0 +1,405 @@ +# Jira Integration Guide + +## Overview + +Fetch Jira issues from redhat.atlassian.net and use for Polarion TC creation. + +**No MCP required** - uses direct Atlassian REST API. + +## Setup + +### Get Jira API Token + +1. Go to: https://id.atlassian.com/manage-profile/security/api-tokens +2. Create API token +3. Copy token + +### Configure .env + +```bash +# Edit .env at repo root +JIRA_URL=https://redhat.atlassian.net +JIRA_EMAIL=you@redhat.com +JIRA_TOKEN=your_api_token_here +``` + +**Note**: Atlassian Cloud requires Basic authentication with email:token credentials. + +### Verify + +```bash +python3 .claude/scripts/fetch_jira.py MCO-2221 +``` + +## Fetch Jira Issue + +### Basic Usage + +```bash +python3 .claude/scripts/fetch_jira.py OCPBUGS-74223 +``` + +**Output**: +- `/tmp/test-specs/jira/OCPBUGS-74223.json` (draft format) +- JSON to stdout + +### What Gets Fetched + +| Field | Jira Source | Draft Field | +|---|---|---| +| Summary | issue.fields.summary | title (with [MCO][KEY] prefix) | +| Description | issue.fields.description | description | +| Fix Versions | issue.fields.fixVersions[0] | version (X.Y from X.Y.Z) | +| Acceptance Criteria | customfield_12311140 | metadata.acceptance_criteria | +| Issue Type | issue.fields.issuetype.name | metadata.jira_type | +| Labels | issue.fields.labels | metadata.labels | + +### Draft Format + +```json +{ + "title": "[MCO][OCPBUGS-74223] Add support for X feature", + "description": "This feature adds support for...", + "component": "Machine Config Operator", + "sub_team": "MCO", + "products": "OCP", + "test_type": "Functional", + "version": "4.18", + "trello_jira": "OCPBUGS-74223", + "test_steps": [], + "metadata": { + "source_jira": "OCPBUGS-74223", + "jira_type": "Bug", + "labels": ["qe-test-coverage"], + "acceptance_criteria": "Verify that feature X works correctly..." + } +} +``` + +## Use Cases + +### Use Case 1: Jira Only + +Create TC from Jira metadata, add steps manually later. + +```bash +# Fetch +python3 scripts/fetch_jira.py OCPBUGS-74223 + +# Validate (will warn about empty test_steps) +python3 scripts/validate_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json + +# Create (empty steps - add later) +python3 scripts/create_polarion_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json +``` + +**When to use**: +- Jira has good description and acceptance criteria +- Test steps not yet defined +- Want to create TC placeholder early + +### Use Case 2: Jira + GitHub PR (Hybrid) + +**Best approach** - Jira metadata + GitHub test steps. + +```bash +# Hybrid creation +python3 scripts/create_tc_hybrid.py OCPBUGS-74223 --from-pr 5691 --comment 4054832254 +``` + +**Workflow**: +1. Fetches Jira issue → metadata +2. Fetches GitHub PR comment → test steps +3. Merges drafts +4. Validates +5. Creates complete TC + +**When to use**: +- Jira has fix versions and acceptance criteria +- GitHub PR has QE test steps +- Want accurate metadata + executable steps + +### Use Case 3: Manual Draft with Jira Reference + +```bash +# Fetch Jira for reference +python3 scripts/fetch_jira.py OCPBUGS-74223 + +# Copy values to manual draft +cat /tmp/test-specs/jira/OCPBUGS-74223.json | jq '.version, .metadata.acceptance_criteria' + +# Create manual draft with copied values +vim /tmp/test-specs/drafts/my_tc.json +``` + +## Hybrid Mode (Jira + PR) + +### Why Hybrid? + +**Jira strengths**: +- ✓ Accurate fix versions +- ✓ Acceptance criteria +- ✓ Issue type (Bug, Feature, etc.) +- ✓ Labels + +**GitHub PR strengths**: +- ✓ Executable test steps +- ✓ Commands with expected output +- ✓ Environment details + +**Hybrid = Best of both** + +### Hybrid Workflow + +```bash +python3 scripts/create_tc_hybrid.py OCPBUGS-74223 --from-pr 5691 +``` + +**Merge logic**: +- Title: `[MCO][OCPBUGS-74223] ` +- Version: From Jira fix versions +- Description: Jira description + acceptance criteria +- Test steps: From GitHub PR comment +- Jira ID: From Jira (obviously) + +**Output**: `/tmp/test-specs/drafts/OCPBUGS-74223-pr5691.json` + +### Example + +**Jira OCPBUGS-74223**: +- Summary: "Add support for feature X" +- Fix Version: 4.18.0 +- Acceptance Criteria: "Verify X works in Y scenarios" +- Description: "Feature X allows..." + +**GitHub PR 5691 QE Comment**: +```markdown +## Test Steps + +- Create MachineConfig with feature X + ```bash + oc create -f mc.yaml + ``` + +- Verify feature X applied + ```bash + $ oc get mc | grep feature-x + feature-x 3.2.0 15s + ``` +``` + +**Merged Draft**: +```json +{ + "title": "[MCO][OCPBUGS-74223] Add support for feature X", + "description": "Feature X allows...\n\nAcceptance Criteria:\nVerify X works in Y scenarios", + "version": "4.18", + "trello_jira": "OCPBUGS-74223", + "test_steps": [ + { + "step": "Create MachineConfig with feature X", + "expected_result": "oc create -f mc.yaml" + }, + { + "step": "Verify feature X applied", + "expected_result": "$ oc get mc | grep feature-x\nfeature-x 3.2.0 15s" + } + ] +} +``` + +## Jira API Details + +### Authentication + +Uses Atlassian REST API with Bearer token: +``` +Authorization: Bearer +``` + +**Not** Red Hat SSO - this is Atlassian's API token system. + +### Endpoint + +``` +GET https://redhat.atlassian.net/rest/api/3/issue/{issue-key} +``` + +### ADF Format + +Jira returns description in ADF (Atlassian Document Format): + +```json +{ + "type": "doc", + "content": [ + { + "type": "paragraph", + "content": [ + {"type": "text", "text": "This is a description"} + ] + } + ] +} +``` + +Script extracts plain text automatically. + +### Custom Fields + +Acceptance Criteria field name varies by project: +- Try: `customfield_12311140` +- Try: `customfield_12315140` +- Try: `acceptanceCriteria` + +Script attempts common field names. + +### Fix Versions + +Extract X.Y from X.Y.Z: +- Jira: "4.18.0" → Draft: "4.18" +- Jira: "4.22" → Draft: "4.22" + +## Integration with Other Tools + +### With validate_tc.py + +```bash +python3 scripts/fetch_jira.py OCPBUGS-74223 +python3 scripts/validate_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json +``` + +**Expected warnings**: +- "No test steps defined" (unless using hybrid mode) + +### With create_polarion_tc.py + +```bash +python3 scripts/fetch_jira.py OCPBUGS-74223 +python3 scripts/create_polarion_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json +``` + +Creates TC with metadata, empty test steps. + +### With update_polarion_tc.py + +```bash +# Create from Jira (empty steps) +python3 scripts/create_polarion_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json +# Returns: OCP-88999 + +# Add test steps later +python3 scripts/update_polarion_tc.py OCP-88999 \ + --steps-file /tmp/test-specs/steps/feature-x.json +``` + +## Troubleshooting + +### Error: "Authentication failed" + +**Fix**: +```bash +# Regenerate token at: +# https://id.atlassian.com/manage-profile/security/api-tokens + +# Update .env +JIRA_TOKEN=new_token_here +``` + +### Error: "Jira issue not found" + +**Fix**: +- Verify issue exists: https://redhat.atlassian.net/browse/OCPBUGS-74223 +- Check issue key format: `OCPBUGS-74223` (not `ocpbugs-74223`) +- Ensure you have access to the issue + +### Warning: "Acceptance Criteria: Not Found" + +**Not critical** - means the custom field wasn't found. + +**Fix**: +- Check Jira issue has acceptance criteria field +- Field name may vary by project +- Can add manually to draft if needed + +### Version not found + +**Symptom**: `version` field is empty + +**Fix**: +```bash +# Set manually in draft or during creation +python3 scripts/update_polarion_tc.py OCP-88999 --version 4.18 +``` + +## Command Reference + +```bash +# Fetch Jira only +python3 scripts/fetch_jira.py OCPBUGS-74223 + +# Hybrid (Jira + PR) +python3 scripts/create_tc_hybrid.py OCPBUGS-74223 --from-pr 5691 + +# Via Claude Code +/mco-create-tc-from-jira https://redhat.atlassian.net/browse/OCPBUGS-74223 +/mco-create-tc-from-pr https://github.com/openshift/machine-config-operator/pull/5691 +``` + +## Comparison: Sources + +| Source | Title/Version | Description | Test Steps | +|---|---|---|---| +| GitHub PR only | From PR (extract Jira) | From PR | ✓ From QE comment | +| Jira only | ✓ From Jira | ✓ From Jira | ✗ Empty | +| Hybrid (Jira+PR) | ✓ From Jira | ✓ From Jira | ✓ From PR | + +**Recommendation**: Use hybrid mode for best results. + +## Example Workflow + +```bash +# 1. Find Jira issue +# https://redhat.atlassian.net/browse/OCPBUGS-74223 + +# 2. Find PR with QE test +# https://github.com/openshift/machine-config-operator/pull/5691 + +# 3. Create hybrid TC +python3 .claude/scripts/create_tc_hybrid.py OCPBUGS-74223 \ + --from-pr 5691 \ + --comment 4054832254 + +# Output: +# [1/5] Fetching Jira issue: OCPBUGS-74223... +# ✓ Title: [MCO][OCPBUGS-74223] Add support for feature X +# ✓ Version: 4.18 +# +# [2/5] Fetching GitHub PR #5691... +# ✓ Test Steps: 5 +# +# [3/5] Merging Jira + PR data... +# ✓ Merged draft: /tmp/test-specs/drafts/OCPBUGS-74223-pr5691.json +# +# [4/5] Validating draft... +# ✓ Validation passed +# +# [5/5] Creating Polarion TC... +# ✓ Created: OCP-88999 +# URL: https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-88999 +``` + +## Limitations + +1. **Acceptance Criteria field**: Custom field name varies by project +2. **ADF parsing**: Complex formatting may not convert perfectly +3. **Authentication**: Requires Atlassian API token (not Red Hat SSO) +4. **Access**: Must have access to Jira issue + +## Next Steps + +After fetching Jira: +1. Review draft metadata +2. If missing test steps, use hybrid mode or add manually +3. Validate and create TC +4. Verify in Polarion diff --git a/.claude/PILOT_PR5691.md b/.claude/PILOT_PR5691.md new file mode 100644 index 0000000000..e514d8d060 --- /dev/null +++ b/.claude/PILOT_PR5691.md @@ -0,0 +1,245 @@ +# Pilot Test: PR 5691 → Create Polarion TC + +## Objective + +Test full workflow: GitHub PR QE comment → validate → create Polarion TC. + +**Target PR**: openshift/machine-config-operator#5691 +**Comment ID**: 4054832254 (QE pre-merge testing) + +## Prerequisites + +1. POLARION_TOKEN configured in `.env` +2. GITHUB_TOKEN configured (optional, for rate limits) +3. Setup verified: `./scripts/setup.sh --verify` + +## Workflow Steps + +### Step 1: Fetch PR Comment + +```bash +cd .claude +python3 scripts/fetch_github_pr.py \ + --repo openshift/machine-config-operator \ + --pr 5691 \ + --comment 4054832254 +``` + +**Expected**: +- ✓ Draft saved to: `/tmp/test-specs/drafts/pr-5691.json` +- Title extracted from PR (with `[MCO][JIRA-ID]` prefix if Jira found) +- Environment parsed (version, platform) +- Test steps parsed (bullets → step, code → expected_result) + +**Verify**: +```bash +cat /tmp/test-specs/drafts/pr-5691.json | jq '.title, .trello_jira, .version, (.test_steps | length)' +``` + +### Step 2: Validate Draft + +```bash +python3 scripts/validate_tc.py /tmp/test-specs/drafts/pr-5691.json +``` + +**Expected**: +- ✓ VALIDATION PASSED (or specific errors to fix) +- Title matches `^\[MCO\]\[.+\] .+` +- All required fields present +- Test steps have commands (not narrative-only) + +**If validation fails**: +1. Review errors +2. Edit `/tmp/test-specs/drafts/pr-5691.json` +3. Re-validate +4. Do NOT proceed until valid + +### Step 3: Review Draft + +```bash +cat /tmp/test-specs/drafts/pr-5691.json | jq '.' +``` + +**Check**: +- [ ] Title: `[MCO][JIRA-ID] ...` +- [ ] Component: `Machine Config Operator` +- [ ] Sub Team: `MCO` +- [ ] Products: `OCP` +- [ ] Test Type: `Functional` (or appropriate) +- [ ] Version: (from PR comment or set manually) +- [ ] Trello/Jira: (from PR title) +- [ ] Test Steps: Each step has: + - Non-empty `step` field + - `expected_result` with commands/YAML + +### Step 4: Dry Run + +```bash +python3 scripts/create_polarion_tc.py /tmp/test-specs/drafts/pr-5691.json --dry-run +``` + +**Expected**: +``` +Validating draft... +✓ Validation passed + +================================================================================ +TEST CASE SUMMARY +================================================================================ +Title: [MCO][OCPBUGS-xxxxx] ... +Component: Machine Config Operator +Sub Team: MCO +Products: OCP +Test Type: Functional +Version: 4.22 +Jira: OCPBUGS-xxxxx +Test Steps: 5 +================================================================================ + +✓ DRY RUN: Draft is valid and ready for creation +``` + +### Step 5: Create Test Case + +```bash +python3 scripts/create_polarion_tc.py /tmp/test-specs/drafts/pr-5691.json +``` + +**Expected workflow**: +1. Creating test case... + ✓ Created: OCP-xxxxx + +2. Patching custom fields... + ✓ Patched 6 custom fields + +3. Adding X test steps... + ✓ Added X test steps + +**Expected output**: +``` +================================================================================ +✓ TEST CASE CREATED SUCCESSFULLY +================================================================================ +ID: OCP-xxxxx +URL: https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-xxxxx +================================================================================ +``` + +### Step 6: Verify in Polarion + +1. Open URL: `https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-xxxxx` + +2. **Verify metadata**: + - [ ] Title: `[MCO][JIRA-ID] ...` + - [ ] Status: Draft + - [ ] Component: Machine Config Operator + - [ ] Sub Team: MCO + - [ ] Products: OCP + - [ ] Test Type: Functional + - [ ] Version: 4.22 + - [ ] Trello/Jira: OCPBUGS-xxxxx + +3. **Verify test steps**: + - [ ] All steps present + - [ ] Step column has action descriptions + - [ ] Expected Result column has commands/output + - [ ] No HTML tags visible + - [ ] YAML/code formatting preserved + +4. **Compare with good format** (OCP-88122.pdf): + - [ ] Title structure matches + - [ ] All required fields filled + - [ ] Test steps have real commands + +## Expected Challenges + +### Challenge 1: Jira ID Not Found + +**Symptom**: `trello_jira` field empty + +**Fix**: +```bash +# Edit draft +vim /tmp/test-specs/drafts/pr-5691.json +# Set: "trello_jira": "OCPBUGS-xxxxx" + +# Re-validate +python3 scripts/validate_tc.py /tmp/test-specs/drafts/pr-5691.json +``` + +### Challenge 2: Version Not Parsed + +**Symptom**: `version` field empty + +**Fix**: +```bash +# Edit draft +vim /tmp/test-specs/drafts/pr-5691.json +# Set: "version": "4.22" + +# Re-validate +``` + +### Challenge 3: Test Steps Narrative-Only + +**Symptom**: Validation warns "Expected result appears to be narrative-only" + +**Fix**: +```bash +# Edit draft expected_result to include commands +# Before: "The feature is applied" +# After: "$ oc get mc | grep feature\nfeature 3.2.0 15s" +``` + +### Challenge 4: API Error During Creation + +**Symptom**: "Failed to create test case: ..." + +**Fix**: +1. Check POLARION_TOKEN validity +2. Verify network connection +3. Check Polarion API status +4. Review error message for field issues +5. Draft file preserved - can retry after fix + +## Success Criteria + +- [ ] Draft fetched from PR comment +- [ ] Validation passes +- [ ] All required fields populated +- [ ] Test steps have commands +- [ ] Dry run shows correct summary +- [ ] Test case created successfully +- [ ] OCP-xxxxx URL returned +- [ ] Polarion shows complete TC +- [ ] Format matches OCP-88122.pdf (good) +- [ ] No empty fields like wrong.pdf (OCP-88941) + +## Comparison Matrix + +| Field | wrong.pdf (bad) | PR 5691 (expected good) | +|---|---|---| +| Title prefix | ❌ Missing | ✓ [MCO][JIRA-ID] | +| Component | ❌ Empty | ✓ Machine Config Operator | +| Sub Team | ❌ Empty | ✓ MCO | +| Products | ❌ Empty | ✓ OCP | +| Test Type | ❌ Empty | ✓ Functional | +| Version | ❌ Empty | ✓ 4.22 | +| Trello/Jira | ❌ Empty | ✓ OCPBUGS-xxxxx | +| Expected Results | ❌ Narrative | ✓ Commands | + +## Next Steps After Pilot + +If pilot succeeds: +1. Document actual PR 5691 comment format +2. Update fetch_github_pr.py parsing if needed +3. Add more test cases from different PR formats +4. Integrate into /mco-create-tc-from-pr command +5. Add to CI/automation workflow + +## Notes + +- Comment ID 4054832254 format may vary from typical QE comments +- Parser may need adjustment based on actual comment structure +- Jira ID extraction regex: `(OCPBUGS-\d+|MCO-\d+)` +- Environment parsing regex: flexible for variations diff --git a/.claude/QUICKSTART.md b/.claude/QUICKSTART.md new file mode 100644 index 0000000000..2ef29ed3e7 --- /dev/null +++ b/.claude/QUICKSTART.md @@ -0,0 +1,170 @@ +# MCO QE Tooling - Quick Start + +Get started in 5 minutes. + +## 1. Setup (One-time) + +```bash +cd .claude +./scripts/setup.sh +``` + +## 2. Configure Tokens + +Edit `.env` at repo root: + +```bash +# Required +POLARION_TOKEN=your_token_here + +# Optional (for GitHub) +GITHUB_TOKEN=your_github_token + +# Optional (for Jira) +JIRA_TOKEN=your_jira_token + +# Optional (for updating test steps) +POLARION_USERNAME=your_username +POLARION_PASSWORD=your_password +``` + +**Get tokens**: +- Polarion: https://polarion.engineering.redhat.com/polarion/#/user/me +- GitHub: https://github.com/settings/tokens +- Jira: https://id.atlassian.com/manage-profile/security/api-tokens + +## 3. Verify + +```bash +cd .claude +./scripts/setup.sh --verify +``` + +Should see: +``` +✓ Python 3.14.5 found +✓ requests already installed +✓ POLARION_TOKEN configured +✓ Polarion connection successful (HTTP 200) + Project: OpenShift (OSE) +``` + +## 4. Try It Out + +### Create TC from GitHub PR + +```bash +# Fetch PR comment +python3 scripts/fetch_github_pr.py \ + --repo openshift/machine-config-operator \ + --pr 5691 \ + --comment 4054832254 + +# Validate +python3 scripts/validate_tc.py /tmp/test-specs/drafts/pr-5691.json + +# Create (dry run first) +python3 scripts/create_polarion_tc.py /tmp/test-specs/drafts/pr-5691.json --dry-run + +# Create (for real) +python3 scripts/create_polarion_tc.py /tmp/test-specs/drafts/pr-5691.json +``` + +### Create TC from Jira + GitHub (Hybrid) + +```bash +# One command - combines Jira metadata + PR test steps +python3 scripts/create_tc_hybrid.py OCPBUGS-74223 --from-pr 5691 +``` + +### Fix Existing TC (wrong.pdf scenario) + +```bash +# Fix empty metadata fields +python3 scripts/update_polarion_tc.py OCP-88941 --fix-metadata \ + --version 4.18 \ + --trello-jira OCPBUGS-83830 + +# Fix title +python3 scripts/update_polarion_tc.py OCP-88941 \ + --title "[MCO][OCPBUGS-83830] Verify MCD password behavior" +``` + +### Fetch Polarion TC for Automation + +```bash +# Fetch TC +python3 scripts/fetch_polarion.py OCP-88122 + +# Use with /automate-test +/automate-test OCP-88122 "mco osstream" disruptive +``` + +## Common Workflows + +### Workflow 1: PR → Polarion + +1. Find PR with QE comment +2. Get PR number and comment ID +3. Run: `python3 scripts/create_tc_hybrid.py JIRA-KEY --from-pr PR-NUM` +4. Get OCP-xxxxx URL + +### Workflow 2: Jira Only + +1. Find Jira issue +2. Run: `python3 scripts/fetch_jira.py JIRA-KEY` +3. Create: `python3 scripts/create_polarion_tc.py /tmp/test-specs/jira/JIRA-KEY.json` +4. Add steps later + +### Workflow 3: Fix Bad TC + +1. Identify TC with empty fields +2. Run: `python3 scripts/update_polarion_tc.py OCP-ID --fix-metadata` +3. Verify in Polarion + +## File Locations + +- **Config**: `.env` (repo root) +- **Scripts**: `.claude/scripts/` +- **Output**: `/tmp/test-specs/` +- **Docs**: `.claude/*.md` + +## Quick Reference + +| Task | Command | +|---|---| +| Setup | `./scripts/setup.sh --verify` | +| Fetch TC | `python3 scripts/fetch_polarion.py OCP-88122` | +| Fetch PR | `python3 scripts/fetch_github_pr.py --repo org/repo --pr NUM` | +| Fetch Jira | `python3 scripts/fetch_jira.py JIRA-KEY` | +| Validate | `python3 scripts/validate_tc.py draft.json` | +| Create | `python3 scripts/create_polarion_tc.py draft.json` | +| Hybrid | `python3 scripts/create_tc_hybrid.py JIRA-KEY --from-pr NUM` | +| Update | `python3 scripts/update_polarion_tc.py OCP-ID --fix-metadata` | + +## Next Steps + +1. Read `.claude/README.md` for full workflows +2. Try pilot test: `.claude/PILOT_PR5691.md` +3. See complete guide: `.claude/COMPLETE_WORKFLOW.md` + +## Troubleshooting + +**"POLARION_TOKEN not configured"** +→ Add token to `.env`, run `./scripts/setup.sh --verify` + +**"Validation failed"** +→ Review errors, edit draft, re-validate + +**"SOAP API requires username and password"** +→ Add POLARION_USERNAME/PASSWORD to `.env` (only for test step updates) + +## Help + +- Full docs: `.claude/README.md` +- Fetch guide: `.claude/FETCH_USAGE.md` +- Create guide: `.claude/CREATE_TC_USAGE.md` +- Update guide: `.claude/UPDATE_TC_USAGE.md` +- Jira guide: `.claude/JIRA_INTEGRATION.md` +- Complete workflow: `.claude/COMPLETE_WORKFLOW.md` +- Final summary: `.claude/FINAL_SUMMARY.md` diff --git a/.claude/README.md b/.claude/README.md new file mode 100644 index 0000000000..ac5e1e3486 --- /dev/null +++ b/.claude/README.md @@ -0,0 +1,630 @@ +# MCO QE Tooling + +Automation scripts for Machine Config Operator quality engineering workflows: fetch, create, and update Polarion test cases from multiple sources. + +**Quick start**: New user? Run `./scripts/setup.sh --verify` and see § Setup below. + +--- + +## Table of Contents + +1. [Setup](#setup) +2. [Commands](#commands) +3. [Workflows](#workflows) +4. [TC Format](#tc-format-good-vs-bad) +5. [Scripts Reference](#scripts-reference) +6. [Troubleshooting](#troubleshooting) +7. [Claude Code Examples](#claude-code-examples) + +--- + +## Setup + +### Prerequisites + +- Python 3.9+ +- `requests` library: `pip install requests` +- Polarion API token +- Optional: GitHub token, Jira token + +### Initial Setup + +**1. Run setup script**: +```bash +cd .claude +./scripts/setup.sh +``` + +Creates `/tmp/test-specs/` directory structure. + +**2. Configure tokens**: + +Create `.env` at **repo root** (gitignored): +```bash +# Required for Polarion +POLARION_URL=https://polarion.engineering.redhat.com +POLARION_TOKEN=your_personal_access_token_here + +# Optional: For updating test steps on existing TCs (SOAP API) +POLARION_USERNAME=your_username +POLARION_PASSWORD=your_password + +# Optional: For GitHub (public repos work without token) +GITHUB_TOKEN=your_github_token_here + +# Optional: For Jira integration +JIRA_URL=https://redhat.atlassian.net +JIRA_EMAIL=you@redhat.com +JIRA_TOKEN=your_jira_api_token_here +``` + +**Get tokens**: +- **Polarion**: https://polarion.engineering.redhat.com/polarion/#/user/me → Personal Access Tokens +- **GitHub**: https://github.com/settings/tokens → Generate new token (public_repo scope) +- **Jira**: https://id.atlassian.com/manage-profile/security/api-tokens → Create API token + +**3. Verify setup**: +```bash +./scripts/setup.sh --verify +``` + +Expected output: +``` +✓ Python 3.14.5 found +✓ requests already installed +✓ POLARION_TOKEN configured +✓ Polarion connection successful (HTTP 200) + Project: OpenShift (OSE) +``` + +--- + +## Commands + +Active commands (use via Claude Code or call scripts directly): + +### /mco-fetch + +Fetch test cases from Polarion, GitHub PRs, or Jira. + +```bash +# Fetch Polarion TC +python3 scripts/fetch_polarion.py OCP-88122 + +# Fetch GitHub PR comment +python3 scripts/fetch_github_pr.py \ + --repo openshift/machine-config-operator \ + --pr 5691 \ + --comment 4054832254 + +# Fetch Jira issue +python3 scripts/fetch_jira.py OCPBUGS-74223 +``` + +**Output**: JSON draft in `/tmp/test-specs/drafts/` or `/tmp/test-specs/jira/` + +### /mco-create-tc-from-pr + +Create Polarion test case from a GitHub PR QE comment or review. + +```bash +# From PR comment (specific anchor) +/mco-create-tc-from-pr https://github.com/openshift/machine-config-operator/pull/5691#issuecomment-4054832254 + +# From PR (scans all comments/reviews for QE content) +/mco-create-tc-from-pr https://github.com/openshift/machine-config-operator/pull/5691 +``` + +**Output**: OCP-xxxxx URL + +### /mco-create-tc-from-jira + +Create Polarion test case from a Jira issue or QE comment. + +```bash +# From Jira issue (scans comments for QE verification) +/mco-create-tc-from-jira https://redhat.atlassian.net/browse/OCPBUGS-86695 + +# From specific Jira comment +/mco-create-tc-from-jira https://redhat.atlassian.net/browse/OCPBUGS-86695?focusedCommentId=17167775 +``` + +**Output**: OCP-xxxxx URL + +### /mco-update-tc *(future scope — MCO-2220)* + +> **TODO**: Standalone update command not yet active. Use `update_polarion_tc.py` directly in the meantime (called automatically by the create workflow). +> +> ```bash +> python3 scripts/update_polarion_tc.py OCP-88941 --fix-metadata --version 4.18 --trello-jira OCPBUGS-83830 +> python3 scripts/update_polarion_tc.py OCP-88122 --title "[MCO][MCO-2136] New title" +> python3 scripts/update_polarion_tc.py OCP-88122 --steps-file /tmp/test-specs/steps/new-steps.json +> ``` + +### /automate-test + +Generate Go e2e test from Polarion test case. + +```bash +# Fetch TC and generate test +/automate-test OCP-88122 "mco osstream" disruptive +``` + +**Workflow**: +1. Fetches OCP-88122 from Polarion → `/tmp/test-specs/OCP-88122.txt` +2. Reads test steps from .txt file +3. Generates `test/extended-priv/mco_osstream.go` +4. Builds with `make machine-config-tests-ext` + +--- + +## Workflows + +### Workflow 1: Create TC from GitHub PR + +**Use when**: PR has QE "Pre-merge testing" comment with test steps. + +```bash +# 1. Fetch PR comment +python3 scripts/fetch_github_pr.py \ + --repo openshift/machine-config-operator \ + --pr 5691 \ + --comment 4054832254 + +# 2. Validate +python3 scripts/validate_tc.py /tmp/test-specs/drafts/pr-5691.json + +# 3. Review draft +cat /tmp/test-specs/drafts/pr-5691.json | jq . + +# 4. Create (dry run first) +python3 scripts/create_polarion_tc.py \ + /tmp/test-specs/drafts/pr-5691.json \ + --dry-run + +# 5. Create for real +python3 scripts/create_polarion_tc.py \ + /tmp/test-specs/drafts/pr-5691.json +``` + +**Result**: OCP-xxxxx created with title, metadata (6 fields), and test steps. + +### Workflow 2: Create TC from Jira + +**Use when**: Jira issue has acceptance criteria, no PR test steps yet. + +```bash +# 1. Fetch Jira +python3 scripts/fetch_jira.py OCPBUGS-74223 + +# 2. Validate (will warn about empty test_steps) +python3 scripts/validate_tc.py /tmp/test-specs/jira/OCPBUGS-74223.json + +# 3. Create TC with metadata only +python3 scripts/create_polarion_tc.py \ + /tmp/test-specs/jira/OCPBUGS-74223.json +``` + +**Result**: TC created with metadata from Jira, empty test steps (add later manually or via update). + +**Jira fields extracted**: +- Summary → title (with `[MCO][JIRA-ID]` prefix) +- Fix Versions → version (extracts `4.18` from `4.18.0`) +- Description + Acceptance Criteria → description +- Issue Type, Labels → metadata + +### Workflow 3: Hybrid (Jira + GitHub PR) - **RECOMMENDED** + +**Use when**: Jira has metadata, GitHub PR has test steps. Best of both worlds. + +```bash +# One command - fetches both, merges, validates, creates +python3 scripts/create_tc_hybrid.py OCPBUGS-74223 \ + --from-pr 5691 \ + --comment 4054832254 +``` + +**What it does**: +1. Fetches Jira OCPBUGS-74223 → version, description, acceptance criteria +2. Fetches GitHub PR 5691 comment → test steps with commands +3. Merges: Jira metadata + PR steps +4. Validates merged draft +5. Shows summary and confirmation prompt +6. Creates TC in Polarion + +**Result**: Complete TC with accurate metadata and executable test steps. + +**Merge logic**: +- Title: `[MCO][JIRA-ID] ` +- Version: From Jira fix versions +- Description: Jira description + acceptance criteria +- Test steps: From GitHub PR QE comment +- Jira ID: From Jira issue key + +### Workflow 4: Fix Existing TC (wrong.pdf scenario) + +**Use when**: TC has empty required fields (Component, Sub Team, etc.). + +See `wrong.pdf` in repo root for bad example vs `OCP-88122.pdf` for good format. + +```bash +# Fix metadata +python3 scripts/update_polarion_tc.py OCP-88941 \ + --fix-metadata \ + --version 4.18 \ + --trello-jira OCPBUGS-83830 + +# Verify changes +python3 scripts/fetch_polarion.py OCP-88941 +``` + +**`--fix-metadata` sets defaults**: +- Component: Machine Config Operator +- Sub Team: MCO +- Products: OCP +- Test Type: Functional +- Version: (from --version flag) +- Trello/Jira: (from --trello-jira flag) + +### Workflow 5: Polarion TC → Go Test + +**Use when**: Polarion TC exists, need to generate automated Go test. + +```bash +# 1. Fetch TC from Polarion +python3 scripts/fetch_polarion.py OCP-88122 +# Creates /tmp/test-specs/OCP-88122.txt + +# 2. Generate Go test (via Claude Code) +/automate-test OCP-88122 "mco osstream" disruptive + +# 3. Build +make machine-config-tests-ext + +# 4. Verify test appears +./_output/linux/amd64/machine-config-tests-ext list | grep OCP-88122 +``` + +**Result**: `test/extended-priv/mco_osstream.go` with test implementation. + +--- + +## TC Format (Good vs Bad) + +### Good Format: OCP-88122.pdf + +See `OCP-88122.pdf` in repo root. + +**Characteristics**: +- ✓ Title: `[MCO][MCO-2136] Validate osImageStream inheritance...` +- ✓ Component: Machine Config Operator +- ✓ Sub Team: MCO +- ✓ Products: OCP +- ✓ Test Type: Functional +- ✓ Version: 4.22 +- ✓ Trello/Jira: MCO-2136 +- ✓ Test Steps: Commands in Expected Result column (not narrative) + +**Example step**: +``` +Step: Check the osstream +Expected Result: +$ oc get mcp rhel9 -ojsonpath='{.status.osImageStream.name}' +rhel9 +``` + +### Bad Format: wrong.pdf (OCP-88941) + +See `wrong.pdf` in repo root. + +**Problems**: +- ❌ Component: (empty) +- ❌ Sub Team: (empty) +- ❌ Products: (empty) +- ❌ Test Type: (empty) +- ❌ Version: (empty) +- ❌ Trello/Jira: (empty) +- ❌ Test Steps: Narrative only, no commands + +**How this happened**: Created without validation, or metadata not set during creation. + +**How to fix**: Use `update_polarion_tc.py --fix-metadata` + +### Validation Rules + +Scripts enforce these rules: + +1. **Title format**: Must match `^\[MCO\]\[.+\] .+` + - Good: `[MCO][OCPBUGS-74223] Add feature X` + - Bad: `Add feature X` (missing prefix) + +2. **Required fields** (all 6 must be non-empty): + - component + - sub_team + - products + - test_type + - version + - trello_jira + +3. **Test steps quality**: + - Each step must have non-empty `step` field + - Expected results should have commands (not narrative-only) + - Warning if expected result has no `oc`, `kubectl`, code syntax + +Run validation before creating: +```bash +python3 scripts/validate_tc.py /tmp/test-specs/drafts/pr-5691.json +``` + +--- + +## Scripts Reference + +| Script | Purpose | Input | Output | +|---|---|---|---| +| `setup.sh` | Environment setup + verification | - | Status messages | +| `validate_tc.py` | Validate draft JSON | Draft JSON path or `-` (stdin) | Validation result | +| `fetch_polarion.py` | Fetch Polarion TC | OCP-88122 | JSON (stdout) + `/tmp/test-specs/OCP-88122.txt` | +| `fetch_github_pr.py` | Parse GitHub PR QE comment | `--repo org/repo --pr NUM --comment ID` | `/tmp/test-specs/drafts/pr-NUM.json` | +| `fetch_jira.py` | Fetch Jira issue | OCPBUGS-12345 | `/tmp/test-specs/jira/OCPBUGS-12345.json` | +| `create_polarion_tc.py` | Create TC from draft | Draft JSON path | OCP-xxxxx URL | +| `create_tc_hybrid.py` | Merge Jira + PR, create TC | `JIRA-KEY --from-pr NUM` | OCP-xxxxx URL | +| `update_polarion_tc.py` | Update existing TC | OCP-xxxxx + flags | Updated field count | +| `polarion_client.py` | REST/SOAP API client | (library) | - | + +### Common Flags + +**validate_tc.py**: +- `--strict`: Fail on warnings +- `--sample`: Show validation examples + +**create_polarion_tc.py**: +- `--dry-run`: Show summary, don't create +- `--project OSE`: Override project (default: OSE) + +**update_polarion_tc.py**: +- `--fix-metadata`: Set all required fields to defaults +- `--title TEXT`: Update title +- `--status STATUS`: Update status (draft, approved, etc.) +- `--version VERSION`: Update version +- `--trello-jira KEY`: Update Jira ID +- `--steps-file PATH`: Update test steps (requires SOAP API) +- `--dry-run`: Preview changes + +**fetch_github_pr.py**: +- `--repo ORG/REPO`: Repository (required) +- `--pr NUM`: PR number (required) +- `--comment ID`: Comment ID (optional - finds QE comment if omitted) +- `--output PATH`: Custom output path + +**create_tc_hybrid.py**: +- `--from-pr NUM`: GitHub PR number (required) +- `--comment ID`: Comment ID (optional) +- `--repo ORG/REPO`: Repository (default: openshift/machine-config-operator) +- `--dry-run`: Preview without creating + +--- + +## Troubleshooting + +### "POLARION_TOKEN not configured" + +**Cause**: Token missing or placeholder value in `.env` + +**Fix**: +1. Generate token: https://polarion.engineering.redhat.com/polarion/#/user/me +2. Add to `.env` at repo root +3. Verify: `./scripts/setup.sh --verify` + +### "Validation failed: Required field 'version' is empty" + +**Cause**: Draft missing required field + +**Fix**: +```bash +# Edit draft manually +vim /tmp/test-specs/drafts/pr-5691.json + +# Or set during creation +python3 scripts/create_tc_hybrid.py JIRA-KEY --from-pr NUM +# (hybrid mode auto-fills from Jira) +``` + +### "SOAP API requires username and password" + +**Cause**: Trying to update test steps on existing TC without SOAP credentials + +**Context**: Polarion REST API can only POST test steps to NEW TCs. For existing TCs, must use SOAP API. + +**Fix**: +```bash +# Add to .env +POLARION_USERNAME=your_username +POLARION_PASSWORD=your_password + +# Then retry +python3 scripts/update_polarion_tc.py OCP-88122 \ + --steps-file /tmp/test-specs/steps/new-steps.json +``` + +### "GitHub API error: 401 Unauthorized" + +**Cause**: Invalid or missing GitHub token + +**Fix**: +- For public repos: Script auto-retries without token +- For private repos or to avoid rate limits: Add GITHUB_TOKEN to `.env` +- Generate token: https://github.com/settings/tokens (public_repo scope) + +### "Jira authentication failed" + +**Cause**: Invalid token or missing email + +**Fix**: +```bash +# Jira (Atlassian Cloud) requires email + token +JIRA_EMAIL=you@redhat.com +JIRA_TOKEN=your_token +``` + +Generate token: https://id.atlassian.com/manage-profile/security/api-tokens + +### Test steps narrative-only (no commands) + +**Symptom**: Validation warns "Expected result appears to be narrative-only" + +**Fix**: Edit draft to add commands: +```json +{ + "step": "Verify feature applied", + "expected_result": "$ oc get mc | grep feature\nfeature 3.2.0 15s" +} +``` + +--- + +## Claude Code Examples + +If using Claude Code assistant: + +### Fetch and create from PR +``` +/mco-create-tc-from-pr https://github.com/openshift/machine-config-operator/pull/5691#issuecomment-4054832254 +``` + +### Create TC from Jira +``` +/mco-create-tc-from-jira https://redhat.atlassian.net/browse/OCPBUGS-74223 +``` + +### Fix bad TC *(direct script — /mco-update-tc is future scope)* +```bash +python3 .claude/scripts/update_polarion_tc.py OCP-88941 --fix-metadata --version 4.18 --trello-jira OCPBUGS-83830 +``` + +### Fetch TC and generate test +``` +/mco-fetch tc OCP-88122 +/automate-test OCP-88122 "mco osstream" disruptive +``` + +### Migrate test from OTP3 +``` +/migrate-tests mco_ocb.go --dry-run +``` + +--- + +## Output Locations + +- **Drafts**: `/tmp/test-specs/drafts/pr-*.json` +- **Jira**: `/tmp/test-specs/jira/*.json` +- **Polarion TCs** (txt format): `/tmp/test-specs/OCP-*.txt` +- **Test steps**: `/tmp/test-specs/steps/*.json` + +All output goes to `/tmp/` (not in repo). + +--- + +## Architecture Notes + +### REST vs SOAP API + +**REST API** (Bearer token): +- Fetch test cases +- Create new test cases +- Update fields (title, description, status, custom fields) +- **Limitation**: Can only POST test steps to NEW TCs + +**SOAP API** (username/password): +- Update test steps on EXISTING TCs +- Script auto-falls back from REST to SOAP when updating steps + +**Why both?** Polarion REST API doesn't support updating test steps on existing TCs. This is a known Polarion limitation. + +### Test Step Format for SOAP API + +When updating test steps via SOAP, use this JSON format: +```json +[ + { + "step": "Do something", + "expected_result": "$ oc get nodes\nNAME STATUS\nnode1 Ready" + } +] +``` + +### Jira Integration (No MCP) + +Uses direct Atlassian REST API v3. Authentication is Basic Auth with email:token (not Red Hat SSO). + +**Why email + token?** Atlassian Cloud requires this authentication method. + +--- + +## Pilot Test + +Want to verify the workflow end-to-end? See pilot test checklist in appendix: + +```bash +# Run full pilot (PR 5691 → Create TC → Verify) +# See PILOT_PR5691.md for step-by-step checklist +``` + +Or test fetch verification: + +```bash +# Verify OCP-88122 fetch matches PDF +# See TEST_FETCH.md for verification checklist +``` + +--- + +## Documentation Files + +- **README.md** (this file) - Complete guide +- **TESTING.md** - Test plan for reviewers +- **PILOT_PR5691.md** - Pilot test checklist (PR 5691 → TC creation) +- **TEST_FETCH.md** - Fetch verification (OCP-88122) + +--- + +## Help & Support + +**For script help**: +```bash +python3 scripts/