auto-backport: enforce a Fixes reference when a backport label is added

[yaronkaikov]

RELENG-175

Force a PR to link a valid, existing issue as early as a backport label is added, instead of only after merge — so the work is visible sooner and gets release-note coverage.

Behaviour

When a backport/<release> (or backport/manager-X.Y) label is added to a PR whose body does not link a valid, existing issue, the automation:

• comments once, mentioning the author and assignee(s), with the agreed message:

  PR body do not contain a Fixes reference to an issue and can not be backported
  please update PR body with a valid ref to an issue and add the backport label again.

• removes all backport labels, so the PR can't be backported until its body is fixed and the label re-added.

backport/none is exempt. The comment is posted only once even when several backport labels are added together.

What "valid" means

Reuses the existing extract_jira_keys() for closing-keyword detection (Fixes:/Closes:/Resolves:...) and project-key validation (hard-coded set first, live Jira project list for unknown prefixes), then additionally confirms the referenced issue exists (Jira GET, 404 ⇒ reject). So:

• Fixes: AAA-123 (unknown project) → rejected
• Fixes: SCYLLADB-9898758758 (real project, non-existent number) → rejected
• Fixes: SCYLLADB-123 (exists) → allowed

If existence can't be confirmed (Jira unreachable), it fails open so a valid PR isn't penalised for a transient outage. A GitHub-only ref (Fixes: #123) does not satisfy it — the link must be to a Jira issue.

Implementation

Hooks into the existing labeled-event path (jira_sync_logic.py, run by main_pr_events_jira_sync.yml), where backport/ labels were already excluded from Jira sync — no new workflow or entrypoint.

• scripts/jira_sync_modules.py — enforce_backport_fixes_reference() + helpers (_pr_body_links_existing_issue, _jira_issue_exists, _build_pr_mentions, …), reusing _gh_api + extract_jira_keys.
• scripts/jira_sync_logic.py — enforce on backport/<release> in manage_labeled_gh_event (+14 lines).
• .github/tests/test_enforce_backport_fixes.py — 18 tests.

Testing

373 passed (full suite, includes the 18 new tests).

Fixes: RELENG-175

[dani-tweig]

Why are you using has_fixes_reference() instead of extract_jira_keys() in https://github.com/scylladb/github-automation/blob/main/scripts/jira_sync_modules.py

Your code will fail in many cases that the extract_jira_keys function handles today.

[yaronkaikov]

Squashed into a single commit and addressed the review feedback.

What this PR does (RELENG-175)

When a backport/X.Y (or backport/manager-X.Y) label is added to an open PR whose body has no valid issue reference, the automation comments (mentioning the author + assignee(s)) and removes all backport labels, so the PR can't be backported until its body is fixed. backport/none is exempt, and only one comment is posted even when several backport labels are added at once (the debounce/concurrency guards now run for open PRs too, so only the latest run enforces). The closed/merge backport path is unchanged.

Change since your review (@dani-tweig)

You were right that has_fixes_reference() would fail on cases extract_jira_keys() already handles. Rather than duplicate the regex, I extracted the canonical closing-keyword pattern into a new scripts/closing_keywords.py, and now both scripts/jira_sync_modules.py (extract_jira_keys) and the backport enforcement import it — single source of truth, so they can't drift again.

This fixes the false negatives you flagged. These are now all recognized:

• other keywords: Closes: / Resolves: / Fix: / Fixed:
• all-caps keyword: FIXES: RELENG-1
• space separator: Closes RELENG-1
• lowercase key: fixes: releng-1

GitHub issue/PR references (Fixes #123, Closes owner/repo#123) stay supported, since those PRs should still be backportable.

A couple of notes:

• I couldn't import jira_sync_modules directly into the backport script — it uses 3.10+ syntax and the backport unit tests + pre-commit hook run on 3.9 — so the shared module is pure-stdlib and 3.9-safe.
• The extracted CLOSING_KEYWORD_RE is byte-identical to the previous jira-sync regex, so extract_jira_keys behavior is unchanged (verified _CLOSING_KEYWORD_RE is closing_keywords.CLOSING_KEYWORD_RE).
• Out of scope here: the Jira-key extractors used for sub-issue creation (extract_jira_key_from_pr_body / extract_all_jira_keys_from_pr_body) still use the narrower pattern — happy to consolidate those too if you'd prefer them in this PR.

Tests

Added test_closing_keywords.py, extended the has_fixes_reference cases for the previously-failing forms, plus the enforcement tests. Full suite green on 3.9 (local/pre-commit) and 3.12 (CI).

@dani-tweig could you take another look? Thanks!

[dani-tweig]

Using an extended regex is good, but extract_jira_keys() also validates the Jira project key.
How will your code handle 'Fixes: AAA-123' (non-existing project key) or 'Fixes: SCYLLADB-9898758758'? (non-existing number in an existing project)
The expected result would be to fail the operation.

Again, why not use the existing function?

[yaronkaikov]

v3 — simplified, and using extract_jira_keys() directly

@dani-tweig you were right on both counts. I've dropped the whole custom-regex / has_fixes_reference() / closing_keywords.py apparatus from v2. The enforcement now just calls extract_jira_keys() — the existing function — so closing-keyword detection and Jira project-key validation are identical to the rest of the tooling and can't drift.

On your second point (a real project with a non-existent number, e.g. SCYLLADB-9898758758): extract_jira_keys() validates the project prefix but not the issue number, so on top of it I now confirm the issue actually exists (a Jira GET, 404 ⇒ reject). All three of your cases now resolve the way you expected:

PR body	result
Fixes: AAA-123 (unknown project)	❌ rejected
Fixes: SCYLLADB-9898758758 (real project, bad number)	❌ rejected
Fixes: SCYLLADB-123 (exists)	✅ allowed

If existence can't be confirmed (Jira unreachable), it fails open so a valid PR isn't blocked by a transient outage.

How simple it ended up

The feature is now one commit, +415 (most of it tests), with no new workflow and no new entrypoint. The labeled event already flows through main_pr_events_jira_sync.yml → jira_sync_logic.py → manage_labeled_gh_event, which already early-exited on backport/ labels — that early-exit is the only hook point needed. The enforcement lives next to extract_jira_keys/_gh_api in jira_sync_modules.py, plus a 14-line branch in the orchestrator.

373 passed (18 new enforcement tests). Could you take another look? Thanks! 🙏