Skip to content

feat: warn on active bounty attempts#385

Merged
ramimbo merged 2 commits into
ramimbo:mainfrom
tolga-tom-nook:bounty/384-active-attempt-warnings
May 26, 2026
Merged

feat: warn on active bounty attempts#385
ramimbo merged 2 commits into
ramimbo:mainfrom
tolga-tom-nook:bounty/384-active-attempt-warnings

Conversation

@tolga-tom-nook
Copy link
Copy Markdown
Contributor

@tolga-tom-nook tolga-tom-nook commented May 26, 2026

Summary

Add read-only active-attempt warnings to the submission quality gate.

Refs #384

Changes

  • Load internal bounty ids from the MergeWork bounties API and fetch /api/v1/bounties/{id}/attempts for the referenced bounty in live mode.
  • Add advisory active-attempt checks to JSON/text output, including safe submitter, source URL, status, and expiry fields.
  • Preserve existing payability, stale/closed, maintainer-activity, similar-PR, and fixture-mode checks; attempts API failures degrade to warnings.
  • Update the agent guide with the new preflight behavior.

Validation

  • ./.venv/bin/python -m pytest tests/test_submission_quality_gate.py -q — 22 passed
  • ./.venv/bin/python scripts/docs_smoke.py — docs smoke ok
  • ./.venv/bin/python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py — passed
  • ./.venv/bin/python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py — passed
  • ./.venv/bin/python -m mypy app — passed
  • git diff --check — passed

Summary by CodeRabbit

  • New Features

    • Submission quality gate now checks and reports on active attempt reservations for referenced bounties, warning when attempts are reserved or unverifiable.
  • Documentation

    • Expanded Submission Quality Gate docs: added detailed validation steps (bounty existence, recent maintainer activity, active attempt lookups, concise summary/evidence checks, multi-bounty mixing prevention, similar open-PR detection) and clarified advisory behavior when live API data is unavailable.
  • Tests

    • Added and updated tests covering active-attempt detection, verification, and API-unavailability warning behavior.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 26, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 9a48cd5b-bf40-4168-b04a-884b410333ae

📥 Commits

Reviewing files that changed from the base of the PR and between 6a6f599 and 4d31e99.

📒 Files selected for processing (2)
  • scripts/submission_quality_gate.py
  • tests/test_submission_quality_gate.py

📝 Walkthrough

Walkthrough

The submission quality gate now verifies active attempt reservations by fetching attempts from the MergeWork API, emitting pass/warn checks, enriching live context with attempts and verification flags, handling API failures with warnings, and showing per-attempt summaries in output; docs updated.

Changes

Active Attempts Quality Gate

Layer / File(s) Summary
Core active attempts validation logic
scripts/submission_quality_gate.py
New helper functions extract, normalize, and format active attempt data. evaluate_submission emits an active_attempts check (pass/warn) and the evaluation payload includes active_attempts. HTTPError imported for API error handling.
API integration and attempts normalization
scripts/submission_quality_gate.py
Adds _load_api_attempts to fetch /attempts, normalizes attempt records, validates API responses, and preserves bounty id for attempt lookups; failures return empty attempts and mark verification false.
Live-context enrichment and warnings
scripts/submission_quality_gate.py
During live-context loading, uses bounty id to fetch attempts, attaches active_attempts and active_attempts_verified to bounty objects, and records load warnings when the id is missing or the attempts API fails.
Output formatting and documentation
scripts/submission_quality_gate.py, docs/agent-guide.md
format_text appends an "Active attempts" section listing formatted attempt summaries when present. docs/agent-guide.md expands the Submission Quality Gate doc to enumerate attempt lookups, validation checks, single-bounty constraint, maintainer activity, duplicate PR detection, and advisory behavior when the attempts API is unavailable.
Test coverage for active attempts
tests/test_submission_quality_gate.py
Adds unit tests asserting pass/warn behavior for no attempts, one attempt, multiple attempts, and unverifiable attempts; adds a live-context test for missing attempt lookup id and updates existing live-context fixtures to include bounty id and empty attempts.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Possibly related issues

Possibly related PRs

  • ramimbo/mergework#335: Earlier extension of the quality gate that added maintainer activity checks and similar live-context enrichment patterns.

Suggested labels

mrwk:accepted, mrwk:paid

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat: warn on active bounty attempts' clearly and concisely describes the main change—adding active bounty attempt warnings to the submission quality gate.
Description check ✅ Passed The description includes a clear summary, detailed changes, test evidence validating all required checks, and proper MRWK reference (#384), meeting template requirements.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@carpedkm carpedkm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR #385 at head 6a6f599 for Bounty #384 / review round #340.

Files inspected: scripts/submission_quality_gate.py, tests/test_submission_quality_gate.py, and docs/agent-guide.md.

I checked that the live path now carries the internal bounty id from the bounties API into a read-only /api/v1/bounties/{id}/attempts lookup, normalizes only safe public attempt fields, degrades attempts API failures into advisory warnings, and keeps the existing payability, maintainer-activity, evidence, multi-reference, and similar-open-PR checks intact. The public endpoint behavior matches the implementation assumption: /api/v1/bounties/59/attempts currently returns an attempts envelope, and running the gate against this PR body reports pass with no active attempts and no similar open PR warning.

Validation run locally:

  • ./.venv/bin/python -m pytest tests/test_submission_quality_gate.py -q -> 21 passed
  • ./.venv/bin/python -m pytest -q -> 357 passed
  • ./.venv/bin/python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • ./.venv/bin/python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • ./.venv/bin/python -m mypy app -> success
  • ./.venv/bin/python scripts/docs_smoke.py -> docs smoke ok
  • git diff --check origin/main...HEAD -> clean

No blockers found. One non-blocking note: I did not treat ruff format --check docs/agent-guide.md as applicable because ruff requires preview mode for Markdown; ruff check docs/agent-guide.md reports no Python files, as expected.

Strict accounting: review submitted only; no payment or acceptance counted until maintainer proof appears.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/submission_quality_gate.py`:
- Around line 527-537: When api_bounty.get("id") is missing or not an int,
currently the code skips setting active_attempts and active_attempts_verified
which hides unverified state; update the branch that handles non-integer
bounty_id so that you explicitly set bounties[-1]["active_attempts"] = [],
bounties[-1]["active_attempts_verified"] = False and append a descriptive
warning to load_warnings (similar to the except block) mentioning
issue['number'] and that the id was missing/invalid, instead of doing nothing;
this mirrors the behavior in the RuntimeError handler and ensures downstream
checks see the attempts as unverified (refer to bounty_id, api_bounty, bounties,
active_attempts, active_attempts_verified, _load_api_attempts, load_warnings,
issue).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 32085230-4098-446f-bf5c-1410e70114e5

📥 Commits

Reviewing files that changed from the base of the PR and between 04153ee and 6a6f599.

📒 Files selected for processing (3)
  • docs/agent-guide.md
  • scripts/submission_quality_gate.py
  • tests/test_submission_quality_gate.py

Comment thread scripts/submission_quality_gate.py
Copy link
Copy Markdown

@aiautotool aiautotool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the active-attempt warning implementation against bounty #384. No blockers found.

Evidence checked:

  • scripts/submission_quality_gate.py keeps the attempts lookup read-only via GET /api/v1/bounties/{id}/attempts and catches API/JSON/URL failures as advisory warnings instead of failing the gate.
  • Existing payability and maintainer-activity checks remain in the same evaluation path before summary/evidence checks, so closed/exhausted bounty behavior is preserved.
  • JSON/text output includes only safe attempt fields: submitter, source URL, status, and expiry.
  • Fixture coverage includes no attempts, one attempt, multiple attempts, unavailable attempts API, and closed/exhausted behavior.

Local validation from a clean PR worktree:

  • /Users/vkct/Documents/Codex/2026-05-26/s-d-ng-ki-n-th/mergework/.venv/bin/python -m pytest tests/test_submission_quality_gate.py -q -> 21 passed
  • /Users/vkct/Documents/Codex/2026-05-26/s-d-ng-ki-n-th/mergework/.venv/bin/python scripts/docs_smoke.py -> docs smoke ok
  • /Users/vkct/Documents/Codex/2026-05-26/s-d-ng-ki-n-th/mergework/.venv/bin/python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • /Users/vkct/Documents/Codex/2026-05-26/s-d-ng-ki-n-th/mergework/.venv/bin/python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • git diff --check origin/main...HEAD -> passed

@tolga-tom-nook
Copy link
Copy Markdown
Contributor Author

Follow-up for CodeRabbit discussion r3301745939: pushed 4d31e99 to mark missing/non-integer API bounty ids as unverified active-attempt state instead of silently omitting the check.

Validation after the fix:

  • ./.venv/bin/python -m pytest tests/test_submission_quality_gate.py -q -> 22 passed
  • ./.venv/bin/python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • ./.venv/bin/python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> passed
  • git diff --check -> passed
  • GitHub Actions on head 4d31e991fd7091d6cbca4eaab94fd4426745055c -> success

@ramimbo ramimbo merged commit ef329db into ramimbo:main May 26, 2026
2 checks passed
@ramimbo ramimbo added mrwk:accepted Maintainer accepted for payout mrwk:paid Ledger payment recorded labels May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mrwk:accepted Maintainer accepted for payout mrwk:paid Ledger payment recorded

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants