Skip to content

develop action: update test plan checklist in PR description after PR creation #88

Description

@monkut

Problem

When the develop action implements an issue and opens a PR, the PR description includes a ## Test plan checklist (generated by Claude Code's standard PR template). However, the develop action does not check off any of those items — even when its own verification gate (tests, lint, type checker) and the new tests it wrote already satisfy them.

The pr-review action already updates the test plan (see askcc/definitions.py:496-503), but by then the PR has been sitting with an entirely unchecked test plan, which misrepresents the actual state of verification at PR creation time.

Expected behaviour

After develop opens the PR, it should:

  • Read the PR description with gh pr view <number> --json body -q .body
  • Find the ## Test plan section (if present)
  • For each - [ ] item, decide if it is already satisfied by the implementation, new tests, or the verification gate run
  • Replace - [ ] with - [x] for satisfied items
  • Leave items requiring manual/external verification (browser QA, deployment validation, etc.) unchecked
  • Update the PR body with gh pr edit <number> --body "<updated body>"
  • Skip silently if no ## Test plan section exists

Actual behaviour

develop opens the PR with the test plan items left as - [ ] regardless of what was already verified locally.

Suggested fix

Mirror the test-plan-update logic already present in REVIEWPR_AGENT_PROMPT (askcc/definitions.py:496-503), but for DEVELOP_AGENT_PROMPT:

  1. Modify the "On completion" block to add a "Update the test plan checklist in the PR description" step between PR creation and the issue comment.
  2. Append a new "Test plan update" section explaining the read → mark → edit flow, with the same conservative rule: only check off items genuinely satisfied by what was just done.

Add tests in tests/test_askcc.py mirroring TestReviewprPromptMergeGuard to assert the new strings exist in DEVELOP_AGENT_PROMPT.

Acceptance Criteria

  • DEVELOP_AGENT_PROMPT instructs the agent to update test plan items in the PR description after PR creation
  • Instruction explicitly says to leave items requiring manual/external verification unchecked
  • Instruction is a no-op when no ## Test plan section exists
  • Unit tests assert the new instruction strings are present in DEVELOP_AGENT_PROMPT
  • uv run pytest, uv run ruff check, and uv run pyright all pass

Dependencies

None identified. Self-contained prompt + test change in askcc/definitions.py and tests/test_askcc.py. Mirrors the pattern established by issue #86 / PR #87 which added the CHANGES_REQUESTED guard to REVIEWPR_AGENT_PROMPT with TestReviewprPromptMergeGuard.

Implementation Plan

1. Update the "On completion" step list in DEVELOP_AGENT_PROMPT

File: askcc/definitions.py, lines 281–285.

Insert a new bullet between the PR-push step and the issue-comment step so the order becomes:

  1. Run /simplify or /refactor.
  2. Commit, push the feature branch, and open a PR linked to the issue.
  3. Update the test plan checklist in the PR description (see "Test plan update" below).
  4. Add an issue comment summarizing what was implemented.

This creates a forward reference to the detailed section added in step 2.

2. Append a "Test plan update" section to DEVELOP_AGENT_PROMPT

File: askcc/definitions.py, immediately after the existing "On completion" block (after line 285).

Mirror the structure used in REVIEWPR_AGENT_PROMPT (lines 496–503), adjusted to the develop context:

Test plan update:
- After opening the PR, read the PR description with `gh pr view <number> --json body -q .body`.
- Look for a `## Test plan` section containing checklist items (`- [ ]` checkboxes).
- For each test plan task, decide whether it is satisfied by the implementation,
  the new tests you wrote, or the verification gate run (pytest/ruff/pyright).
- Check off satisfied tasks by replacing `- [ ]` with `- [x]` in the PR body.
- Leave items requiring manual/external verification (browser QA, deployment
  validation, etc.) unchecked.
- Update the PR description with `gh pr edit <number> --body "<updated body>"`.
- If no `## Test plan` section exists, skip this step silently.

The wording must contain the literal substrings the tests will assert on (see step 3): the heading Test plan update, the no-op clause If no \## Test plan` section exists, skip this step, and the manual-verification clause Leave items requiring manual/external verification ... unchecked`.

3. Add a TestDevelopPromptTestPlanUpdate test class in tests/test_askcc.py

File: tests/test_askcc.py, immediately after TestDevelopPromptTddContent (line 251–279) and before TestReviewprPromptMergeGuard (line 282), to keep develop-related tests grouped.

Mirror TestReviewprPromptMergeGuard (lines 282–298) with develop-specific assertions:

class TestDevelopPromptTestPlanUpdate:
    """The develop prompt must update the PR test plan after PR creation (issue #88)."""

    def test_includes_test_plan_update_section_heading(self):
        assert "Test plan update:" in DEVELOP_AGENT_PROMPT

    def test_includes_pr_body_read_command(self):
        assert "gh pr view <number> --json body -q .body" in DEVELOP_AGENT_PROMPT

    def test_includes_pr_body_edit_command(self):
        assert 'gh pr edit <number> --body "<updated body>"' in DEVELOP_AGENT_PROMPT

    def test_includes_manual_verification_carveout(self):
        assert "manual/external verification" in DEVELOP_AGENT_PROMPT
        assert "unchecked" in DEVELOP_AGENT_PROMPT

    def test_includes_noop_when_no_test_plan_section(self):
        assert "If no `## Test plan` section exists, skip this step" in DEVELOP_AGENT_PROMPT

    def test_completion_step_references_test_plan_update(self):
        assert "Update the test plan checklist in the PR description" in DEVELOP_AGENT_PROMPT

No new imports required — DEVELOP_AGENT_PROMPT is already imported (line 15).

4. Run the verification gate

uv run pytest tests/test_askcc.py -k "TestDevelopPromptTestPlanUpdate"  # new tests pass
uv run pytest                                                           # full suite green
uv run ruff check
uv run pyright

5. Open PR

Branch: feature/88-develop-update-pr-test-plan (or similar — develop action picks the slug). PR title should reference issue #88. Standard ## Verification and ## Key Flows sections per DEVELOP_AGENT_PROMPT requirements; the change is small enough that the Key Flows diagram can be skipped (single-string-edit + new test class).

Risks / open questions

  • String drift risk: tests assert on literal substrings. If a future edit rewords the prompt, tests will fail loudly — that is the intended behaviour (per the TestReviewprPromptMergeGuard precedent), but document it in the test class docstring so reviewers don't "fix" the assertions away.
  • Step ordering: the new test plan update step runs after PR creation but before the issue comment, so the issue comment can also confirm the test plan was updated. The plan reflects this ordering.
  • No behavioural code change: this is a prompt-only change — no Python logic in askcc/functions.py is affected, and no new CLI flags are introduced.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions