Skip to content

testbot: require go_test BUILD targets for new Go test files#959

Open
jiaenren wants to merge 2 commits intomainfrom
jiaenr/testbot-go-test-target
Open

testbot: require go_test BUILD targets for new Go test files#959
jiaenren wants to merge 2 commits intomainfrom
jiaenr/testbot-go-test-target

Conversation

@jiaenren
Copy link
Copy Markdown
Collaborator

@jiaenren jiaenren commented May 5, 2026

Summary

PR #956 (testbot-generated) added src/runtime/pkg/common/common_test.go but didn't add a corresponding go_test rule to the package's BUILD file. The test file is invisible to Bazel:

$ bazel test //src/runtime/pkg/common:all
ERROR: No test targets were found, yet testing was requested
$ bazel query 'kind(go_test, //src/runtime/pkg/common/...)'
INFO: Empty results

Net effect: a testbot run that picked common.go because of "0% coverage" closed the gap with a PR that ships zero new measured coveragebazel coverage //... won't run a test target it doesn't know about, and Codecov never sees the result.

This PR teaches the testbot to wire up Go tests properly going forward:

  • TESTBOT_PROMPT.md step 6: generalized from "Python only" to cover Go. Includes the canonical pattern (go_library + go_test with embed = [":<pkg>"]) verified against src/runtime/pkg/data/BUILD, and a "check it worked" instruction — re-run bazel test and watch for No test targets were found.
  • TESTBOT_RULES.md verification step: explicitly call out the "No test targets were found" failure mode and point at BUILD wiring. New "BUILD wiring (Python + Go)" subsection explains why a go_library-only BUILD silently drops adjacent _test.go files.
  • TESTBOT_PROMPT.md guardrails: include go_test alongside py_test in the allowed BUILD edits.

Documentation-only change to the prompt files. No code paths or tests change.

Issue - None

Files tested

  • N/A — this updates the agent's instructions, not source code.

Checklist

  • I am familiar with the Contributing Guidelines
  • New or existing tests cover these changes (14/14 testbot bazel tests still pass)
  • The documentation is up to date with these changes (this is the documentation)

Test plan

  • bazel test //src/scripts/testbot/tests/... (still green)
  • Next testbot generate run picks a Go target and produces a PR with both _test.go AND a go_test BUILD entry, then bazel coverage //... actually shows non-zero coverage on the targeted file.

Follow-up

PR #956 itself still needs the go_test rule appended to src/runtime/pkg/common/BUILD to take effect. I'd suggest either pushing the fix to that branch directly or asking testbot to follow up via /testbot add a go_test rule to BUILD for common_test.go.

Summary by CodeRabbit

  • Documentation
    • Expanded test-generation guidance with detailed BUILD file handling and Bazel wiring instructions for Python and Go, including verification steps and a Bazel-specific test failure path.
    • Clarified that test-generation changes are restricted to test files and BUILD entries (py_test/go_test), with no modifications to source code or configuration files.

PR #956 (testbot-generated) added src/runtime/pkg/common/common_test.go
without adding a corresponding go_test rule to the package's BUILD file.
The file ended up invisible to Bazel — `bazel test //src/runtime/pkg/common:all`
returns "No test targets were found", and `bazel coverage //...` (the
job that feeds Codecov) skips it entirely. Net effect: the testbot run
that picked common.go because of "0% coverage" closes the issue with a
PR that ships zero new coverage.

Update both prompts so the next run gets it right:

- TESTBOT_PROMPT.md step 6: generalized from "Python only — BUILD file"
  to cover Go too. Includes the canonical pattern (go_library + go_test
  with embed = [":<pkg>"]) verified against runtime/pkg/data/BUILD, and
  a check-it-worked instruction (re-run bazel test, look for the
  "No test targets were found" error).
- TESTBOT_RULES.md verification step: explicitly call out the
  "No test targets were found" failure mode pointing at BUILD wiring,
  and add a "BUILD wiring (Python + Go)" subsection that explains why
  a go_library-only BUILD silently drops adjacent _test.go files.
- TESTBOT_PROMPT.md guardrails: include go_test alongside py_test in
  the allowed BUILD edits.

This is a documentation-only change to the prompt files; no code paths
or tests change. The 14 testbot bazel tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jiaenren jiaenren requested a review from a team as a code owner May 5, 2026 23:41
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 5, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bd7581bc-fb43-421f-b9da-db1cd76dad62

📥 Commits

Reviewing files that changed from the base of the PR and between 2791843 and 19e6a07.

📒 Files selected for processing (1)
  • src/scripts/testbot/TESTBOT_PROMPT.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/scripts/testbot/TESTBOT_PROMPT.md

📝 Walkthrough

Walkthrough

This PR augments testbot documentation with cross-language Bazel BUILD wiring guidance for Python and Go tests, clarifies Bazel test failure handling, and tightens guardrails to permit edits only to test files and BUILD entries (no source/config changes).

Changes

Testbot BUILD wiring & Guardrails

Layer / File(s) Summary
Test Failure Guidance
src/scripts/testbot/TESTBOT_RULES.md (lines 56–59)
Adds Bazel-specific note under "Run the test": if bazel test reports no test targets, the test file may not be wired into the BUILD; points to BUILD wiring section.
BUILD File Guidance (data / examples)
src/scripts/testbot/TESTBOT_PROMPT.md (lines 32–51)
Introduces cross-language BUILD guidance with example snippets for py_test() and go_test() wiring, showing load statements, go_library/go_test pattern, and embedding for coverage.
BUILD Wiring Rules (requirements)
src/scripts/testbot/TESTBOT_RULES.md (lines 69–81)
Specifies required wiring: Python tests need a py_test() including the new file; Go tests require a go_test() in the package BUILD to ensure _test.go files are recognized and run.
Modification Scope Guardrail
src/scripts/testbot/TESTBOT_PROMPT.md (lines 60–63)
Restricts allowed edits to test files (.py, .go, .ts/.tsx) and BUILD entries (including py_test and go_test); prohibits changes to source, config, or other non-test files.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

"I hopped through BUILDs with a tiny cheer,
Taught py_test and go_test to appear,
Pointed the runner when tests won't show near,
Guardrails snug so only tests we steer. 🐇"

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'testbot: require go_test BUILD targets for new Go test files' directly and specifically describes the main change—adding guidance to require go_test rules for new Go test files in Bazel BUILD files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jiaenr/testbot-go-test-target

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.93%. Comparing base (c87cc34) to head (19e6a07).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #959   +/-   ##
=======================================
  Coverage   42.93%   42.93%           
=======================================
  Files         218      218           
  Lines       28458    28458           
  Branches     4255     4255           
=======================================
  Hits        12218    12218           
  Misses      15599    15599           
  Partials      641      641           
Flag Coverage Δ
backend 43.71% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/scripts/testbot/TESTBOT_PROMPT.md`:
- Line 42: The fenced code block containing the Bazel snippet starting with
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") is missing a
language tag and triggers markdownlint MD040; update the opening fence from ```
to ```bzl so the block is fenced as ```bzl ... ``` to properly label the code as
Starlark/Bazel.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f9ac1f4d-2f3d-4e14-a719-e4e22efeb778

📥 Commits

Reviewing files that changed from the base of the PR and between c87cc34 and 2791843.

📒 Files selected for processing (2)
  • src/scripts/testbot/TESTBOT_PROMPT.md
  • src/scripts/testbot/TESTBOT_RULES.md

Comment thread src/scripts/testbot/TESTBOT_PROMPT.md Outdated
…D040

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants