From 27918434772118b65aef44055f1d92d67de6e750 Mon Sep 17 00:00:00 2001 From: jiaen-nv Date: Tue, 5 May 2026 16:41:09 -0700 Subject: [PATCH 1/2] testbot: require go_test BUILD targets for new Go test files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR #956 (testbot-generated) added src/runtime/pkg/common/common_test.go without adding a corresponding go_test rule to the package's BUILD file. The file ended up invisible to Bazel — `bazel test //src/runtime/pkg/common:all` returns "No test targets were found", and `bazel coverage //...` (the job that feeds Codecov) skips it entirely. Net effect: the testbot run that picked common.go because of "0% coverage" closes the issue with a PR that ships zero new coverage. Update both prompts so the next run gets it right: - TESTBOT_PROMPT.md step 6: generalized from "Python only — BUILD file" to cover Go too. Includes the canonical pattern (go_library + go_test with embed = [":"]) verified against runtime/pkg/data/BUILD, and a check-it-worked instruction (re-run bazel test, look for the "No test targets were found" error). - TESTBOT_RULES.md verification step: explicitly call out the "No test targets were found" failure mode pointing at BUILD wiring, and add a "BUILD wiring (Python + Go)" subsection that explains why a go_library-only BUILD silently drops adjacent _test.go files. - TESTBOT_PROMPT.md guardrails: include go_test alongside py_test in the allowed BUILD edits. This is a documentation-only change to the prompt files; no code paths or tests change. The 14 testbot bazel tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/scripts/testbot/TESTBOT_PROMPT.md | 29 ++++++++++++++++++++++----- src/scripts/testbot/TESTBOT_RULES.md | 18 ++++++++++++++++- 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/src/scripts/testbot/TESTBOT_PROMPT.md b/src/scripts/testbot/TESTBOT_PROMPT.md index 8b60b8e37..b512dd579 100644 --- a/src/scripts/testbot/TESTBOT_PROMPT.md +++ b/src/scripts/testbot/TESTBOT_PROMPT.md @@ -29,10 +29,28 @@ The targets are appended below this prompt. For each target you receive: - Target: boundary values, empty/None inputs, error paths, off-by-one. 5. Write (or extend) the test file targeting the uncovered line ranges. Place new test files in the same location convention as step 2. -6. **Python only — BUILD file**: - - Check the `BUILD` file in the test directory for an existing `py_test()` entry. - - If missing, add a `py_test()` rule. Infer `deps` from other `py_test` entries - in the same BUILD file. Do NOT guess target names. +6. **BUILD file** (Python and Go — TypeScript uses Vitest discovery, no BUILD edit): + - **Python**: check the `BUILD` file in the test directory for an existing + `py_test()` entry. If missing, add a `py_test()` rule. Infer `deps` from + other `py_test` entries in the same BUILD file. Do NOT guess target names. + - **Go**: the test file lives next to the source (`_test.go`), so + check the source package's `BUILD` for an existing `go_test()` entry. If + the BUILD only has `go_library` and you added a `_test.go`, you MUST also + add a `go_test` rule referencing it. Without the rule the test file is + invisible to Bazel and `bazel coverage` will not run it. Pattern (verified + against `src/runtime/pkg/data/BUILD`): + ``` + load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") + go_test( + name = "_test", + srcs = ["_test.go"], + embed = [":"], # the existing go_library target name + # add deps only if your test imports something the library doesn't + ) + ``` + Confirm the rule is loaded by running `bazel test `; if you get + `ERROR: No test targets were found, yet testing was requested`, the + `go_test` rule is missing or misnamed. 7. Run the test and verify code style per TESTBOT_RULES.md. If the test fails, follow the bug detection steps in TESTBOT_RULES.md. 8. Move to the next target. @@ -41,7 +59,8 @@ The targets are appended below this prompt. For each target you receive: - **Test files only**: You may ONLY create or modify test files (`test_*.py`, `*_test.go`, `*.test.ts`, `*.test.tsx`) and `BUILD` files (for `py_test` - entries). Do NOT modify source code, configuration, or other non-test files. + and `go_test` entries). Do NOT modify source code, configuration, or other + non-test files. - **No git or gh commands**: Do NOT run `git`, `gh`, or any commands that modify version control state. The harness script handles branch creation, committing, pushing, and PR creation. diff --git a/src/scripts/testbot/TESTBOT_RULES.md b/src/scripts/testbot/TESTBOT_RULES.md index 8062b8f62..ece7106a4 100644 --- a/src/scripts/testbot/TESTBOT_RULES.md +++ b/src/scripts/testbot/TESTBOT_RULES.md @@ -53,7 +53,10 @@ When a test fails: > see both the failure and the output. Just run the command directly. 1. **Run the test**: - - Python/Go: `bazel test ` (derive the Bazel target from the BUILD file) + - Python/Go: `bazel test ` (derive the Bazel target from the BUILD file). + If `bazel test` reports `ERROR: No test targets were found, yet testing + was requested`, the test file isn't wired into BUILD — see "BUILD wiring" + below before retrying. - TypeScript: `pnpm --dir src/ui test -- --run ` 2. If the test fails, read the error and fix. Retry up to 3 times. 3. **Verify code style** (same checks as PR CI). Fix and re-verify until clean: @@ -63,6 +66,19 @@ When a test fails: `pnpm --dir src/ui format` to auto-fix. - Go: no additional checks beyond step 1. +### BUILD wiring (Python + Go) + +A test file that isn't registered as a Bazel target is invisible to CI — +`bazel coverage //...` won't run it and Codecov won't see the new coverage, +defeating the whole point of the testbot run. + +- **Python** (`test_.py`): the `BUILD` file in the test directory must + have a `py_test()` rule whose `srcs` includes the new file. +- **Go** (`_test.go`): the source package's `BUILD` file must load + `go_test` and declare a `go_test()` rule pointing at the new file. A + `go_library`-only BUILD file silently drops adjacent `_test.go` files + on the floor. + ## Language Conventions ### Python From 19e6a072f74b3c675d1443d78e39f7fd61ec6636 Mon Sep 17 00:00:00 2001 From: jiaen-nv Date: Tue, 5 May 2026 16:47:17 -0700 Subject: [PATCH 2/2] testbot: tag bzl fence in TESTBOT_PROMPT.md to satisfy markdownlint MD040 Co-Authored-By: Claude Opus 4.7 (1M context) --- src/scripts/testbot/TESTBOT_PROMPT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/scripts/testbot/TESTBOT_PROMPT.md b/src/scripts/testbot/TESTBOT_PROMPT.md index b512dd579..8dc257743 100644 --- a/src/scripts/testbot/TESTBOT_PROMPT.md +++ b/src/scripts/testbot/TESTBOT_PROMPT.md @@ -39,7 +39,7 @@ The targets are appended below this prompt. For each target you receive: add a `go_test` rule referencing it. Without the rule the test file is invisible to Bazel and `bazel coverage` will not run it. Pattern (verified against `src/runtime/pkg/data/BUILD`): - ``` + ```bzl load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") go_test( name = "_test",