Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions src/scripts/testbot/TESTBOT_PROMPT.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,28 @@ The targets are appended below this prompt. For each target you receive:
- Target: boundary values, empty/None inputs, error paths, off-by-one.
5. Write (or extend) the test file targeting the uncovered line ranges.
Place new test files in the same location convention as step 2.
6. **Python only — BUILD file**:
- Check the `BUILD` file in the test directory for an existing `py_test()` entry.
- If missing, add a `py_test()` rule. Infer `deps` from other `py_test` entries
in the same BUILD file. Do NOT guess target names.
6. **BUILD file** (Python and Go — TypeScript uses Vitest discovery, no BUILD edit):
- **Python**: check the `BUILD` file in the test directory for an existing
`py_test()` entry. If missing, add a `py_test()` rule. Infer `deps` from
other `py_test` entries in the same BUILD file. Do NOT guess target names.
- **Go**: the test file lives next to the source (`<name>_test.go`), so
check the source package's `BUILD` for an existing `go_test()` entry. If
the BUILD only has `go_library` and you added a `_test.go`, you MUST also
add a `go_test` rule referencing it. Without the rule the test file is
invisible to Bazel and `bazel coverage` will not run it. Pattern (verified
against `src/runtime/pkg/data/BUILD`):
```bzl
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
go_test(
name = "<pkg>_test",
srcs = ["<name>_test.go"],
embed = [":<pkg>"], # the existing go_library target name
# add deps only if your test imports something the library doesn't
)
```
Confirm the rule is loaded by running `bazel test <target>`; if you get
`ERROR: No test targets were found, yet testing was requested`, the
`go_test` rule is missing or misnamed.
7. Run the test and verify code style per TESTBOT_RULES.md.
If the test fails, follow the bug detection steps in TESTBOT_RULES.md.
8. Move to the next target.
Expand All @@ -41,7 +59,8 @@ The targets are appended below this prompt. For each target you receive:

- **Test files only**: You may ONLY create or modify test files (`test_*.py`,
`*_test.go`, `*.test.ts`, `*.test.tsx`) and `BUILD` files (for `py_test`
entries). Do NOT modify source code, configuration, or other non-test files.
and `go_test` entries). Do NOT modify source code, configuration, or other
non-test files.
- **No git or gh commands**: Do NOT run `git`, `gh`, or any commands that
modify version control state. The harness script handles branch creation,
committing, pushing, and PR creation.
18 changes: 17 additions & 1 deletion src/scripts/testbot/TESTBOT_RULES.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,10 @@ When a test fails:
> see both the failure and the output. Just run the command directly.

1. **Run the test**:
- Python/Go: `bazel test <target>` (derive the Bazel target from the BUILD file)
- Python/Go: `bazel test <target>` (derive the Bazel target from the BUILD file).
If `bazel test` reports `ERROR: No test targets were found, yet testing
was requested`, the test file isn't wired into BUILD — see "BUILD wiring"
below before retrying.
- TypeScript: `pnpm --dir src/ui test -- --run <test_file_path>`
2. If the test fails, read the error and fix. Retry up to 3 times.
3. **Verify code style** (same checks as PR CI). Fix and re-verify until clean:
Expand All @@ -63,6 +66,19 @@ When a test fails:
`pnpm --dir src/ui format` to auto-fix.
- Go: no additional checks beyond step 1.

### BUILD wiring (Python + Go)

A test file that isn't registered as a Bazel target is invisible to CI —
`bazel coverage //...` won't run it and Codecov won't see the new coverage,
defeating the whole point of the testbot run.

- **Python** (`test_<name>.py`): the `BUILD` file in the test directory must
have a `py_test()` rule whose `srcs` includes the new file.
- **Go** (`<name>_test.go`): the source package's `BUILD` file must load
`go_test` and declare a `go_test()` rule pointing at the new file. A
`go_library`-only BUILD file silently drops adjacent `_test.go` files
on the floor.

## Language Conventions

### Python
Expand Down
Loading