fix(go): extract bodyless function declarations (#1756) - #1760
Conversation
Go's bodyless function declarations (assembly-backed implementations and //go:linkname targets, e.g. func memmove(to, from unsafe.Pointer, n uintptr)) have no brace body, and Go's automatic-semicolon-insertion rule ends the declaration at the signature line without a literal semicolon. The generic Mode-B brace-only fallback in _slice_by_braces dropped every one of them when no unrelated { appeared in the bounded window, and mis-attributed a later struct/interface literal as the body when one did. Add a Go-specific branch that resolves the declaration bound from Go's own ASI rule: after the parameter list closes, the first top-level { is the body, and a literal ; or end-of-line means the declaration is bodyless. A return-type brace group (struct{...} / interface{...}) is skipped when its balanced close is followed by another { on the same line, so those functions keep their real body span. Mirrors the bodyless handling already in place for rust (squid-protocol#1319), objc (squid-protocol#1314/squid-protocol#1336), and csharp (squid-protocol#789).
squid-protocol
left a comment
There was a problem hiding this comment.
Thanks for this, Ruiming — the bodyless-declaration diagnosis is spot on, and I like that you tied it into the same class of gap as #1319/#1314/#789 rather than treating it as a one-off. The new regression tests (struct-return-type wrinkle, mis-attribution guard) are exactly the right shape too.
I did find one regression before merging, though. The new branch's hand-rolled depth tracker treats </> as a bracket pair to skip over:
elif ch == "<":
depth_angle += 1
elif ch == ">":
depth_angle = max(0, depth_angle - 1)Go has no angle-bracket generic syntax ([T any] is square-bracket, always has been), so there's no legitimate Go construct this is meant to balance. But Go does have a bare, unmatched < in a very common place: the channel-direction operator, chan<- / <-chan. That single < bumps depth_angle to 1 and it never comes back down (no closing > ever follows), which permanently blocks the depth_paren == 0 and depth_bracket == 0 and depth_angle == 0 gate for the rest of the scan — so the function's real { body is never found and the whole function gets silently dropped, same failure mode as the bug this PR fixes, just for a different trigger.
Repro against this branch:
from gitgalaxy.core.detector import StructuralExtractor
from gitgalaxy.standards.language_standards import LANGUAGE_DEFINITIONS
code = """package main
func makeSendChan() chan<- int {
ch := make(chan int)
return ch
}
func makeRecvChan() <-chan int {
ch := make(chan int)
return ch
}
func normalFunc() int {
return 42
}
"""
d = StructuralExtractor("go", LANGUAGE_DEFINITIONS)
result = d.splice(code, "", raw_content=code)
print([fn["name"] for fn in result.get("functions", [])])
# -> ['normalFunc'] (makeSendChan and makeRecvChan both vanish)Directional channels are everyday Go — they're all over the stdlib and, not incidentally, all over kubernetes, which is this PR's own Differential Scan target. I'd guess this at least partially offsets the recall gain from the bodyless fix on a real corpus, which the golden-master diff wouldn't surface as a problem on its own (net additions can hide individual regressions).
Since Go doesn't have angle-bracket generics at all, the simplest fix is to just drop the depth_angle tracking entirely rather than special-case <-:
params_end_idx = self._find_balanced_end(safe_code, match.end() - 1, "(", ")")
depth_paren = depth_bracket = 0
pos = params_end_idx
term_idx, term_kind = -1, None
while pos < search_limit:
ch = safe_code[pos]
if ch == "(":
depth_paren += 1
elif ch == ")":
depth_paren = max(0, depth_paren - 1)
elif ch == "[":
depth_bracket += 1
elif ch == "]":
depth_bracket = max(0, depth_bracket - 1)
elif depth_paren == 0 and depth_bracket == 0:
...Worth adding a regression test for a channel-typed return (both directions) alongside the struct-literal one you already have, so this doesn't come back silently. Happy to take another look once that's in — the core fix is solid, this is a narrow miss.
The Go branch of _slice_by_braces tracked < and > as a bracket pair, but Go has no angle-bracket grouping (generics are [T any]): < and > only ever appear as operators, most notably the channel-direction operator (chan<- / <-chan). A lone < bumped the angle depth and never came back down, so the depth_paren == 0 and depth_bracket == 0 and depth_angle == 0 gate never opened again and every function after the first channel operator was silently dropped. Drop angle tracking from the Go branch and track parens and brackets only. Regression test covers send-only and receive-only channel-typed functions followed by a plain braced function.
|
Good catch, thanks for the repro — the channel-direction operator hadn't crossed my mind at all. Go really has no angle-bracket grouping ([T any] is square-bracket, always has been), so tracking < and > in that branch was just wrong, and a lone < from chan<- / <-chan stalls the scan for good. Removed the angle-depth tracking from the Go branch entirely; the gate now only checks paren/bracket depth. Added a regression test with a send-only chan, a receive-only chan, and a plain braced function after them, plus the existing bodyless/struct-return-type cases still pass. Verified: full test_detector.py is green (128 tests), the five go tests pass, ruff/dead-key/ast-accuracy are clean. Couldn't run crucible locally (venv creation fails on Windows here), so it'll need the CI run. |
Two same-day PRs (#1760, #1761) both ran crucible_check.py and the full test suite, checked off the PR template honestly, and still shipped regressions the Differential Scan's 80-repo corpus diff didn't cover -- caught only by manually running tree_sitter_accuracy_audit.py, which wasn't mentioned anywhere in CONTRIBUTING.md or the PR template. Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Removing the angle-depth tracker from the Go body scan means send-only and receive-only channel-typed functions are now extracted, so the Go corpus metrics (impact, total mass, exposures) shift slightly. Regenerated both fixtures via update_golden_master.py; the golden crucible test passes against them.
|
Also reblessed both golden masters - the Go corpus metrics shift slightly because channel-typed functions are now extracted (send-only/receive-only signatures were being dropped before, which is what you caught). Regenerated via update_golden_master.py; golden crucible passes against them locally, and the go tree-sitter audit reports no regressions. |
…ion-declarations-1756 # Conflicts: # tests/golden_master_audit.json # tests/golden_master_zero_dep_audit.json
squid-protocol
left a comment
There was a problem hiding this comment.
The Go angle-bracket fix looks great and perfectly addresses the directional channel edge case. Golden masters are updated with the merge. Approving and merging!
Summary
Fixes #1756. Go's bodyless function declarations -- assembly-backed implementations and
//go:linknametargets likefunc memmove(to, from unsafe.Pointer, n uintptr)-- were being dropped entirely by_slice_by_braces. The generic Mode-B brace-only fallback required a{inside the bounded search window, and since a bodyless declaration ends at the end of its signature line (Go's automatic-semicolon-insertion rule, no literal;), the brace search came back empty and the whole match was discarded. When an unrelated{(a later struct/interface literal) did appear in the window, it was silently attributed as the function's body instead.This adds a Go-specific branch to
_slice_by_bracesthat resolves the declaration bound the way Go's own grammar does: after the parameter list closes, the first top-level{is the body; a literal;or -- far more commonly -- end-of-line means the declaration is bodyless. Return types that themselves contain a brace group (func f() struct{ X int } { ... },interface{ ... }) are handled by skipping a top-level{whose balanced close is followed by another{on the same line, so those functions keep their real body span.funcat line start is unambiguous in Go, so a bodyless terminator is never a false match.Same class of gap already handled for rust (#1319), objc (#1314/#1336) and csharp (#789) -- this is Go's flavor of it.
Type of change
gitgalaxy/core/detector.py,language_standards.py,prism.py, or a per-language rule)CI checklist
python tests/tools/audit_check.py-- ruff/dead-key/ast-accuracy clean; mypy's 6 local findings are platform noise (WindowsSIGALRM+ missingyamlstubs), none in the changed filespython -m pytest tests/-- 6963 passed; two local failures are environment-specific (Windows symlink privilege in test_guidestar_lens.py, and a load-sensitive ReDoS timing check in test_css_strict.py that passes in isolation)pytest -m golden_cruciblepasses in both full-precision and zero-dependency modes against the regenerated baselinesDifferential Scan target
https://github.com/kubernetes/kubernetes -- Go files in the corpus (kubernetes, go core) are where the recall gap shows up.
Verification
The two golden master fixtures (
tests/golden_master_audit.json,tests/golden_master_zero_dep_audit.json) were regenerated withpython tests/tools/update_golden_master.py --yesbecause the output legitimately changed: bodyless Go declarations are now extracted, so Go files in the corpus gained functions (runtime-style asm-backed declarations) and the global 3D coordinate space renormalized. The drift was confirmed Go-only (plus the coordinate renormalization every entity inherits) before blessing.Local runs:
New regression tests in
tests/core_engine/test_detector.py://go:linknamecomment case), spanning only their signature linestruct{ ... }return type does not truncate the function's span to the type literal