port: make the host build link against current main, and gate it

[andrewboudreau]

Nothing in the decomp toolchain compiles or links port/, so a src/ or include/ change that shifts a symbol's spelling breaks the host build silently. #1049 did exactly that on 2026-08-03 and took 13 of the 14 gate binaries with it. It went unnoticed until someone tried to build. Two more landed on top of it since.

The four breaks

All the same family: a HAL definition kept a spelling the src/ side stopped using.

Symbol	What happened
func_02059650, NestedHeapIterator::Init, func_0206e2f8, data_0209b44c	Defined at C++ linkage while decl_*.h now emits C-linkage references (#1049). SharedFilePtr::Release also needed a real __thiscall method; _ZTV8Platform needed storage.
_Z14ApproachLinearRiii	shims.cpp dropped this bridge because the symbol is "exactly what a host C++ build emits for ApproachLinear(int&, int, int)". That's the Itanium mangling — MSVC emits ?ApproachLinear@@YAHAAHHH@Z. Restored, with the GCC/Clang case noted so it isn't re-deleted.
GX::LoadTexPltt	Migrated to namespace-style C++ as "language-mode migration only … nothing outside this file can shift." True for the ROM link, where the filename is the symbol either way. False here: the name went from hand-spelled to MSVC-mangled. The identical bridge for LoadTex already existed at model_host.cpp:183.
data_020a60b0	Reached at C++ linkage while the storage sits in an extern "C" block. Aliased, not redefined — a second definition links cleanly and then leaves LoadTexPltt writing to a different VRAM base than the budget code reads.

The last two shipped with a premise that is correct for the ROM link and wrong for the host. That's a systematic gap, not carelessness — so this adds the thing that closes it.

The gate

port/tools/port_linkcheck.py — builds the port, fails if it doesn't link. ~2s incremental. Skips loudly without MSVC ("This did NOT pass; it did not run") rather than reading as green. Passes ninja -k 0: the default -k 1 stops at the first failure, which reported 3 breaks when there were 4 and turned diagnosis into fix-one-discover-more. (The report itself then had to be fixed for the same reason — see Review follow-up.)

tools/port_refcheck.py doesn't cover this — it catches a src/ rename stranding a reference, and none of these renamed anything. Only linking catches a linkage break.

port/tools/port_evidence.py — how much of what the port compiles is proven to be the game's code, via enrollment in the byte-exact ROM build. 12 files aren't, including Model::UpdateFileOffsets (recovered code 36 bytes larger than the real function) and Model::LoadCompressedTextureToVram (NONMATCHING). --ratchet fails only when that set grows, matching the must-not-regress shape in notes/pr-validation.md rather than being red from day one.

Also: build-port.cmd and host_frontier.py now locate MSVC via vswhere — the hardcoded VS 2022 BuildTools path no longer exists on this machine.

Verification

On main @ ee0c07fa:

port-linkcheck: OK (all gate binaries)      14/14 link
gates                                       14/14 pass
soak:      455 models, 455 rendered (89,840 tris), 0 faulted
anim soak: 473 pairs, 473 animated+rendered, 0 faulted
ratchet OK: no new unproven files (12 known, baseline 12)

The ratchet holding at 12 across the 616 changed src/ files since the last port work means nothing unproven was pulled in during that churn.

Review follow-up (commit 2)

Code review of commit 1 ran both new tools against a real break instead of reading them, and found two defects in them. Neither touches the linkage fix.

port_linkcheck.py truncated its own damage report. keep[:40], no marker. On the actual failure that is 40 lines of 114, and the truncated report reads as "6 of 14 binaries broke" when 13 did — the same fix-one-discover-more -k 0 was added to end, reintroduced four lines below the comment explaining why it must not be. Now grouped: distinct unresolved symbols (11) and failing binaries (13) are never capped, only the open-ended tail is, and it says how much it dropped. A totals footer keeps a capped run honest.

The symbol capture also had to stop splitting on whitespace — MSVC prints C++ linkage as "char data_020a4d38" (?data_020a4d38@@3DA), so a \S+ capture keys every such symbol on the token "char and merges distinct ones. Undercounting, in the report written to prevent undercounting, in a tool built for a C-vs-C++ linkage outage where both spellings are guaranteed to appear.

port_evidence.py could never mark a file under a src/ subdirectory as proven. Enrolled keys keep their subdirectory (engine/fader/_ZN15FaderBrightness…); the join used Path(rel).stem, which never contains a slash. 138 of the 9,149 enrolled objects live in subdirectories. It put 7 byte-exact FaderBrightness files into the unproven ledger and shipped that in the committed baseline — hence 19 → 12 above. Fixed by keying on the src-relative path, with no bare-stem fallback: that fallback would let a name collide with an unrelated top-level object and report a file proven on another file's evidence.

Also: --gate now refuses to combine with --ratchet/--update-baseline. It narrows the file set before the unproven set is computed, so --update-baseline --gate 9 would have written one gate's files over the whole ledger and failed much later, as a REGRESSION blamed on whoever next ran a full ratchet.

Re-verified: port/hal/ reverted to main → 13 binaries, 11 symbols, exit 1; restored → 14/14 link, exit 0. Ratchet green at 12, unknown bucket empty across all 14 gates.

Gate wiring (commits 3–4)

The follow-up this PR originally deferred is now in it, because reviewing it turned up the reason it mattered: nothing referenced either tool — no workflow, no validation script, no hook, no doc, only each other — and port_linkcheck.py returned 0 when it SKIPPED. On any machine or CI image without 32-bit MSVC it would have read green forever while checking nothing. That is this branch's own bug one level up: #1049 went unnoticed for two days because nothing built port/. A gate nobody runs, which cannot fail when it does not run, is not a gate.

Both tools now share an exit contract:

code	meaning
0	checked, passed
1	checked, FAILED
2	could not check — "this did not run"

--require collapses 2 into 1, for CI or any caller that must not go green without the check actually running.

port_evidence.py raises CannotCheck instead of sys.exit for missing ROM artifacts, a non-byte-exact build on record, and a missing baseline. All three used to exit 1 — indistinguishable from "someone compiled an unproven file into the port", and the two demand opposite responses: re-run the ROM build, versus stop the merge.

port_linkcheck.py also stops cold-configuring by default: building 14 binaries from scratch is minutes, fine when asked for and wrong to spring on someone mid-git push. Unconfigured is now "could not check"; --configure opts in. Same shape the hook already uses for check_references.py.

tools/hooks/pre-push runs both in the every-push section beside port_refcheck, reading the exit code rather than grepping stderr for phrases the way the check_references block above them has to. An exit code cannot drift out of sync with a reworded message.

Verified in a worktree, all six paths:

linkcheck  no toolchain / unconfigured -> 2      --require -> 1
           clean tree -> 0 (14/14)               hal reverted to main -> 1
evidence   no artifacts -> 2                     --require -> 1
           ratchet -> 0 at 12
hook       inputs absent   -> skips loudly, exit 0
           broken linkage  -> REFUSES, exit 1
                              (caught by the cheap default pair alone:
                               2 binaries, 5 symbols)

Install note: this repo can run the hook either via core.hooksPath=tools/hooks (tracked file runs directly; updates arrive with a pull) or by copying it into .git/hooks/. The copy needs re-copying whenever the hook changes, and a stale copy fails silently — it still runs, just without the newer checks. Commit 4 documents both.

🤖 Generated with Claude Code

https://claude.ai/code/session_017tYjz5v5R228Kh7YCH4gkR

[tangos-validator]

✅ PR validation — Passed

noverify: no source/build-data changes in this PR

_{Each changed src/*.c|*.cpp is compiled and its relocated bytes compared to the binary data on a private build box. Passing requires every changed file to reproduce the ROM byte-for-byte with correct relocation targets — this catches WRONG-DEST relocations and non-reproducing near-misses that ledger-scoped linkcheck skips.}

[andrewboudreau]

Rebased onto 0758bc99. This was 314 commits behind, and its last validation ran two days ago against that old base — for a PR whose subject is linking against current main, that verdict had stopped meaning anything.

All five commits replayed with no conflicts.

Checked the one place a clean rebase can still be wrong: tools/hooks/pre-push changed on main while this was open, and a hook file is exactly where a semantic conflict hides behind a textual non-conflict. The composition is purely additive — main's current body is intact and this PR's two checks are appended ahead of remote_ref_check. Diff reviewed by hand.

port_refcheck on the rebased tree: 403/403 references resolve (227 manifests, 21 cmake-symbols, 155 hal-links).

What I could not run: port/tools/port_linkcheck.py, the gate this PR is really about. This machine has cmake and ninja but no C++ compiler — no cl, no clang++, no g++ — so it reports SKIPPED / did NOT pass, exit 2. That is the tool behaving exactly as its own comment says it should, and it is an environment limit here, not a finding about the PR. Whether the host build still links against today's main is precisely the open question, and re-validation against the new base should now answer it.

[andrewboudreau]

Correction to my note above: the rebase alone was not enough — this PR could not be pushed at all, and its own gate is why.

port/tools/port_evidence.py --ratchet aborted with AttributeError: 'str' object has no attribute 'get', and since this PR's pre-push installs that ratchet, the hook refused every push. It has been broken since build/rombuild-eligibility.json gained its {commit, dirty, files} wrapper — it used to be a bare list, and iterating the stamped shape as a list yields its three keys. The first thing done to a row is .get, so it dies on a string rather than saying the format moved.

tools/eligible.py:load_report already reads both vintages and exists so consumers do not have to care. Now used, rather than keeping a third opinion about the file. Added as commit 6e210ee4 on top of your five.

python port/tools/port_evidence.py --ratchet → OK, 2 known unproven files against a baseline of 12.

This is a good illustration of the PR's own thesis. Nothing renamed, no reference went stale, port_refcheck was 403/403 green throughout — and a gate had been dead for however long, exactly the shape of the two-day linkage break #1049 caused that this PR was written to prevent.

port_linkcheck still cannot run here (no cl/clang++/g++), so it exits 2 and the hook correctly treats it as "could not check" rather than a pass.

[andrewboudreau]

Rebased onto cbd57a83. Main took five commits while this sat — #1370, #1372, #1375 and two progress refreshes. All six commits replayed with no conflicts.

Re-checked the one file where a clean rebase can still be wrong, tools/hooks/pre-push, since #1370's line landed on main in the meantime. The composition is still additive: the only removed line is # Install:, which this PR deliberately replaces with the core.hooksPath block, and main's current body is otherwise intact ahead of remote_ref_check.

• port_refcheck: 403/403 resolve
• port/tools/port_evidence.py --ratchet: OK, 2 known unproven files, baseline 12 — still working after last sweep's fix
• No src/, include/ or config/ changes, so the ROM build is not an input here: 11 files under port/, 1 under tools/

port_linkcheck still cannot run on this machine — cmake and ninja are present but there is no C++ compiler (cl, clang++, g++ all absent), so it exits 2 and the hook treats it as "could not check" rather than a pass. That remains the one claim in this PR I cannot verify locally, and re-validation against the new base is what will answer it.

[andrewboudreau]

I found MSVC on this machine and ran your link gate for the first time. It works, and it says the port does not build against current main. Your PR's premise no longer holds — through no fault of the PR.

I have been reporting "cannot check, no C++ compiler" for three sweeps. That was wrong: VS 18 Community with the VC tools is installed, just not on PATH, and port/build-port.cmd locates it via vswhere exactly as its comment promises. It configured and built without complaint.

What the gate found

summary: 12 binaries failed to link, 21 distinct unresolved symbols, 16 other diagnostic line(s)
port-linkcheck: FAILED -- the port does not build against the current src/ and include/.

The unresolved symbols fall into four groups:

• Heap family (12): ExpandingHeapAllocator::{Allocate, Deallocate, AllocateForwards, AllocateBackwards, LinkNode, UnlinkNode, CreateNode, SizeofInternal, MemoryLeft}, ExpandingHeap::VDeallocate, Heap::{SetupRootHeap, CreateExpandingHeapAllocator}, plus the data symbol Memory::defaultHeapPtr
• cstd shims (3): cstd::abs, cstd::sqrt, cstd::fdiv_result
• OAM (2): OAM::GetObjWidth, OAM::GetObjHeight
• Actor (1): Actor::ClosestPlayer, referenced from Platform::IsClsnInRangeOnScreen

And a newer failure mode worth naming separately — LNK2005, duplicate definitions:

_ZN22ExpandingHeapAllocator8AllocateEji.cpp.obj : error LNK2005:
  "ExpandingHeapAllocator::Allocate(unsigned int,int)" already defined in heap_vtable.cpp.obj

port/'s hand-written heap_vtable.cpp shim defines methods that the decomp has since migrated into real C++ methods. That is not a stale reference and not a missing symbol — it is the same symbol arriving from both sides, and it will keep happening as the C++ conversion advances. Whether the shim shrinks or the manifest excludes those TUs is your design call, not something I should guess at.

One thing I did fix, and could not push

The gate could not even reach the link phase: port/tests/smoke.cpp asserts offsetof(Timer, unk_004) == 4 and reads t.unk_004, and neither exists — include/Timer.h on main models the tick count as one s64 unk_000. Neither file is touched by this PR, so this came from main.

That is a clean illustration of your thesis: port_refcheck is 403/403 green and cannot see it, because nothing was renamed and no reference went stale — two fields became one inside a header. Only compiling catches it, which is the argument for the gate.

-static_assert(offsetof(Timer, unk_004) == 4, "Timer +0x4");
+// Timer's first member is ONE s64 at 0x0, not two words at 0x0 and 0x4 --
+// include/Timer.h says why (seven history accesses read it as s64; the paired
+// 4-byte ROM accesses are what a 64-bit value looks like with no 64-bit
+// register). Asserting its start and the field that follows still pins it to
+// 0x0..0x8, which is what this check was really for.
+static_assert(offsetof(Timer, unk_000) == 0, "Timer +0x0");
+static_assert(sizeof(((Timer *)0)->unk_000) == 8, "Timer tick count is 64-bit");
 static_assert(offsetof(Timer, mIsRunning) == 8, "Timer +0x8");
@@
-    CHECK(t.mIsRunning == 0 && t.unk_000 == 0 && t.unk_004 == 0);
+    CHECK(t.mIsRunning == 0 && t.unk_000 == 0);   /* one s64 covers 0x0..0x8 */

I did not push it. With build/port now configured, your own pre-push hook runs the link check, it fails on the breakage above, and it refuses:

port-linkcheck: FAILED -- the port does not build against the current src/ and include/.
pre-push: REFUSING to push (see above). Override with --no-verify if you are certain.

The gate is right, so I am not going to --no-verify past it to land a partial fix. Patch is above, apply it whenever the link work happens.

Worth knowing before you start: that refusal is a bootstrap trap. Once anyone configures build/port, no commit can be pushed to this branch until the port links again, including the commits that would fix it. Consider whether the hook should demand --require only in CI, or accept a recorded-baseline failure the way the other ratchets in this tree do.

Status

Rebase from earlier is intact — six commits on cbd57a83, zero behind, MERGEABLE/CLEAN, and the relay reports PR validation: success. That green is what "could not check" looks like: the worker cannot run port_linkcheck either, so it is not evidence the port builds. This comment is.

[andrewboudreau]

Follow-up triage on the link failure — it is smaller and more mechanical than "21 unresolved symbols" makes it sound.

All 18 named symbols already have real implementations in src/. Nothing here needs a stub written:

group	symbols	src/ file exists
ExpandingHeapAllocator	Allocate, Deallocate, AllocateForwards, AllocateBackwards, LinkNode, UnlinkNode, CreateNode, SizeofInternal, MemoryLeft	9/9
Heap / ExpandingHeap	SetupRootHeap, CreateExpandingHeapAllocator, VDeallocate	3/3
cstd	abs, sqrt, fdiv_result	3/3
OAM / Actor	GetObjWidth, GetObjHeight, ClosestPlayer	3/3

And the heap family is already in slice_gate2.txt — every path in that manifest exists on disk. So this is not missing manifest entries either.

The first thing to fix is the collision, and it is our doing, not yours. port/hal/heap_vtable.cpp stubs four things:

ExpandingHeapAllocator::Allocate      ExpandingHeapAllocator::SizeofInternal
ExpandingHeapAllocator::MemoryLeft    ExpandingHeap::VDeallocate

Every one of those is now a real C++ method in src/, in slice_gate2.txt, so any binary linking both the shim and the slice gets LNK2005. The stubs were correct when the decomp had not migrated them; the C++ conversion landed underneath. That class will keep recurring — each time a function this shim covers becomes a real method, the shim has to give it up. Deleting those four definitions (keeping whatever vtable the file exists to provide) is the obvious first step.

Memory::defaultHeapPtr is a separate shape: the shim provides data_020a0ea0, and the link wants the C++-mangled ?defaultHeapPtr@Memory@@3PAUHeap@@A. That is a linkage mismatch, not a missing symbol.

On the numbers: port_linkcheck.py locates vcvars itself and refuses to run unless LIB is set — the comment at line 92 says that is exactly what makes it a link check rather than a compile check. So its 12-binaries/21-symbols verdict is trustworthy. (A bare ninja smoke_heap from outside the MSVC environment gives LNK1181: cannot open kernel32.lib, which is an artifact of the invocation and not a finding — worth knowing before someone chases it.)

Still not pushing anything: your hook correctly refuses while the link is broken, and which of these to fix by extending a slice versus shrinking the shim is a design call in your workstream. Happy to take it on if you want it — say so and I will open it against main, since that is where port/hal/heap_vtable.cpp and the breakage actually live.