feat: integrate fork-choice compliance spec test suite

[GrapeBaBa]

Summary

Integrates the consensus-specs Fork Choice Compliance suite (ethereum/consensus-specs#3831) into the existing forkChoiceTest runner. It runs alongside the standard fork_choice / sync runners under test:spec:* once the compliance fixtures are downloaded, and cleanly skips when they are absent (so existing CI is unaffected).

Scope: test integration only. This PR wires up the suite and makes the runner honor the compliance test-format semantics. Genuine Lodestar-vs-spec behavior differences it surfaces are not fixed here — they are deferred to dedicated follow-up PRs (see Still failing).

What changed, and the reasoning

1. getViableHeads() on ProtoArray — packages/fork-choice
The suite's headline new check is viable_for_head_roots_and_weights, so we expose a getter to answer it. It enumerates the filtered-block-tree leaves (a node that is viable AND has no viable descendant), restricted to the justified-checkpoint subtree to match the spec's get_filtered_block_tree (rooted at store.justified_checkpoint.root).
Consideration: the justified-subtree restriction is the correctness-critical part. Enumerating all proto-array nodes would wrongly include FFG-viable leaves that hang off the finalized checkpoint on a branch not under the current justified checkpoint. getHead/findHead were always correct (they start the walk at the justified root); only a full-set getter needs this guard added explicitly.

2. Wire the suite into forkChoiceTest — packages/beacon-node
Register fork_choice_compliance as a default runner in specTestIterator, reusing the existing block / attestation / tick step handling. Two new check fields: viable_for_head_roots_and_weights (compared against getViableHeads(), sorted by root since the spec fixes no order) and head_payload_status (gloas, mapped between our internal enum order and the spec's).

3. Test-format accommodations in the runner
Consideration: each of these mirrors an invariant production already enforces in another layer — they are NOT changes to fork-choice behavior, and each is anchored to a test-format signal (not a global "compliance mode" flag), matching Teku's flagless executor.

• bls_setting: 2 — every compliance fixture uses placeholder signatures. Skip signature verification via validSignatures (standard fixtures are bls_setting: 1, unchanged). gloas envelope / payload-attestation sig steps get the same guard.
• ALREADY_KNOWN on duplicate blocks — fixtures intentionally re-import the same block; spec on_block(known_block) is a no-op success. The runner treats ALREADY_KNOWN as success only when the step is valid: true. The production import path is untouched.
• Cross-epoch attestation shuffling — on_attestation must decode aggregation_bits with the state at the attestation's target checkpoint, not the head state. The runner resolves the right shuffling via ShufflingCache + regen, mirroring the production gossip-validation path.
• Attestation timing precondition — replicate the spec validate_on_attestation precondition current_slot >= data.slot + 1 before calling onAttestation (Lodestar enforces this in the gossip/pool layer, not inside fork choice).

4. scripts/download-compliance-fc-tests.sh
Compliance fixtures are not in the standard release tarball — they are produced by make comptests and published only as a transient GH Actions artifact by the daily comptests.yml workflow.
Consideration: the script extracts into the same packages/beacon-node/spec-tests/tests/... root the release tarball uses; paths don't collide because the compliance generator only emits fork_choice_compliance/ subtrees. Supports --run-id / --tarball / auto-fetch latest run, and handles both raw tar.gz and zip-wrapped artifact responses.

Still failing

Run against the latest comptests small.tar.gz (minimal preset, fulu + gloas, 1472 cases each), rebased on current unstable:

Standard fork_choice + sync (all forks): 0 failures — additive integration, no regression.

fulu compliance: 1469 / 1472 (3 fail)

• 2 × perf timeout (block_tree_test_20_598691297_0/1, ~15 s hard limit) — heavy scenario, environmental
• 1 × straggler (block_tree_test_24_300285871_1) — attestation/head divergence

gloas compliance: 166 / 1472 (1306 fail) — all genuine, deferred fork-choice divergences; none are runner/infra issues:

• Invalid head (dominant) — gloas ePBS head-selection divergence (PTC votes / payload-status weighting). Root cause not yet located.
• Invalid viable head roots — getViableHeads() weight/root attribution differs on gloas payload-status variants (the EMPTY vs FULL leaf of one block root). Same root-cause family as Invalid head.
• a handful of proposer-boost-root and beacon-chain-error stragglers.

gloas ePBS is still moving upstream (~alpha.11); closing these belongs in dedicated gloas fork-choice PRs, not here.

Running it

pnpm download-compliance-fc-tests --run-id <id>
pnpm vitest run --project spec-minimal -t fork_choice_compliance test/spec/presets/fork_choice.test.ts

Test plan

• pnpm build + pnpm check-types clean
• pnpm lint clean
• Standard fork_choice / sync runners: 0 failures (no behavior change)
• Compliance run on latest artifact: fulu 1469/1472, gloas 166/1472 — all failures are deferred spec-behavior divergences, no infra-side failures

AI Assistance Disclosure

Used Claude Code (Opus).

[gemini-code-assist]

Code Review

This pull request introduces support for the fork-choice compliance test suite by adding a fixture download script and updating the test runner to handle cross-epoch attestations and duplicate block imports. It also implements a getViableHeads method in the fork-choice logic to allow verification of the internal filtered block tree state. Feedback was provided to use bigint instead of number for weight calculations in getViableHeads to prevent potential precision loss on Mainnet.

[gemini-code-assist]

The current implementation of getViableHeads uses number for weights, which leads to precision loss on Mainnet.

In Lodestar, node.weight is stored in EFFECTIVE_BALANCE_INCREMENT units (1 increment = 1 ETH = 10^9 Gwei). On Mainnet, the total active balance is currently ~33M ETH. Multiplying 33,000,000 * 1,000,000,000 results in 3.3 * 10^16, which exceeds Number.MAX_SAFE_INTEGER (9,007,199,254,740,991 ≈ 9 * 10^15).

To maintain accuracy for production use cases (even if compliance tests currently use smaller weights), this should return bigint.

Suggested change

	getViableHeads(currentSlot: Slot): {root: RootHex; weight: number}[] {
	const result: {root: RootHex; weight: number}[] = [];
	for (const node of this.nodes) {
	if (node.bestChild === undefined && this.nodeIsViableForHead(node, currentSlot)) {
	result.push({root: node.blockRoot, weight: node.weight * EFFECTIVE_BALANCE_INCREMENT});
	}
	}
	return result;
	}
	getViableHeads(currentSlot: Slot): {root: RootHex; weight: bigint}[] {
	const result: {root: RootHex; weight: bigint}[] = [];
	for (const node of this.nodes) {
	if (node.bestChild === undefined && this.nodeIsViableForHead(node, currentSlot)) {
	result.push({root: node.blockRoot, weight: BigInt(node.weight) * BigInt(EFFECTIVE_BALANCE_INCREMENT)});
	}
	}
	return result;
	}

[gemini-code-assist]

Update the return type to bigint to match the fix in ProtoArray and avoid precision loss.

Suggested change

	getViableHeads(): {root: RootHex; weight: number}[] {
	return this.protoArray.getViableHeads(this.fcStore.currentSlot);
	}
	getViableHeads(): {root: RootHex; weight: bigint}[] {
	return this.protoArray.getViableHeads(this.fcStore.currentSlot);
	}

[gemini-code-assist]

Update the interface to return bigint for weights.

Suggested change

	getViableHeads(): {root: RootHex; weight: number}[];
	getViableHeads(): {root: RootHex; weight: bigint}[];

[gemini-code-assist]

Since getViableHeads should return bigint to avoid precision loss, the test comparison should also operate on bigint values directly instead of converting them to number via bnToNum.

Suggested change

	const expected = step.checks.viable_for_head_roots_and_weights
	.map((entry) => ({root: entry.root, weight: bnToNum(entry.weight)}))
	.sort((a, b) => a.root.localeCompare(b.root));
	const actual = (chain.forkChoice as ForkChoice)
	.getViableHeads()
	.map(({root, weight}) => ({root, weight}))
	.sort((a, b) => a.root.localeCompare(b.root));
	expect(actual).toEqualWithMessage(expected, `Invalid viable heads at step ${i}`);
	const expected = step.checks.viable_for_head_roots_and_weights
	.map((entry) => ({root: entry.root, weight: entry.weight}))
	.sort((a, b) => a.root.localeCompare(b.root));
	const actual = (chain.forkChoice as ForkChoice)
	.getViableHeads()
	.sort((a, b) => a.root.localeCompare(b.root));
	expect(actual).toEqualWithMessage(expected, `Invalid viable heads at step ${i}`);

[ensi321]

superseded by #9670