diff --git a/README.md b/README.md index df49bdd..ed3c9e4 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,8 @@ > behaviours across **all** the Shen implementations and asserts they produce > the same observable result (differential / conformance testing). -Bifrost sits alongside [Ratatoskr / Yggdrasil](../ratatoskr) in the same Norse -lineage — Ratatoskr is the squirrel that runs *up and down* the world-tree (the -two-stage shaker); Bifrost is the bridge that verifies the worlds at the ends of -the tree agree. +It sits alongside [Ratatoskr / Yggdrasil](../ratatoskr) — the two-stage shaker — +in the same Norse lineage, and drives it for the `--shake` deploy-path checks. Differential testing is its origin, but Bifrost is now three things: @@ -51,28 +49,19 @@ The corpus (`cases/*.json`, driven by `programs/*.shen`) includes: - **CLI parity** — `eval -e` prints the value; `(version)` / `--version` carry the kernel version **41.2**; **stdin-EOF causes a clean exit** (no hang) on every impl. -- **Known divergences** (documented, *not* hard failures): **none currently** — - all previously-tracked divergences have converged (see below). -- **Resolved divergences** (converged across all available impls; now asserted - as **hard agreements**, no longer tagged): - - ~~`float-formatting`~~ — **RESOLVED.** `(+ 0.1 0.2)` now prints the shortest - round-trippable form `0.30000000000000004` on *every* impl. shen-go - previously printed `0.300000` (`%f`, fixed in pyrex41/shen-go#11) and - shen-lua printed `0.3` (`%.14g` via `tostring`, fixed in - pyrex41/shen-lua#24). Exactly-representable floats such as `2.5` and `4.75` - also agree. Now asserted as a hard agreement — any impl that prints a - non-shortest-round-trip float is a real FAIL. - - ~~`int-div-zero`~~ — **RESOLVED.** `(/ 1 0)` now raises a *catchable* kernel - error on every impl, so `(trap-error (/ 1 0) …)` yields `divide-by-zero` - everywhere. shen-go previously returned `maxint` - (`9223372036854775807`); fixed on `fix-go-divzero-and-floatfmt`. Any impl - that fails to raise is now a real FAIL. - - ~~`hush-file-write`~~ — **RESOLVED.** Under `-q` (`*hush* = true`), `pr` to a - **file** stream now writes on *every* impl (`*hush*` gates only the standard - output stream). shen-lua (`fix/hush-pr-file-22`) and shen-rust - (`fix/hush-pr-file-2`) previously silenced the write (zero-byte file); both - fixed. Now asserted as a hard agreement — any impl that silences a - file-stream `pr` is a real FAIL. +- **Divergences** — **none open.** Every tracked cross-port difference has + converged and is now asserted as a **hard agreement** (a regression is a real + FAIL, no longer a documented difference): + - `float-formatting` — `(+ 0.1 0.2)` prints the shortest round-trippable + `0.30000000000000004` everywhere (was shen-go `0.300000` / shen-lua `0.3`; + pyrex41/shen-go#11, pyrex41/shen-lua#24). + - `int-div-zero` — `(/ 1 0)` raises a catchable `divide-by-zero` everywhere + (was shen-go `maxint`). + - `hush-file-write` — under `-q`, `pr` to a **file** stream writes everywhere + (`*hush*` gates only stdout; was a zero-byte file on shen-lua/shen-rust). + - `load-toplevel-echo` — `(load FILE)` echoes each top-level form's value + regardless of fasl-cache state (shen-lua dropped the echo on a warm cache + hit; pyrex41/shen-lua#40). - **Heavy** (`--heavy`) — **Ratatoskr stage-1 parity**: run `(ratatoskr.shake ["tests/fib.shen"] OUT)` on every host and assert the produced `kernel.kl` + `ratatoskr.manifest` are **byte-identical** across diff --git a/bifrost.py b/bifrost.py index 6c6781a..987ad99 100644 --- a/bifrost.py +++ b/bifrost.py @@ -252,8 +252,12 @@ def resolve(rel, default): def normalize(text, extra_prefixes=()): """Normalise an impl's stdout for comparison. - - strip a trailing run-time banner line ("run time: ... secs") emitted by - the kernel `load`/timer on several ports, + - strip the run-time banner ("run time: ... secs") emitted by the kernel + `load`/timer on several ports, INCLUDING the blank line the banner's + leading newline produces (`(cn "\nrun time: " ...)` -> a blank line then + the text line). Stripping only the text left the blank as residue, which + diverged against a port that omits the whole banner (e.g. shen-lua on a + warm fasl-cache hit re-emits the value echo but not the cosmetic banner), - strip `(fn NAME)` declaration echoes that `load` prints (shen-lua's file path is `(load FILE)`, which echoes every define), - strip lone `0`/`nil`/`true` load-echo lines that `load` prints as the @@ -271,6 +275,10 @@ def normalize(text, extra_prefixes=()): s = ln.rstrip() stripped = s.strip() if stripped.startswith("run time:"): + # also drop the blank line the banner's leading "\n" produced, so + # the WHOLE cosmetic banner is stripped, not just its text line. + if out and out[-1].strip() == "": + out.pop() continue if stripped.startswith("(fn ") and stripped.endswith(")"): continue diff --git a/cases/divergences.json b/cases/divergences.json index 8dce293..544f32b 100644 --- a/cases/divergences.json +++ b/cases/divergences.json @@ -6,6 +6,13 @@ "mode": "special-hush", "expect": "agreement", "doc": "RESOLVED (was a known divergence): under -q (*hush* = true), `pr` to a FILE stream now WRITES on every impl. *hush* only gates the standard output stream, never an explicit file stream. Previously shen-lua/shen-rust SILENCED the write (zero-byte file) under -q while shen-cl/shen-go/ShenScript wrote; the shen-lua (fix/hush-pr-file-22) and shen-rust (fix/hush-pr-file-2) fixes converged this. Now asserted as a HARD agreement: every available impl must land in the same bucket (all 'written'); any impl that silences the write is a real FAIL." + }, + { + "name": "load-toplevel-echo", + "mode": "eval", + "expr": "(load \"programs/load-echo-probe.shen\")", + "expect": "agreement", + "doc": "RESOLVED (was a known divergence): `(load FILE)` now echoes each toplevel form's evaluated value on every impl, cache state notwithstanding. The canonical kernel load (klambda/load.kl -> shen.eval-and-print) PRINTS each toplevel result before returning `loaded` (the echo from `shen eval -l FILE` / `(load FILE)` at the REPL; NOT `script` mode, which is quiet everywhere -- so this case uses eval mode). shen-rust and shen-go always emitted it; shen-lua emitted it on a COLD fasl cache but DROPPED it on a WARM fasl-cache HIT (boot.lua returned `loaded` straight out of fasl_replay without re-running the kernel load), making `(load file)` stdout cache-state dependent -- a cross-port divergence AND an intra-port nondeterminism. Fixed in shen-lua by recording the per-form echo bytes as fasl \"e\" records on the miss and re-pr'ing them on the hit (pyrex41/shen-lua#40, PR #41). The cosmetic `run time:` banner stays dropped on a warm hit by design; the normaliser here strips the whole banner (text line + the blank line its leading newline produces) so that omission is not itself a divergence. Now asserted as a HARD agreement: every available impl must echo `\"PROBE\"` / `42` / `(fn p)` (the last stripped by the normaliser) then `loaded`; any impl that drops the value echo is a real FAIL." } ] } diff --git a/programs/load-echo-probe.shen b/programs/load-echo-probe.shen new file mode 100644 index 0000000..d3b3c76 --- /dev/null +++ b/programs/load-echo-probe.shen @@ -0,0 +1,17 @@ +\\ load-echo-probe.shen -- probe whether (load ...) echoes each toplevel form's +\\ evaluated value, as the canonical kernel load requires. +\\ +\\ klambda/load.kl: (defun shen.eval-and-print (Forms) +\\ (map (lambda F (pr (shen.app (eval-kl (shen.shen->kl F)) +\\ "\n" shen.s) (stoutput))) Forms)) +\\ i.e. load PRINTS each toplevel result before returning `loaded`. +\\ +\\ Canonical impls (shen-cl / shen-rust / shen-go, and shen-lua with a COLD fasl +\\ cache or SHEN_FASL=off) print: "PROBE" then 42 then (fn p). +\\ shen-lua on a WARM fasl-cache hit DROPS all three (boot.lua ~796; the +\\ documented "replayed load does not echo per-form values" degrade). That makes +\\ (load file) stdout depend on cache state -- a cross-port divergence and an +\\ intra-port nondeterminism. See cases/divergences.json load-toplevel-echo; pyrex41/shen-lua#40. +"PROBE" +(+ 40 2) +(define p -> ok)