Add host-backed GPU computation and a Metal graphics demo - #190
Merged
Conversation
…4.29% The C runtime suites added in the previous pass had never been executed. Running them found real bugs, all fixed here: - string_runtime: %.10g silently lost value — 1234567890.5 rendered as "1234567890" and then gained a fabricated ".0". Widen precision only as far as a round trip requires, so short decimals keep their short spelling and no printed float names a different double than the one held. - http_shared: base64_encode never emitted RFC 4648 padding, so the trailing zero octets encoded as literal 'A's. Every 16-byte Sec-WebSocket-Key and 20-byte SHA-1 accept token was therefore base64 no conforming peer could decode — the handshake could not match RFC 6455. - http_shared: generate_websocket_key re-seeded srand(time(NULL)) on every call and drew from rand(), so two connections opened in the same second sent an IDENTICAL nonce. It now draws from the runtime's existing OS CSPRNG (random_runtime's entropy source, exported rather than duplicated). - profiler_runtime: OspProfSlot advertised fiber_id/label but fill_slot never populated them, so every osp_prof_self_slot consumer read 0/"". - test_http_length_validation composed ~4KB of JSON into a 2000-byte stack buffer (_FORTIFY_SOURCE trapped it the first time the suite was built) and asserted a hand-written 33 for a string that is 37 bytes. Two test expectations were wrong about intended behavior, not the runtime: deterministic fibers report ready before running by design (a poll loop would otherwise spin forever), and the boundary-stress float now prints its exact value. C coverage thresholds now carry each library's measured gcov line coverage with cross-platform slack rather than an aspirational 90, so the gate is real and ratchets; term_runtime/test_runtime are documented as unmeasurable under the fork harness instead of gated at a fake number. Duplication: 5.12% -> 4.29% (the branch had breached the committed 5% ceiling, now ratcheted to 4.4%). One statement walker for the LSP outline and hover; one receiver-first desugaring in effect rows; one endpoint factory and one LSP trace helper in the web compiler; one shared support module for the two wasm smoke scripts; one Prism grammar module for the 11ty build and the browser studio; macro-generated type constructors; one group-scanning lookahead in the ML parser; shared list/range loop and predicate helpers in iterator codegen; and type_is_resolved delegating to has_type_var.
This was referenced Aug 11, 2026
Open
Open
Ten open effect defects reduce to three root causes — the fixed-width untyped operation mailbox, per-handler resumption-mode scanning, and a handler set that lives on the thread stack rather than travelling with the continuation. The umbrella sequences them; this records the framing where the plan already tracks #182/#184/#185 individually.
… green `tests/core/gpu/kernel_frontier` and `tests/core/gpu/scalar_contracts` were authored red as forward contracts. Both now pass in both flavors under default/GC/ARC and wasm32, and `cross_flavor_ir_equiv` passes on the ML kernel-frontier twin. None of the defects behind them were GPU-specific. Unconstrained arithmetic int-defaulted at the definition (plan 0022 F10). `fn plus(a, x) = a + x` typed as `(int, int) -> Result<int, MathError>` before any call site was in sight, so `gpuFold(0.0, plus)` over a float buffer was rejected with `cannot unify int with float`. A site whose operands are both still unconstrained now records a PENDING overload; the choice is settled once, after all unification, by re-running the ordinary selection over the operands' final types. Re-running rather than restating the rules is what keeps `p.x * p.x + p.y * p.y` right: by the time the outer `+` resolves its operands have become `Result<int, MathError>`, and only the real selection knows to unwrap them and keep one flattened error channel. The operand does not generalize -- with no numeric class to quantify over, one definition gets one overload, so a helper used at both `int` and `float` in a single program is a type error rather than a silent reinterpretation. That limit is now written into [GPU-KERNEL-ELEM-TYPING]. `any` parameters absorbed open variables. Unification binds a variable before it reaches the `any` wildcard arm, so `expect(add(1, 1), 2)` bound the pending overload of `fn add(a, b) = a + b` to `any` -- the published signature then said a plain word where codegen emitted a `Result`, and every Test Explorer case returned nothing. Assigning an UNRESOLVED variable to an `any` parameter is now skipped: `any` unifies with everything, so a resolved argument already learns nothing there and an unresolved one learns nothing either while costing the variable. Constrained built-ins already deferred this by name; it now holds by shape. The `?:` payload was typed as a fresh variable. `bind_result_fields` matched on field NAME and the desugarer binds the `?:` payload through an unspellable name it did not recognise, so the fallback never constrained the payload: `listGet([1, 2, 3], 0) ?: 9.5` type-checked over an `int` list and only failed in codegen with `match arms disagree on type`. A `Success`/`Error` pattern over an unresolved discriminant now pins it to a `Result` with an open payload instead of auto-wrapping it. An empty list literal reached the backend with no element type. `Expr::List` carries a position, inference publishes each literal's resolved `List<T>` there (`ProgramTypes::lists`), and both `[]` and `toGpu([])` tag their handle from it instead of letting reads default to `int`. The GC's conservative stack scan missed every caller frame on Windows. `gc_stack_base` fell through to a generic fallback returning the address of a local in itself, and it runs on the FIRST allocation -- arbitrarily deep -- so the scan range excluded the frames holding the roots. clang rejects returning a local's address; gcc does not once it is cast to an integer, so it built silently and mis-collected. Windows now reads `NT_TIB::StackBase`; every other platform without an exact query uses a pre-`main` sample. Forcing the old fallback on macOS reproduces the exact assertion `memory_gc_stack_root_tests.c` was failing with on CI. Static-handler discharge could overflow the stack. `REWRITE_BOUND` capped how many rewrite steps run, not how deep they nest, and each substitution re-enters the rewrite -- so two static handlers whose arms perform each other's effect recursed ~10,000 frames and aborted the process instead of reporting. A depth bound now reports the same [STAGE-STATIC-FINITE] violation. Test corpus: - `staged_mutual_static_handlers_diverge.ospo` and `staged_static_arm_arity_mismatch.ospo` pin two discharge diagnostics that had no case at all. - `every_language_test_compiles_to_ir` now compiles BOTH flavors. The ML frontend builds AST forms the Default lowerer never does (`Expr::MethodCall` among them), so compiling only `.osp` left the ML half of every shared walker unexercised in-process. This is what returns `osprey-ast` to 97.7% (from 95.9%, below its 97% gate). - `recursive_generic_needs_annotation.ospo` was retargeted: the program it carried is WELL-FORMED under the arithmetic fix -- it compiles and prints the right answer -- so it now holds a recursive helper that is still genuinely unspecialisable. That leaves `genfn.rs`'s re-entry diagnostic without a corpus case, tracked as #201. Verified locally: `make test` green end to end (all 9 Rust crates and all 25 C libraries over their coverage gates, corpus 179/179 byte-exact under default/GC/ARC with `TEST_CORPUS_ARC_LEAKY=0`, 267 extension tests), wasm32 126/126, `cargo clippy --workspace --all-targets -- -D warnings` and `cargo fmt --all --check` clean. `tests/core/gpu/stress.test.ospml` crashed Node's WASI host on the CI wasm job (`Check failed: storage_.is_populated_`). It does not reproduce locally across four runs and two Node versions, and the two flavors compile to byte-identical modules apart from the embedded filename, so it cannot be a flavor-specific defect. Tracked as #202.
…inux `test_security_edge_cases` proves snprintf's truncation contract: that it reports the length it NEEDED and still terminates what it wrote. gcc can see the source string's length at the call site, proves the truncation statically, and rejects the call under -Werror=format-truncation — so the contract became uncompilable rather than tested. clang does not diagnose it, which is why the suite was green on macOS and red on CI. Reaching the source through a volatile pointer keeps its length out of the optimizer's reach, so the call is checked where it is meant to be: at runtime, by the three assertions that follow. No assertion changed and nothing is suppressed. This step had never run on CI before — the job failed earlier, at the Rust tests, on every previous attempt.
…first
The image list is written by a `dl_iterate_phdr` callback, and its
"already wrote one" flag lived in a FUNCTION-LOCAL STATIC. The flag therefore
survived the dump that set it, so every later capture in the same process
opened its array with the separator -- `"images":[,{...}` -- which no JSON
reader accepts. A second profile lost the image list its sample addresses are
symbolized against. macOS derives the separator from its loop index and was
never affected, which is why this only ever showed on Linux.
The state now travels with the callback.
`test_end_to_end_capture` caught it only because an earlier test in the suite
happened to dump first; the shape it asserts is now pinned deliberately by a
SECOND capture in the same process, so the defect cannot come back disguised as
suite ordering.
Verified on the platform that was failing: all 21 C suites built with gcc 12.2
on Linux and run green, including the profiler suite and the
format-truncation fix from the previous commit.
… platform Two defects, one in the gate and one in the numbers it enforces. The gate could pass VACUOUSLY. It reads the library list with jq; when jq is absent the list is empty, the loop body never runs, fail stays 0, and it prints 'all C libraries meet their thresholds' having measured nothing. Reproduced in a Debian container with no jq: a clean green with zero libraries checked. It now refuses to run without jq, and refuses an empty library list, exactly as the deslop gate above it already does -- a gate that cannot run must not report success. The thresholds were set from macOS alone. macOS and Linux gcov disagree by far more than the '~2 points of rounding' this file assumed, and they disagree in BOTH directions -- measured on the same commit: system_runtime 78.24 macOS 70.24 Linux fiber_runtime 74.83 macOS 70.64 Linux websocket_client_runtime 37.96 macOS 30.34 Linux random_runtime 47.50 macOS 50.00 Linux websocket_server_runtime 24.89 macOS 27.01 Linux So three thresholds were unreachable on the platform the required job runs on, and passed locally. A threshold has to hold on EVERY platform the gate runs on, so each is now the weakest measured platform minus ~2 points: system_runtime 76 -> 68, fiber_runtime 72 -> 68, websocket_client_runtime 35 -> 28. The other 22 already held on both and are unchanged. This is a recalibration, not a relaxation: the numbers were never valid for the gating platform, and the gate is strictly harder to fool than before. The distance to the 90% target is unchanged and stays tracked in #197. Verified: the gate passes on macOS, its Linux numbers are reproduced exactly in a container (70.24 / 70.64 / 30.34, matching the CI run), and both new guards fail closed.
The 30-minute budget was written against a job that had never finished. It always died early — at the Rust tests, then the C compile, then the C coverage gate — so the later stages had never run and were never in the number. The first run to get through them all was killed at 30 minutes, part-way through the extension suite and still making progress. Measured stage times from that run are recorded beside the setting. The extension suite dominates: it drives a real VS Code under xvfb and spawns lldb-dap per debug-adapter case, minutes slower on a runner than the ~2 min it takes locally. 60 minutes leaves headroom without hiding a hang: a stuck run still fails, it just fails later.
… uses freed memory Neither red check was an Osprey bug, and no test, assertion, golden or corpus floor changes here. Debugger E2E (8 timeouts): action.yml pinned lldb-dap to llvm-toolchain-<dist>-22, which is not a pin. LLVM 22 is still in development, so that suite is a rolling trunk nightly whose builds all report themselves as "22.1.8" regardless of which day's trunk they are; only a released major gets a frozen branch. The file is byte-identical on main and gpu — a month of upstream trunk moved underneath it, onto a snapshot whose lldb-dap never emits the `initialized` event and so deadlocks every DAP client on every binary, a trivial clang-compiled C program included, while the plain lldb CLI debugs the same binaries fine. Verified in a noble container against all six debug.e2e.test.ts workflows: the 22 snapshot passes 0/6, the released 21 channel passes 6/6. So pin to 21, resolve the binary by name instead of "newest lldb-dap on the box", and drop the fallback that silently degraded to distro lldb 18 — the known-broken server the pin exists to avoid. lldb-dap-preflight.py now runs the same handshake against a three-line C program right after install, so a defective adapter fails there by name in seconds instead of as eight mystery test timeouts. wasm32 (stress.test.ospml, empty output and exit 139): node:wasi caches the module's memory backing store when the instance starts and never refreshes it after memory.grow, so every later WASI call touches freed memory — a SIGSEGV inside node on x86_64 with no stderr and no wasm trap, or, where the stale page is still mapped, the module's output silently dropped. Twelve lines of hand-written wat reproduce it with no Osprey involved. Node 20 fails the minimal probe; 22 fixes that one but still dies on repeated grow-then-write; 24 runs both it and the whole corpus clean. The twins emit byte-identical IR, which is why one run failed one twin and another failed both. The jobs that execute modules move to Node 24, and wasm-smoke.mjs refuses older hosts by name so a defective runner can never again present as a compiler bug; wasm-browser-smoke.mjs reads memory afresh per call and stays a second, independent oracle. Verified under Node 24: GOLDEN_PASS=126 FAIL=0 (floor 126), GPU_MODE_PASS=18 FAIL=0 (floor 18). Root causes recorded on #202 and #203.
Its `npm test` starts with `build:wasm` -> `make wasm-site`, which runs the studio modules under node:wasi, so it belongs with the other module-executing jobs on 24. Missed in the previous commit, which bumped only the wasm job and deploy-pages; wasm-smoke.mjs then correctly refused the node 20 host and took the job red.
…dule osprey-cli's wasm::tests::build_and_run_end_to_end_when_toolchain_present shells out to scripts/wasm-smoke.mjs, so `make test` executes a module under node:wasi and needs the same Node 24 as the other module-running jobs. wasm-smoke.mjs correctly refused the node 20 host and failed the test rather than let the use-after-free through. That is the last of them: wasm-smoke.mjs is reached from the wasm job, the Website E2E job (via make wasm-site), this job (via the Rust test), the Makefile and run_test_corpus.sh. release.yml runs none of them.
This was referenced Aug 12, 2026
String interpolation of a function value prints raw pointer bytes instead of failing to compile
#205
Open
MelbourneDeveloper
added a commit
that referenced
this pull request
Aug 12, 2026
## What was wrong Both branch rulesets were `enforcement: disabled`. `main` had **no required status checks at all** — any PR could merge red and nothing would stop it. PR #190 (212 files, +60,766/−37,428) merged at 10:38 on 2026-08-12. Issues #202 (wasm corpus aborts Node's WASI host), #203 (all 8 lldb-dap E2E tests time out on Linux) and #204 (website E2E flakes red) were filed at 10:09–10:10 — **29 minutes earlier**. The failures weren't missed; they were written down and merged past. The websocket showstoppers #192–#197 were filed the previous evening. ## Two structural traps kept the gates off **Phantom context.** Ruleset 7726557 required a check named `"CI"`. No job reports that name — check runs are named by job `name:`, not by workflow. Enabling it would hang every PR pending forever, so the reachable fix under pressure is to switch the ruleset off. Its `required_status_checks` rule is dropped; it now carries the PR requirement (squash-only merges, thread resolution) and ruleset 6154907 owns the checks. **Path-filtered required job.** `ci-windows.yml` skipped website-only PRs via `on: paths-ignore:`, which *cannot* be a required check: a filtered-out run never reports, and a required check that never reports blocks the merge forever. Its own comment said as much and concluded "so it isn't required". Converted to job-level `if:` skipping — which reports `skipped`, and a skipped check counts as **passing** — so `windows-core` can now be required. ## State now Both rulesets `active`, **no bypass actors** (applies to admins too), strict up-to-date policy, and six required checks: | Check | Was | |---|---| | Detect changed areas | not required | | Test, Format, Build & Validate | the only required check — and unenforced | | Rust Compiler (fmt, clippy, test, corpus) | not required | | WebAssembly target (wasm32-wasip1) | **advisory by design** — how #202 landed unopposed | | Website E2E (Playwright) | not required | | Windows Core Build & Smoke Test | not required, and not requirable | ## The part that stops it recurring Branch protection lives in GitHub's settings, not in this tree, so it drifts silently and by definition no test covers it. [`scripts/verify-branch-protection.mjs`](scripts/verify-branch-protection.mjs) is that test. It runs in the `changes` job — the one required job that never skips — and fails on: - a ruleset that isn't `active`, or that has bypass actors - drift **in either direction** between the pinned list and the live ruleset - a required context matching no job `name:` (the phantom trap) - a required job whose workflow path-filters at the `on:` level (the deadlock trap) Verified against the real broken state rather than assumed: fed the exact 2026-08-12 configuration, it reports every fault, phantom `"CI"` included. Fed the pre-fix `ci-windows.yml`, it reports the path-filter deadlock. Stale comments asserting these jobs were unrequired are corrected in place, and `CLAUDE.md` gains the rule this violated: a gate you can turn off is not a gate, "advisory" means deleted, and an open issue about a red check is a blocker rather than a footnote. ## Not covered here The failures themselves are still open — #202, #203, #204, the websocket showstoppers #192–#197, and the effects umbrella #200. This PR only ensures the next one can't merge past them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Adds typed
GpuBuffer<T>computation with eleven pure-kernel built-ins and a deterministic CPU host backend across native and WebAssembly targets, plus a separate macOS Cocoa/Metal graphics demo. Thegpu*built-ins do not offload work to GPU hardware yet; device code generation remains roadmap work.It also now blankets the C runtime in assertion-dense tests behind a real per-library coverage gate, and cuts workspace duplication below the committed ceiling.
Details
Added
int,float, andbool, withtoGpu,fromGpu,gpuLength,gpuMap,gpuFold,gpuZipWith,gpuIota,gpuGet,gpuScan,gpuFilter, andgpuDevice.make gpu-demofor the host-backed raster suite and a separate macOS-only Cocoa/Metal FFI bridge with an animated fragment-shader demo.make testthat both runs them and gates each library's gcov line coverage fromcoverage-thresholds.json. Two previously orphaned suites (test_system_runtime,test_http_length_validation) are now actually built and run.Fixed — real defects the new suites caught
osp_float_to_stringused%.10g, which silently lost value:1234567890.5rendered as1234567890and then gained a fabricated.0. It now widens precision only as far as a round trip requires, so short decimals keep their short spelling and no printed float names a different double than the one held.base64_encodenever emitted RFC 4648 padding — the trailing zero octets encoded as literalAs. Every 16-byteSec-WebSocket-Keyand 20-byte SHA-1 accept token was base64 no conforming peer could decode, so the handshake could not match RFC 6455.generate_websocket_keyre-seededsrand(time(NULL))per call and drew fromrand(), so two connections opened in the same second sent an identical nonce. It now draws from the runtime's existing OS CSPRNG.parse_websocket_framesign-extended its extended-length bytes throughchar, so every payload of 128–255 bytes was rejected.OspProfSlotadvertisedfiber_id/labelthatfill_slotnever populated, so everyosp_prof_self_slotconsumer read0/"".test_http_length_validationcomposed ~4 KB of JSON into a 2000-byte stack buffer (_FORTIFY_SOURCEtrapped it the first time the suite was built).Changed
term_runtime/test_runtimeare documented as unmeasurable under the fork harness rather than gated at a fake number.type_is_resolveddelegating tohas_type_var.Spec / Docs
Breaking Changes
toStringon a float now prints the shortest representation that round-trips rather than 10 significant digits, so a value needing more digits prints them (16.666666666666668, not16.66666667). Two goldens updated accordingly.How Do The Automated Tests Prove It Works?
.osp/.ospmlgolden suites mirror 34 cases across both syntaxes: dense round trips; integer, float, and bool kernels; every combinator and edge case; game and ML workloads; raster composition; and million-element stress pipelines.Local validation on this revision:
make _coverage_check_c_runtimegreen for all 25 gated libraries.cargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warningsclean.0 live objectsat exit.test.sh, and a direct 200/422/400 status check on both endpoints; extension typecheck and Shipwright manifest validation.The forward contracts are now green
tests/core/gpu/scalar_contractsandtests/core/gpu/kernel_frontierwere authored red as forward contracts. Both now pass, in both flavors, under default/GC/ARC and wasm32, andcross_flavor_ir_equivpasses on the ML kernel-frontier twin. Three defects had to be fixed for that, none of them GPU-specific:fn plus(a, x) = a + xtyped as(int, int) -> Result<int, MathError>before any call site was in sight, sogpuFold(0.0, plus)over a float buffer was rejected withcannot unify int with float. A site whose operands are both still unconstrained now records a PENDING overload; the choice is made once, after all unification, by re-running the ordinary selection over the operands' final types. The operand does not generalize — one definition gets one overload, so a helper used at bothintandfloatin one program is a type error rather than a silent reinterpretation, and that limit is now written into[GPU-KERNEL-ELEM-TYPING].?:payload was typed as a fresh variable.bind_result_fieldsmatched on field NAME, and the desugarer binds the?:payload through an unspellable name it did not recognise — so the fallback never constrained the payload.listGet([1, 2, 3], 0) ?: 9.5type-checked over anintlist and only failed in codegen withmatch arms disagree on type. ASuccess/Errorpattern over an unresolved discriminant now pins it to aResultwith an open payload instead of auto-wrapping it.Expr::Listnow carries a position, inference publishes each literal's resolvedList<T>there (ProgramTypes::lists), and both[]andtoGpu([])tag their handle from it instead of defaulting reads toint.Two more defects the same sweep turned up and fixed:
gc_stack_basefell through to a generic fallback that returns the address of a local in itself, and it runs on the FIRST allocation — arbitrarily deep — so the scan range excluded the frames actually holding roots. clang rejects returning a local's address; gcc does not once it is cast to an integer, so it built silently and mis-collected. Windows now readsNT_TIB::StackBase; every other platform without an exact query uses a pre-mainsample. Forcing the old fallback on macOS reproduces the exact assertionmemory_gc_stack_root_tests.cwas failing with on CI.REWRITE_BOUNDcapped how many rewrite steps run, not how deep they nest, and each substitution re-enters the rewrite — so two static handlers whose arms perform each other's effect recursed ~10,000 frames and aborted the process instead of reporting. A depth bound now reports the same[STAGE-STATIC-FINITE]violation.examples/failscompilation/staged_mutual_static_handlers_diverge.ospopins it, alongsidestaged_static_arm_arity_mismatch.ospo.examples/failscompilation/recursive_generic_needs_annotation.ospohad to be retargeted: the program it carried is WELL-FORMED under the arithmetic fix (it compiles and prints the right answer), so it now holds a recursive helper that is still genuinely unspecialisable. That leavesgenfn.rs's re-entry diagnostic without a corpus case — tracked as #201.tests/core/gpu/stress.test.ospmlcrashed Node's WASI host on the CI wasm job (Check failed: storage_.is_populated_). It does not reproduce locally across four runs and two Node versions, and the two flavors compile to byte-identical modules apart from the embedded filename, so it cannot be a flavor-specific defect — tracked as #202.