fix(conformance): preserve ceil boundaries and pin truthful library probes#998
Merged
Conversation
Keep Math.ceil calculations in binary64 so values above the signed 32-bit boundary are not narrowed before reaching the Number value.
Validate named public-package behaviors against the integrity-pinned es-toolkit release and keep semantic failures, bytecode divergence, disabled capabilities, host globals, and harness failures distinct.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe PR adds a pinned es-toolkit compatibility probe lane with harnesses, integrity-checked package handling, dual-mode execution, classified JSON reports, CI integration, tests, and documentation. It also updates ChangesPinned es-toolkit validation
Math.ceil behavior
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CI
participant ValidationRunner
participant Goccia
participant ProbeHarness
participant ArtifactStore
CI->>ValidationRunner: validate runner and compute integrity cache key
CI->>ArtifactStore: restore pinned es-toolkit tarball
ValidationRunner->>ValidationRunner: verify package and generate import map
ValidationRunner->>Goccia: execute probes in interpreted and bytecode modes
Goccia->>ProbeHarness: run es-toolkit harness
ProbeHarness-->>ValidationRunner: return structured result marker
ValidationRunner->>ValidationRunner: classify outcomes and build JSON report
ValidationRunner-->>CI: write es-toolkit-validation.json
CI->>ArtifactStore: upload validation report
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
Contributor
Web Tooling Benchmark
18 pinned Web Tooling workloads; 18 workloads produced at least one Goccia sample. Raw results from 1 sample per workload; full stdout/stderr for failures and min/max/CV stay in the |
Contributor
Suite TimingTest Runner (interpreted: 11,520 passed; bytecode: 11,520 passed)
MemoryGC rows aggregate the main thread plus all worker thread-local GCs. Test runner worker shutdown frees thread-local heaps in bulk; that shutdown reclamation is not counted as GC collections or collected objects.
Benchmarks (interpreted: 439; bytecode: 439)
MemoryGC rows aggregate the main thread plus all worker thread-local GCs. Benchmark runner performs explicit between-file collections, so collection and collected-object counts can be much higher than the test runner.
Boot
Empty-script ( Measured on ubuntu-latest x64. |
Contributor
Benchmark Results439 benchmarks · PR vs same-runner Interpreted: 🟢 47 improved · 🔴 39 regressed · 353 unchanged · avg +0.6% Typical per-run noise (median variance): interpreted ±1.7%, bytecode ±2.0%. Deltas within noise overlap and read as unchanged. arraybuffer.js — Interp: 🔴 3, 11 unch. · avg -0.5% · Bytecode: 🔴 1, 13 unch. · avg +0.9%
arrays.js — Interp: 🟢 2, 17 unch. · avg +1.5% · Bytecode: 🟢 3, 16 unch. · avg +3.0%
async-await.js — Interp: 6 unch. · avg +2.4% · Bytecode: 6 unch. · avg +2.1%
async-generators.js — Interp: 2 unch. · avg +5.0% · Bytecode: 2 unch. · avg -10.1%
atomics.js — Interp: 6 unch. · avg +1.5% · Bytecode: 🟢 1, 5 unch. · avg +2.3%
base64.js — Interp: 🔴 1, 9 unch. · avg -0.2% · Bytecode: 🟢 2, 8 unch. · avg +0.8%
classes.js — Interp: 🟢 4, 🔴 1, 26 unch. · avg +0.3% · Bytecode: 🟢 1, 🔴 2, 28 unch. · avg +0.3%
closures.js — Interp: 🟢 2, 9 unch. · avg +1.0% · Bytecode: 11 unch. · avg +0.0%
collections.js — Interp: 🟢 1, 11 unch. · avg +1.7% · Bytecode: 🟢 1, 11 unch. · avg -0.4%
csv.js — Interp: 13 unch. · avg +1.5% · Bytecode: 🟢 2, 11 unch. · avg +2.8%
destructuring.js — Interp: 🟢 2, 🔴 3, 17 unch. · avg -1.0% · Bytecode: 🟢 2, 20 unch. · avg +1.1%
fibonacci.js — Interp: 🔴 3, 5 unch. · avg -3.3% · Bytecode: 8 unch. · avg +2.1%
float16array.js — Interp: 🟢 3, 29 unch. · avg +0.6% · Bytecode: 🟢 4, 🔴 4, 24 unch. · avg -0.2%
for-in/for-in.js — Interp: 3 unch. · avg +0.4% · Bytecode: 3 unch. · avg -0.6%
for-of.js — Interp: 7 unch. · avg -1.1% · Bytecode: 🔴 2, 5 unch. · avg -4.6%
generators.js — Interp: 🟢 1, 3 unch. · avg +0.6% · Bytecode: 🟢 1, 3 unch. · avg +4.5%
intl.js — Interp: 🔴 2, 4 unch. · avg -0.3% · Bytecode: 🔴 1, 5 unch. · avg -3.1%
iterators.js — Interp: 🔴 8, 34 unch. · avg -4.0% · Bytecode: 🟢 24, 18 unch. · avg +7.3%
json.js — Interp: 🟢 1, 🔴 2, 20 unch. · avg -0.0% · Bytecode: 🔴 4, 19 unch. · avg -2.7%
jsx.jsx — Interp: 🔴 3, 18 unch. · avg -1.7% · Bytecode: 🟢 1, 🔴 2, 18 unch. · avg -2.2%
modules.js — Interp: 9 unch. · avg -1.3% · Bytecode: 9 unch. · avg +0.5%
numbers.js — Interp: 🔴 1, 11 unch. · avg -3.4% · Bytecode: 🔴 2, 10 unch. · avg -0.6%
objects.js — Interp: 7 unch. · avg -0.5% · Bytecode: 🔴 1, 6 unch. · avg +0.7%
promises.js — Interp: 🟢 5, 7 unch. · avg +5.1% · Bytecode: 🟢 1, 11 unch. · avg +1.1%
property-access.js — Interp: 🔴 2, 3 unch. · avg -4.8% · Bytecode: 5 unch. · avg +1.3%
regexp.js — Interp: 🔴 4, 9 unch. · avg -3.8% · Bytecode: 🟢 2, 11 unch. · avg +3.1%
strings.js — Interp: 🟢 2, 🔴 1, 16 unch. · avg +0.4% · Bytecode: 🟢 1, 18 unch. · avg +0.4%
temporal.js — Interp: 🟢 3, 3 unch. · avg +3.8% · Bytecode: 🟢 1, 5 unch. · avg +1.3%
tsv.js — Interp: 🟢 1, 8 unch. · avg +1.8% · Bytecode: 🟢 1, 🔴 2, 6 unch. · avg -2.2%
typed-arrays.js — Interp: 🟢 11, 🔴 2, 9 unch. · avg +7.7% · Bytecode: 🟢 5, 🔴 10, 7 unch. · avg -8.6%
uint8array-encoding.js — Interp: 🟢 7, 🔴 3, 8 unch. · avg +9.6% · Bytecode: 🟢 4, 🔴 4, 10 unch. · avg -6.5%
weak-collections.js — Interp: 🟢 2, 13 unch. · avg +4.1% · Bytecode: 🟢 5, 🔴 3, 7 unch. · avg +21.3%
Deterministic profile diffDeterministic profile diff: no significant changes. Measured on ubuntu-latest x64. Each PR run also builds the |
Contributor
JetStream 3 Performance Barometer
Geomean reference ratio: QuickJS 26.06×; Node.js 299.30×. 1.00× means aligned; values above 1.00× mean Goccia was proportionally slower after normalizing JetStream’s higher-is-better score. This is a directional barometer across runtimes with different goals, not a product ranking. Raw samples and failure details remain in the |
Contributor
AWFY Results
Geomean Ratios
14 pinned AWFY benchmarks. Medians from 5 interleaved samples per engine; raw JSON includes min/max/CV and is attached as the |
Contributor
test262 Conformance
Areas closest to 100%
Per-test deltas (+0 / -0 / timeout +2 / -2)New timeouts (2):
Resolved timeouts (2):
Steady-state failures and timeouts are non-blocking; PASS → non-timeout failure transitions fail the conformance gate. Measured on ubuntu-latest x64, bytecode mode. Areas grouped by the first two test262 path components; minimum 25 attempted tests, areas already at 100% excluded. Δ vs main compares against the most recent cached |
frostney
marked this pull request as ready for review
July 21, 2026 16:58
Owner
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
coderabbitai
Bot
added
documentation
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
scripts/run_es_toolkit_validation.py (1)
57-67: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winPath-traversal already mitigated; consider
filter="data"for defense-in-depth.Static analysis flags
archive.extractall(destination)as a Zip-Slip risk, but the preceding loop already resolves every member path, checksos.path.commonpathagainst the destination root, and rejects anything that isn't a plain file/dir (blocking symlink-based escapes too) — so this is already safe. Passingfilter="data"would make that guarantee explicit and align with Python's evolvingtarfileextraction-filter defaults.🛡️ Optional hardening
- archive.extractall(destination) + archive.extractall(destination, filter="data")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/run_es_toolkit_validation.py` around lines 57 - 67, Update the archive extraction call in extract_verified_package to pass tarfile’s data filter, while preserving the existing path validation and file/directory entry checks.Source: Linters/SAST tools
.github/workflows/ci.yml (1)
1358-1370: 🧹 Nitpick | 🔵 TrivialConsider caching the pinned es-toolkit tarball download.
This step re-downloads and re-verifies the same pinned tarball on every CI run with no
actions/cache, and sincecligatesartifacts/release, a transient npm outage would block releases. This is consistent with howtoml-compliance/json5-compliancealready do live corpus fetches inside blocking jobs, so it's not a new pattern — but caching keyed on the pinned integrity value (shared with the identical step in.github/workflows/pr.yml) would cut redundant network round-trips and reduce that flakiness surface.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 1358 - 1370, Cache the pinned es-toolkit tarball used by Check pinned es-toolkit compatibility probes, using a key derived from its pinned integrity value and shared with the equivalent workflow step in pr.yml. Restore the cached tarball before running scripts/run_es_toolkit_validation.py, while preserving validation and report upload behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 1358-1370: Cache the pinned es-toolkit tarball used by Check
pinned es-toolkit compatibility probes, using a key derived from its pinned
integrity value and shared with the equivalent workflow step in pr.yml. Restore
the cached tarball before running scripts/run_es_toolkit_validation.py, while
preserving validation and report upload behavior.
In `@scripts/run_es_toolkit_validation.py`:
- Around line 57-67: Update the archive extraction call in
extract_verified_package to pass tarfile’s data filter, while preserving the
existing path validation and file/directory entry checks.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b1eb78f7-2473-4d4b-a8a5-5750971f4609
📒 Files selected for processing (14)
.github/workflows/ci.yml.github/workflows/pr.ymldocs/es-toolkit-validation.mddocs/testing.mdscripts/es_toolkit_harness/deburr-combining-marks.mjsscripts/es_toolkit_harness/empty-unzip-with.mjsscripts/es_toolkit_harness/environment.mjsscripts/es_toolkit_harness/invariant-true.mjsscripts/es_toolkit_harness/large-array-sorted-index.mjsscripts/es_toolkit_harness/manifest.jsonscripts/es_toolkit_harness/ordinal-words.mjsscripts/run_es_toolkit_validation.pysource/units/Goccia.Builtins.Math.pastests/built-ins/Math/ceil.js
Reuse the integrity-pinned package tarball across PR and main validation while retaining integrity checks and hardened extraction.
Owner
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/run_es_toolkit_validation.py`:
- Around line 84-85: Validate manifest["upstream"]["tarball"] before the urlopen
call in the validation flow: require an HTTPS URL whose host is
registry.npmjs.org, and reject any redirect that changes the scheme or host
before reading the response. Keep the existing download and integrity-check
behavior for approved URLs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2dd22979-f8b1-4dc6-92cb-8db7a4ace7b6
📒 Files selected for processing (4)
.github/workflows/ci.yml.github/workflows/pr.ymldocs/es-toolkit-validation.mdscripts/run_es_toolkit_validation.py
🚧 Files skipped from review as they are similar to previous changes (2)
- .github/workflows/pr.yml
- docs/es-toolkit-validation.md
Validate the initial tarball URL and every redirect before opening it, allowing only HTTPS requests to registry.npmjs.org.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
coderabbitai
Bot
removed
the
spec compliance
coderabbitai
Bot
added
bug
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
scripts/test-es-toolkit-validation.ts (1)
107-151: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for the remaining classification branches.
Current cases cover matching pass/fail, known divergence, and timeout, but not: (1) only one mode producing a marker (→
bytecode-divergenceper run_es_toolkit_validation.ts L368-370), and (2) mismatchedmarker.id(→harness-failureper L371-373). A redirect-limit-exceeded case fordownloadTarballwould also round out coverage.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/test-es-toolkit-validation.ts` around lines 107 - 151, Extend the classification assertions in the test block around classifySemanticProbe to cover a run set where only one mode has a marker and assert bytecode-divergence, plus a run set with mismatched marker.id values and assert harness-failure. Also add downloadTarball coverage for redirect-limit-exceeded, asserting the expected failure behavior from that path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@scripts/test-es-toolkit-validation.ts`:
- Around line 107-151: Extend the classification assertions in the test block
around classifySemanticProbe to cover a run set where only one mode has a marker
and assert bytecode-divergence, plus a run set with mismatched marker.id values
and assert harness-failure. Also add downloadTarball coverage for
redirect-limit-exceeded, asserting the expected failure behavior from that path.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 999fcbc3-7644-455c-b479-105d0269df31
📒 Files selected for processing (5)
.github/workflows/ci.yml.github/workflows/pr.ymldocs/es-toolkit-validation.mdscripts/run_es_toolkit_validation.tsscripts/test-es-toolkit-validation.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- .github/workflows/ci.yml
- docs/es-toolkit-validation.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Math.ceilso non-integral Number values above the signed 32-bit boundary stay in binary64 instead of narrowing through the FreePascal integer result. The regression covers the confirmedMath.ceil(4294967295 / 2)boundary and its negative counterpart.c8d3c367dd509f5d8f137c6b623a30926fecd1f4and npm SHA-512 integrity. Downloads are restricted to HTTPS onregistry.npmjs.org, including redirects. It reports exact named probes and does not reuse or publish unreproduced whole-suite percentages.ReferenceErrorpattern.Buffer,Blob, orprocessstubs; no unsafe Function-constructor flag.Functionremains disabled by default and host globals are observations rather than ECMAScript conformance failures.0248456c758431e4bb8e5d26333ff1865123c9cd.Testing
./build/GocciaTestRunner tests: 1,442 files, 11,520/11,520 tests, 25,745 assertions./build/GocciaTestRunner tests --mode=bytecode: 1,442 files, 11,520/11,520 tests, 25,745 assertions2147483648in interpreter and bytecode modesbun run scripts/test-es-toolkit-validation.tsbun run scripts/check-test-structure.tsactionlint -shellcheck= .github/workflows/pr.yml .github/workflows/ci.yml./format.pas --check