prepare npm 1.1.0 package contract#22
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83169ed221
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const selectedPath = requestedPath ?? bundledPath; | ||
| const workspacePath = path.resolve(selectedPath); | ||
| const workspaceFile = await stat(workspacePath).catch(() => null); | ||
| if (workspaceFile?.isFile()) { |
There was a problem hiding this comment.
Prefer package fixtures when no dataset is supplied
When an installed user runs a default benchmark such as recall bench temporal --read-only from a target repository that happens to contain evals/temporal/gold.v1.json (or the matching sufficiency/LoCoMo path), this workspace probe wins before the packaged fixture is considered, even though the user did not pass --dataset. That lets unrelated repo files change the supposedly fixture-backed benchmark result and dataset.ref, making the package contract non-deterministic; only explicit --dataset values should be resolved from the workspace before falling back to PACKAGE_ROOT.
Useful? React with 👍 / 👎.
What changed
Why
The previous consumer smoke mostly invoked source files directly. It could pass even when the npm artifact was missing a runtime file. Benchmark commands also looked for bundled fixtures in the user's repository instead of the installed package.
Impact
The 1.1.0 tarball now has 828 files, is about 1.21 MB compressed, and is about 5.21 MB unpacked. The previous artifact had 907 files, was about 1.46 MB compressed, and was about 6.57 MB unpacked. Runtime code, public docs, benchmark fixtures, and Rust source remain included.
Verification