Conversation
Snapshot of the GHCJS web IDE work before starting the typecheck-cache
investigation. Includes:
- Solcore.Api: in-memory compileSolcore entry point (source in, hull/yul out)
- Frontend.Module.Loader: SourceFS abstraction + loadModuleGraphFromSource
(virtual in-memory filesystem, embedded std bundle)
- Std.{Bundle,Embed}, Util.Keccak: TH-embedded std sources, pure Keccak-256
- yule -> library module reorg (Language.Hull.*, Language.Yul.Builtins)
- web/: React IDE (index.html/ide.jsx), simple.html, GHCJS FFI (Main.hs),
worker.js, build.sh, vendored react
Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Keccak.hs and MastEval.hs used foldl' unqualified, which only compiles under the GHC 9.10 JS backend (base 4.20 re-exports foldl' from Prelude); the native GHC 9.8 toolchain needs it imported. Import Data.List qualified and use L.foldl' at the call sites so it builds under both toolchains without the redundant-import warning that an explicit (foldl') import would trigger on 9.10 (no-warnings policy). Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Use pkgs.haskell.packages.ghc910 (was ghc98) in the native flake so the native compiler matches the GHCJS cross-compiler (also 9.10). One base version across both backends removes version-divergence bugs (e.g. foldl' being in Prelude on 9.10 but not 9.8) and lets precompiled typecheck-cache dumps round-trip between the native producer and the JS consumer. No dependency jailbreak needed: the cabal file uses only open lower bounds (no caret/upper caps), so the set resolves on 9.10 (base 4.20) as-is. Validated: nix build .#sol-core (library, exes, and unit tests) passes on native 9.10 with deps prebuilt in the binary cache; the devShell (incl. hevm/HLS) evaluates; and the typecheck-cache PoC runs identically to 9.8. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
The branch is already on the 9.10 native flake; bring it to the same clean footing as the move-to-ghc910 PR: - drop redundant `import Data.List (foldl')` (Prelude re-exports foldl' on base 4.20): TcModule, FieldAccess - remove the now-redundant genStmt catch-all in Language.Hull.ToYul.Translate — the SComment equation made the match exhaustive, so the `error` fallback tripped -Woverlapping-patterns under -Werror - reformat with ormolu 0.8.0.2 (shipped by the 9.10 toolchain) nix build .#tests-no-warnings (-Werror) and the ormolu check both pass. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the validated typecheck-cache library onto the ghcjs branch: - Solcore.Pipeline.TypecheckCache: Merkle content-addressed key per module (source + reference keys + typecheck-affecting flags), with precise invalidation and the moduleHasContracts -g refinement - Solcore.Pipeline.TcCacheSerialize: binary (de)serialization of a cached module's minimal payload (ModuleId, typed CompUnit, typeTable), behind a magic+version header guard - SolcorePipeline.compileGraphWithCache: reuse supplied CheckedModules instead of re-typechecking, returning the full checked set for reseeding - sol-core.cabal: add binary dep, expose the two modules Tier 1 wiring in Solcore.Api: a session-level IORef cache keyed by TcCacheKey. compileSolcore seeds each compile from modules whose current key is already cached and writes freshly-checked modules back, so the edit-recompile loop reuses unchanged std modules within a worker session. No FFI/worker changes needed. Cross-session persistence and the embedded std dump are later tiers. Builds on native GHC 9.10; ormolu clean. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist the std-library subset of the typecheck cache across page reloads, so the first compile of a session can reuse std rather than retypecheck it. - Solcore.Api: pure dumpStdCacheBlob / loadStdCacheBlob (filter to StdLibrary, encode/decode behind the magic+version header so a stale or foreign blob is a clean miss). indexCheckedByKey extracted and shared with the session-cache writer. IO wrappers dumpStdCache / loadStdCache marshal the blob as a Latin-1 string (one byte per BMP char) for the JS FFI. - web/Main.hs: register globalThis.solcoreDumpStdCache / solcoreLoadStdCache. - web/worker.js: seed the session cache from IndexedDB before signalling ready, and write the std dump back after the first successful compile. Every failure path (no IndexedDB, corrupt/foreign blob) degrades to a clean recompute. - web/build.sh: tolerate the absent all.js.gz on dev builds under pipefail. - test/TcCacheTests: dump -> load -> recompile reproduces cold output byte-for-byte, exercising the binary round-trip and the fromCachedModule error-thunk restoration in the live pipeline. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Warm even the browser's first-ever compile (before IndexedDB is populated) by shipping the std typecheck-cache dump as web/site/std-cache.bin. - web/gen-std-cache.js: drives the freshly-built JS compiler under Node to dump the std cache and write its raw bytes. Using the same JS build that consumes the blob guarantees the content-hash keys match at runtime. - web/build.sh: generate std-cache.bin after all.js is deployed, and precompress it on release builds. - web/worker.js: when the IndexedDB store is empty, fetch std-cache.bin and load it (iso-8859-1 decodes the Latin-1 byte stream back to the marshalling string) before signalling ready. Absent asset / offline degrades to a cold first compile. Subsequent sessions still come from IndexedDB. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Under Node the loader's import resolution dispatches an async fs call, which a synchronous GHCJS callback cannot wait on, so compileSolcore intermittently returns null and the release build failed dereferencing it. The blocked thread continues on the event loop, so retry on the next tick until a call returns a result. The browser has no async fs, so this path never triggers there. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Driving the browser compiler under Node to dump the cache was non-deterministic: compileSolcore is a synchronous GHCJS callback, but under Node the compile touches async fs, which a synchronous callback cannot wait on — so it intermittently returned null / left the cache unpopulated, and the release build failed (or not) as a coin flip. Generate the blob with a small native gen-std-cache executable instead. Native fs is synchronous, so it is deterministic; and the blob is byte-identical to one the JS build would dump (verified by SHA256), because the content-hash keys and serialized AST are toolchain-independent. build.sh now runs it via the default (native) cabal project; web/gen-std-cache.js is removed. flake ormolu check extended to cover the new source dir. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface, for each module (in dependency order), whether its typecheck was reused from the session cache or recomputed — so cache behaviour is observable in the IDE instead of inferred from a stopwatch. - Api.hs: CompileResult gains compileCacheStatus [(moduleDisplay, reused)], computed from the seed (a module is a hit iff it was seeded from the cache). - web/Main.hs: render it as "std: cache hit; std.dispatch: cache miss; ..." and pass it through the js_result object. - worker.js: forward cache in the result message. - ide.jsx: show it next to the compile-time status. - test: an identical recompile reports every module as a cache hit. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
mbenke
marked this pull request as draft
July 2, 2026 18:16
Generate a JSON ABI for every entry-module contract by default and add it to the IDE's artifacts, reusing the CLI's contractAbiJson so both paths agree. Api.CompileResult carries the ABIs (entryContractAbis), Main.hs marshals them to the worker, and InMemoryApiTests covers both the initial compile and the cache-hit recompile path. Rearrange the workspace tree to be Remix-like: an editable contracts/ directory, a read-only std/ directory (std, opcodes, dispatch), and an artifacts/ directory that accumulates hull, yul and one .abi per contract. Collapse the separate hull/yul panes into a single output pane showing the compile outcome (success/error). Also stop a stale per-origin IndexedDB std blob from shadowing the freshly bundled std-cache.bin: seed the session cache from both the bundled blob and any persisted dump (both content-addressed, so unioning is safe), which fixes every std module reporting a cache miss on the first local compile. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
A `cabal update` (or any resolve run without the JS toolchain in scope) can rewrite dist-ghcjs/cache/plan.json with the native ghc's package-ids; the JS compiler then fails later with a cryptic `cannot satisfy -package-id base-...`. Before building, compare the base package-id baked into the cached plan against the one this JS toolchain actually provides and wipe the builddir on a mismatch, so recovery no longer needs an index-state pin (deps stay in the global store, so only the local packages recompile). Also name the file that triggered the existing metadata-drift clean. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the GHCJS branch up to current main: native move to GHC 9.10, the ormolu 0.8.0.2 reformat, yule moved into the sol-core library, and — most consequential for this branch — main's source-location diagnostics (NodeLocation/SourceSpan carried through the AST, structured CompileDiagnostics threaded through the pipeline). Non-trivial conflict reconciliations: - SolcorePipeline: main wrapped compilation in compileWithDiagnostics (ExceptT CompileDiagnostics) while this branch split it into a cache-aware compileGraphWithCache for the browser. Unified them: compileGraphWithCache is now the shared core in main's diagnostic style, taking the graph as a parameter, threading `sources = moduleSourceMap graph`, and returning the checked-module map for cache seeding; compile/compileWithDiagnostics/ compileGraph delegate to it. Browser compiles now get source-mapped diagnostics too. - Module/Loader: combined this branch's virtual-filesystem abstraction (SourceFS/fsReadFile/firstExisting) with main's path-aware parsing and structured module-reference diagnostics, so in-memory (browser) loads keep working and produce real source maps. - TcCacheSerialize: main added optional source spans to Name and NodeLocation, which their Eq/Ord ignore. Serialize both span-free (manual Binary instances) so the content-addressed typecheck-cache blob stays toolchain-independent — the native gen-std-cache blob must remain byte-identical to what the browser would dump, independent of build paths. - Api / TcCacheTests / gen-std-cache: flatten CompileDiagnostics via compileDiagnosticsText at the String error boundaries. - Reformatted TcResolution.hs, which main HEAD left un-ormolu'd after a post-reformat edit. Verified: 802 native tests pass, native -Werror build clean, flake ormolu check clean, and the full GHCJS web build (all.js, ide.js, regenerated std-cache.bin) is green. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings in main's typechecker performance work (PR #521) on top of the earlier main merge: * The double typecheck is gone: modules are inferred once, and the lambda/expected-type reconciliation the retired no-desugar pass used to provide is now folded into tcExpWithExpected' (TcStmt). The CheckedModule record loses its checkedModuleNoDesugar field and Options loses noDesugarOpt. * Assorted TcSubst/TcUnify/TcMonad/TcResolution speedups and a new Syntax/Traversal module. Auto-merged cleanly; one hand-fix was required, folded in here: * TcCacheSerialize.fromCachedModule constructs a CheckedModule and set the now-removed checkedModuleNoDesugar field to an error thunk. With the field gone the record no longer compiles, so the line is dropped (only checkedModuleInput remains an unread thunk). The serialized cache format is unaffected: that field was never persisted. Verified: native -Werror build clean; 811 unit tests pass (including the typecheck-cache dump/load round-trip); optimised web/build.sh --release green with a regenerated std-cache.bin. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose javascript-unknown-ghcjs-ghc (GHC 9.10 retargeted at ghcjs, via the same override the sibling ghcjs flake uses) in the default dev shell, so web/build.sh runs alongside the native tools (solc, foundry, evmone) from a single `nix develop` instead of a separate ghcjs shell. Sharing GHC 9.10 with the native compiler keeps base/version in step, which the browser build's precompiled typecheck-cache dumps rely on to round-trip. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
web/build.sh now assembles web/site/solcore-node.cjs (web/node-driver.cjs prepended to the GHCJS all.js), runnable as: node web/site/solcore-node.cjs <file.solc> [iterations] It drives the same globalThis.compileSolcore the browser worker uses, warming the std typecheck cache from std-cache.bin first, so the measured path matches a warm in-browser compile. With an iterations argument it reports min/median/mean compile time. A test tool, not for production: the driver stubs the RTS's exit-on-main-return and polls for the registered global, because all.js only runs correctly as a genuine top-level module. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
The JS cross-compiler now has an in-repo home (ghcjs-flake.nix), so the missing-toolchain hint suggests swapping it in rather than only the sibling ../ghcjs shell. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
…n null Run straight in a terminal, the compiler's stdout diagnostics (e.g. "Emitting hull for contract ...") made compileSolcore return null: with isTTY true, GHCJS writes fd 1/2 through the async process.stdout.write path, which back-pressures on a terminal and suspends the Haskell thread — so the synchronous callback never yields its result. Piped/redirected output already took the blocking fs.writeSync path and worked. Mask isTTY so the sync path is always used. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Core Solidity playground PoC with compiler compiled via GHCJS and React UI