feat: add zksync EraVM source verification and Abstract chain - #2788
feat: add zksync EraVM source verification and Abstract chain#2788coffee-the-dev wants to merge 54 commits into
Conversation
17d6e57 to
76ac6f8
Compare
66d406d to
01861fb
Compare
|
Happy to break this up into smaller PRs by area (compiler then lib then api etc) if that is easier for review/merging |
The function always returns false for `vm-1.5.0-a167aa3` regardless of the `target` argument. That only happens to produce correct behavior because every caller compares against 1.5.0; for any other target (e.g. 1.0.0), the helper would mis-classify that pre-release as below the threshold. Hardcoding 1.5.0 in the name and the body makes the contract honest: this is specifically an "is zksolc ≥ 1.5?" check, and the vm- guard is the correct answer to that specific question. Updates the inline explanatory comment to match. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Explain that the -gnu suffix on the Windows platform string is the upstream MinGW filename (not a libc choice), and that the Linux candidate list carries both glibc and musl filenames for fallback. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Document the zksolc / era-solc / upstream-solc compiler model: their naming and roles, the zksolc 1.5.0 CLI/output-selection split, the modern vs legacy release repos, and the Linux gnu/musl libc handling. Add a pointer comment at the top of zksolcCompiler.ts. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Covers logic that previously ran only behind network-dependent integration tests or not at all: the pre/post-1.5 standard-JSON CLI argument mapping, the isZkSolcVersionAtLeastV15 edge cases (including the unparseable-version fail-open), the primary/legacy download fallback in getZkSolcExecutable, and the era-solc vs upstream-solc routing in getZkSolcBaseSolcExecutable. Network seams (fetchWithBackoff, getSolcExecutable) are stubbed via module-namespace reassignment so the tests stay offline and fast. getZkSolcStandardJsonArgs and getZkSolcBaseSolcExecutable are exported to make them testable, consistent with the file's existing pattern of exporting internal helpers. zksolcCompiler.ts coverage: lines 76% -> 87%, functions 90% -> 100%. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a ZKSOLC.md section describing how Sourcify spawns zksolc and how zksolc in turn spawns a solc backend via --solc, plus the zkSync compiler-toolchain diagram. Because zksolc spawns the backend as a native child process, the Emscripten soljson build can never be used for EraVM verification. That made solJsonRepoPath dead weight: it was threaded through useZkSolcCompiler, getZkSolcBaseSolcExecutable and getUpstreamSolcExecutable but never used -- yet still load-bearing as a guard condition, so omitting it silently disabled upstream-solc resolution. Remove it from those functions, from ZkSolcLocal, and from both ZkSolcLocal call sites; the guard now checks only solcRepoPath. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a zksolc binary downloads (HTTP 200) but fails the --version validation (e.g. a gnu binary on a musl host), delete it instead of leaving it cached in the repo dir before falling through to the next candidate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the zksolc request checks (Solidity-only, settings require a zksolc compilerVersion, zksolc enabled on this server, chain supports zksolc) out of the async worker and into a validateZkSolcRequest middleware, so an unservable request returns an immediate 400 instead of a polled job error. This drops createInvalidParameterErrorExport and the apiErrorMessage type that was Pick-extracted from the Etherscan error variant; invalid_parameter job errors no longer carry a custom message. Adds VerificationService.isZkSolcEnabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Raw merge commit with conflict markers intentionally left in place for the 5 conflicting files. This captures the unresolved 3-way merge state so the subsequent resolution can be reviewed as a clean diff on top. Picks up the codecov orb bump 5.4.3 -> 6.0.0 (PR argotorg#2821) that fixes the failing '(Codecov) Validate CLI' GPG step in CI. Conflicted files still containing <<<<<<< / ======= / >>>>>>> markers: - packages/bytecode-utils/src/lib/bytecode.ts - packages/lib-sourcify/src/Verification/Transformations.ts - packages/lib-sourcify/src/Verification/Verification.ts - services/server/src/server/services/utils/database-util.ts - services/server/test/unit/utils/database-util.spec.ts Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tyle) Resolve the 5 conflicts from merging origin/staging into the zksolc PR, combining staging's refactors with the branch's ZKSYNC support: - bytecode-utils/bytecode.ts: use staging's getBytes() rename on the branch's zksync padding-stripped cborHex. - lib-sourcify/Transformations.ts: keep staging's sorted immutable-entries structure and Vyper length safeguard; fold ZKSYNC into the 'replace' paths (zksolc emits Solidity-format immutableReferences, so it replaces at offset). - lib-sourcify/Verification.ts: keep both the branch's ZKSYNC link-reference fallbacks and staging's additionalInput destructuring of jsonInput. - server/database-util.ts: adopt staging's compilation.additionalInput ?? null and drop the now-redundant getAdditionalInputFromVerification helper. - server/database-util.spec.ts: keep both tests (zksolc + Vyper immutables) under a single describe tree. Verified: bytecode-utils + lib-sourcify build, server tsc --noEmit, prettier, and the affected Transformations/database-util unit tests all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-run path Make zksolc (zksync EraVM) verification opt-in so custom Sourcify instances that don't verify EraVM contracts aren't required to run it. zksolc is enabled only when both `zksolcRepo` and `eraSolcRepo` are configured: - remove the zksolc repo paths from the default config (disabled by default) - keep them in the test config so the suite still exercises zksolc - resolve the paths via config.has() in cli.ts and only build ZkSolcLocal when configured; drop the unused top-level zksolc export Wire zksolc into the private API like solc/vyper/fe: - add an optional zksolc compiler to ServerOptions and app state - pass it through the /private stateless verification compilation Support zksolc in PreRunCompilation (reconstruction from stored DB output). A stored zksolc contract has language "Solidity" but targets EraVM, which has no deployedBytecode split and uses the ZKSYNC auxdata style. Detect it from the combined `zksolc:<v>;solc:<v>` version and mirror ZkSolcCompilation: - ZKSYNC auxdata style - runtime bytecode, immutable references and link references read from evm.bytecode - compilationExportMetadata exports compiler "zksolc" and the combined version - metadata restored via setMetadata() from the stored candidate - add PreRunCompilation zksolc unit tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zksolc verification was gated both by server config (compiler repos) and by a per-chain `zksolc.supported` flag. Drop the per-chain dependency so zksolc is purely a server capability: zksolc bytecode only matches EraVM chains, so a request on a non-EraVM chain simply fails to match instead of needing an explicit allowlist. - remove the chain `zksolc.supported` check from validateZkSolcRequest - remove the `zksolc` field from SourcifyChainExtension/SourcifyChain and its chain construction/serialization - update tests: the request is now rejected only when zksolc is disabled on the server (isZkSolcEnabled), not per chain Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… code zksolc request validation moved to the API middleware (validateZkSolcRequest), which throws the request-level InvalidParametersError. The verification-level "invalid_parameter" code left in VerificationErrorCode/getVerificationErrorMessage is no longer produced by any code path, so remove it; invalid parameters are reported with the existing request-level invalid_parameter error code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mpilation compiler identity Each Compilation maps 1:1 to a compiler, so the compiler name belongs on the compilation rather than being spread from an unknown-shape metadata object or inferred from the language (which is ambiguous: Solidity -> solc or zksolc). - remove compilationExportMetadata and the CompilationExportMetadata type - add abstract compilerName getter; SolidityCompilation -> solc, VyperCompilation -> vyper, FeCompilation -> fe, ZkSolcCompilation -> zksolc (YulCompilation inherits solc) - PreRunCompilation reconstructs any language, so it derives compilerName from the language (or zksolc when the stored version is a zksolc toolchain string) - add resolvedCompilerVersion getter: defaults to compilerVersion, overridden by zksolc to the combined zksolc:<v>;solc:<v> string (compilerVersion stays a plain semver for the Solidity heuristics in Verification) - Verification.export() sets compiler/compilerVersion explicitly - compiler is now required on VerificationExport; drop the unused zksolc export field (solcCompilerVersion is already encoded in the combined version) - move getCompilerNameFromLanguage into lib-sourcify; remove it from the server Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The key problem is that the user doesn't really control this at all, and is reliant on their tooling which seems to be broken all over the place. They can at most set solc version + zksolc version in configs, but the actual semantic "edition" is not controllable or even known by the developer/user at compile time. |
Split the interleaved zksolc logic out of the shared bytecode.ts into a
clean 3-file DAG:
- cbor.ts: generic CBOR-tail primitives + AuxdataStyle enum (no EraVM
knowledge). Holds splitCborAuxdata and the extracted
decodeSolidityCborObject that EraVM reuses.
- eravm.ts: all EraVM-specific logic — splitEraVmAuxdata, the new
decodeEraVmAuxdata (padding strip), and ERA_VM_* constants.
Depends only on cbor.ts.
- bytecode.ts: public facade — decode/splitAuxdata dispatch and the
Solidity/Vyper branches. Its only EraVM awareness is two thin
dispatch lines; it re-exports the CBOR public API so the
package's exported surface is unchanged.
Move the four EraVM test cases + fixtures into a dedicated eravm.spec.ts.
Update the stale bytecode.ts reference in ZKSOLC.md to eravm.ts.
No behavior change: all 22 bytecode-utils tests pass and lib-sourcify
consumers need no changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…p version retries
- Add ZkSolcVerification with EraVM creation matching via the versioned bytecode
hash (sha256 of the recompiled runtime bytecode, referenced in the
ContractDeployer create/create2 calldata). Reuses the canonical matchBytecodes
+ constructor-args transform; creationMatch inherits the runtime match type.
- Add eraBytecodeHash() to bytecode-utils (0x0100 | lenWords | sha256[-28:]).
- Gate the four solc-only steps in Verification (source-id <0.3.6, extra-file and
IR-ordering bugs, exact runtime-length) on the compiler (isSolidityViaSolc)
instead of the language, so zksolc (language 'Solidity', targets EraVM) is
excluded while Solidity and Yul stay unchanged.
- Remove the version-candidate retry/iteration from the verification flow
(Verification retry wrapper, AbstractCompilation.useNextCompilerVersionCandidate,
ZkSolcCompilation candidate iteration + compile retry). ZkSolcCompilation now
takes the concrete {zksolc, solc} versions; candidate resolution will be
reinstated at a higher (Etherscan) level.
- Instantiate ZkSolcVerification on-site in the worker (compilerName === 'zksolc').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ompilation It was defined in AbstractCompilation but only PreRunCompilation reconstructs a compiler name from a stored language; single-compiler compilations know their compiler directly. Colocate it with its only consumer. Still exported via the PreRunCompilation barrel, so the public API is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…inline The runCompiler override point wrapped only the main compile path and was a thin passthrough in every solc/vyper/fe compilation. Restore the base compileAndReturnCompilationTarget to call compiler.compile directly (as before the zksolc work) and let ZkSolcCompilation, which already fully overrides compileAndReturnCompilationTarget, invoke its two-version compiler.compile internally. Leaves the other compilations untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ata structurally Rework EraVM creation matching so it no longer borrows the Solidity [creationCode][args] prefix model, which doesn't fit EraVM: - Gate on the creation tx targeting the ContractDeployer system contract (0x…8006), so only direct deploys are matched. The check lives entirely in ZkSolcVerification; the base class only gains a generic protected creationTxTo field kept off the already-fetched creation tx. - Replace normalizeDeployerCalldata (positional slice + concat into a fake creation bytecode) with decodeContractDeployerCalldata, a single ABI decode against the selector's param types that yields the versioned bytecode hash and the constructor args directly. - Match by hash equality (decoded bytecodeHash vs eraBytecodeHash of the recompiled runtime bytecode), not a byte-prefix comparison. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zksolc support was removed from SourcifyChain / SourcifyChainInstance (zksolc is derived from the compilerVersion input, not the chain object), but this test still referenced sourcifyChain.zksolc and no longer compiled — a tsc error that was failing the whole SourcifyChain.spec.ts suite. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the compilerVersion/resolvedCompilerVersion duality: ZkSolcCompilation and PreRunCompilation now keep the combined `zksolc:<v>;solc:<v>` string as compilerVersion (overriding the base cleanCompilerVersion, which would strip the prefix) and parse the zksolc/solc versions internally. Remove the resolvedCompilerVersion getter from AbstractCompilation, ZkSolcCompilation, and PreRunCompilation; export() now reads compilerVersion. This also fixes the similarity re-verify path, which passed the plain zksolc semver (misdetected as plain Solidity) instead of the composite. Drop the makeZkSolcCompilation test helper and pass the composite string directly at all call sites, matching production and the PreRun test helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add end-to-end verification tests for native zkSync EraVM (zksolc) contracts, replaying 14 already-verified Abstract mainnet contracts against the public RPC (ZkSolcVerification.e2e.spec.ts + committed source fixtures). Coverage spans the keccak (<=1.5.12) and CBOR (>=1.5.13) metadata eras, legacy zksolc down to 1.3.8, explicit and resolved era-solc editions, constructor args, and perfect/partial runtime plus perfect/null creation matches. The suite self-skips when the RPC is unreachable or ZKSYNC_E2E=false. Also move the unit-level EraVM matching tests out of ZkSolcCompilation.spec.ts into ZkSolcVerification.spec.ts, and extract the shared zksolc mock helpers into test/utils/zksolcTestHelpers.ts. Add a real zksolc IZkSolcCompiler to test/utils. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…storage
Mark stored bytecode with the VM it targets so EraVM (zksolc) contracts are
handled distinctly from EVM ones:
- lib-sourcify: add a `Vm` type and a `compilation.vm` getter ('evm' by
default, 'eravm' for ZkSolcCompilation); expose it on VerificationExport.
- db: add a `vm` column to the `code` table (default 'evm'); `insertCode`
only writes it for non-EVM bytecode so the shared insert stays compatible
with the Verifier Alliance schema (which has no `vm` column).
- Skip EraVM contracts when storing to the Verifier Alliance database — they
are stored only in the Sourcify database.
- EtherscanVerify: submit EraVM contracts with the zksync fields
(compilermode=zksync, zksolcVersion, solc `compilerversion`); keep skipping
EraVM for Blockscout (separate API) and Routescan (unverified support).
Adds unit + integration coverage for the code `vm` tagging, the Alliance skip,
and the Etherscan zksync payload.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep EraVM (zksolc) contracts out of all external explorer verify backends for now (Etherscan/Blockscout/Routescan); they are stored only in the Sourcify DB. Etherscan does support EraVM verification (compilermode=zksync + zksolcVersion) and this can be enabled later — see PR notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… shape The verifyFromJsonInput/similarity deep-equal fixture was missing `compiler` and (after this PR) `vm` from the expected `compilation` object, failing all 7 assertVerificationExport cases. Add both keys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ed chain Adds an end-to-end zksolc/EraVM verification that runs the real worker (real zksolc + era-solc compilation) against a mocked Abstract chain, then asserts the full /v2/contract response. - MockRpcServer: a local JSON-RPC server replaying captured Abstract mainnet responses, so the verification worker (separate thread, can't be nock'd) reads deterministic on-chain data with no live RPC. - Fixture: DealersMulticall on Abstract (2741) — CBOR metadata, constructor args, perfect runtime + perfect creation match. input.json + captured eth_getCode/getTransactionByHash/getTransactionReceipt. - Asserts match statuses, compilation identity (compiler=zksolc, combined version), runtime/creation transformations (constructorArguments), deployment info, and that stored code is tagged vm='eravm'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
I'm getting a bit overwhelmed by it so I think I need someone else to look at this. I think it's nicely isolated from the normal verification logic. I'm just not too comfortable with the testing. I've mostly vibecoded it to import various cases from Etherscan so I'm not sure to what extent we are able to cover cases. Maybe we don't have to have a perfect e2e solution and we can keep fixing things along the way. I've left the "compiler iteration" completely outside the scope. As is, it would not be possible to import from Etherscan as the solc version will be incomplete. We either offload this to people importing from Etherscan or maybe have some iteration on the server side. Most of the added files and code are tests so don't get surprised by 92 files added. |
Summary
Adds ZKsync EraVM/zksolc source verification support to Sourcify, extending the standard JSON verification API surface.
Zksolc verification is selected by passing
zksolcVersionto the normal/v2/verify/{chainId}/{address}JSON-input flow The underlyingcompilerVersionremains the solc or era-solc version requested by the user, and Sourcify records the final era-solc/solc version used for the verified compilation.Changes by module
packages/compilers@ethereum-sourcify/compilers.zksolc-bin.--standard-jsonand--solc.settings.enableEraVMExtensions/settings.isSystemto--system-modesettings.forceEVMLA/settings.forceEvmlato--force-evmlaevmoutput for legacy zksolc versionsv0.8.26+commit.8a97fa7aby resolving Sourcify's normal native solc binaries and passing them to zksolc.packages/lib-sourcifyZkSolcCompilationfor EraVM Solidity compilations.IZkSolcCompilerand includes zksolc in the shared compilation type surface.Soliditybut marks the compilation target VM aseravm.0.8.26-1.0.10.8.26v0.8.26+commit.8a97fa7a1.0.2,1.0.1, and1.0.01.0.2for zksolc versions before 1.5VerificationExport:compiler: "zksolc"compilerVersion: zksolc versionzksolc.solcCompilerVersion: final underlying solc/era-solc version used1.5.xwith newer solc/era-solc behavior1.4.1 + 0.8.4-1.0.11.3.17 + 0.7.6-1.0.1services/serverzksolcVersionto the request body.ZkSolcJsonInputas a Solidity standard JSON superset for zksolc-specific settings.zksolcVersionand zksolc-specific settings.zksolcVersionis present when zksolc-specific settings are usedZkSolcLocaland passes zksolc/era-solc/solc repository paths through CLI and worker initialization.zksolcRepoanderaSolcRepo.compiler = "zksolc"version = <zksolcVersion>additional_input.era_solc_version = <final underlying solc/era-solc version>Chain configuration
zksolc.supported: true | falseSourcifyChainand generated chain config loading.2741https://api.mainnet.abs.xyzzksolc.supported: trueDatabase
additional_input.era_solc_versiononcompiled_contracts.services/database/database-specssubmodule.Verification behavior
For zksolc requests, supports both explicit era-solc compiler versions and normal Solidity compiler versions:
compilerVersionis an era-solc version, Sourcify uses that exact era-solc binary.compilerVersionis a commit-bearing Solidity release, for examplev0.8.26+commit.8a97fa7a, Sourcify first tries the exact upstream solc binary through zksolc.compilerVersionis a plain Solidity release, Sourcify expands it directly to compatible era-solc candidates.This is intended to support explorer-style submissions where users only know the Solidity compiler version while still allowing exact era-solc requests when that is known; this supports existing tooling and legacy verifications/compile artifacts that may not bear the exact solc fork edition (v1.0.x)
Testing
Automated coverage added or expanded:
packages/compilers/test/zksolcCompiler.spec.tspackages/lib-sourcify/test/Compilation/ZkSolcCompilation.spec.tspackages/lib-sourcify/test/SourcifyChain.spec.tsservices/server/test/integration/apiv2/verification/verify.json.spec.tsservices/server/test/unit/VerificationService.spec.tsservices/server/test/unit/utils/database-util.spec.tsservices/server/test/unit/verificationWorker.spec.tsManual verification performed against Abstract Mainnet contracts from abscan/Etherscan-compatible source metadata, including exact upstream solc fallback behavior for commit-bearing compiler versions.
Candidate contracts tested include:
https://abscan.org/address/0xbc176ac2373614f9858a118917d83b139bcb3f8c#code - zksolc 1.5.7, solc v0.8.26+commit.8a97fa7a (resolves to v0.8.26-1.0.1)
https://abscan.org/address/0x4f7589c619d59443db52489dd375de63e03e671d#code - zksolc v1.3.19, solc v0.6.12+commit.27d51765 (resolves to direct v0.6.12+commit.27d51765)
https://abscan.org/address/0x0929d81a73a83b73e5de2ba63a15ce2a18addbe2#code - zksolc v1.5.15, solc v0.8.26+commit.8a97fa7a (resolves to v0.8.26-1.0.2)