Skip to content

README-build.md

Latest commit

 

History

History
75 lines (46 loc) · 3.53 KB

README-build.md

File metadata and controls

75 lines (46 loc) · 3.53 KB
Raw

Building @ioai/hdf5 WASM

This package is scripts-first: CI and local dev can run TypeScript tests without a WASM binary. Production use requires building the Emscripten module.

Prerequisites (custom / non-Docker builds)

  • Emscripten (emcc, emcmake, emmake)
  • CMake ≥ 3.18 (HDF5 1.14.x)
  • Official HDF5 source and official zlib if you build HDF5 with compression support.

Production WASM (CI / npm default)

Production builds compile HDF5 from the HDF Group official source tarball inside the Emscripten Docker image, link official zlib (madler/zlib) for DEFLATE/gzip filters, and do not enable SZIP/AEC (HDF5_ENABLE_SZIP_SUPPORT=OFF).

Pinned URLs and SHA-256 checksums:

  • scripts/hdf5-source-pin.env
  • scripts/fetch-hdf5-source.sh — downloads into .cache/sources/ (gitignored under .cache/)

Emscripten toolchain version is pinned in hdf5-source-pin.env (EMSCRIPTEN_DOCKER_IMAGE_DEFAULT). A small upstream compatibility patch is applied to HDF5 sources before configure (see scripts/patches/hdf5-1.14.6-H5Tinit_float-fe-invalid.patch — wraps FE_INVALID use for environments where that macro is absent).

Build steps are orchestrated by:

  • scripts/build-full-wasm-docker.sh — host runs fetch-hdf5-source.sh, then Docker runs scripts/docker-emcc-full.sh (zlib static install → HDF5 static install → emcc link native/src/ioai_h5.cpp).

From the repository root:

npm run fetch:hdf5-source          # optional; full docker script runs it anyway
npm run build:wasm:full:docker   # official HDF5 + zlib + WASM
npm run build                      # TypeScript
# or:
npm run build:all:docker

After each emcc output, dist/wasm/package.json is written with { "type": "commonjs" } so Node require() can load the Emscripten glue under the package’s root "type":"module".

Host requirements: Docker and curl. CI caches .cache/sources and .cache/build-wasm.

Verification:

npm run smoke:wasm

This does not compile WASM again: it loads the existing dist/wasm/ioai_hdf5.{js,wasm} in Node and opens the real fixtures under test/fixtures/ (minimal.h5, minimal_deflate.h5) via Emscripten MEMFS — a quick integration check that the production artifact is usable (HDF5 + zlib/DEFLATE path).

Unit tests (Vitest, TypeScript-only) live under test/*.test.ts and run with npm run test.

Stub WASM (optional)

If you only need to validate the JS loader, compile ioai_h5.cpp with IOAI_HDF5_HAVE_LIB=OFF using emcc to produce a tiny module that returns NOT_BUILT errors from ioai_h5_open.

Docker (stub)

npm run build:wasm:stub:docker

The default Emscripten image is read from scripts/hdf5-source-pin.env. Override if needed:

EMSCRIPTEN_DOCKER_IMAGE=emscripten/emsdk:4.0.23 npm run build:wasm:stub:docker

npm release (GitHub Actions)

Workflow: .github/workflows/publish.yml. Configure repository secret NPM_TOKEN (automation token with publish rights for @ioai).

  • Tag push: push a SemVer tag v<version> (e.g. v0.1.0) that matches version in package.json.

Limitations (default production build)

  • Supported: DEFLATE / gzip-compressed datasets (via linked zlib).
  • Not enabled: SZIP / AEC filters (HDF5_ENABLE_SZIP_SUPPORT=OFF). Files that require those filters will fail at read time unless you maintain a separate audited build.