@ioai/hdf5

Performance-oriented HDF5 access for browsers (Web Workers), Node.js, and future native targets. Architecture:

• WASM / HDF5 C core — HDF5 semantics, filters, chunk cache (when built with HDF5).
• TypeScript API — async, typed, batch-oriented calls to minimize JS/WASM boundary crossings.
• JS IO layer — ArrayBuffer, Blob/File, Node fs, and experimental HTTP Range + LRU block cache.

The default entry is Vite/bundler-first: Emscripten glue and the .wasm binary sit under dist/wasm/ and are referenced from the published JS so bundlers (Vite, etc.) own the asset URLs.

Install

npm install @ioai/hdf5

Quick start (Vite / bundlers — recommended)

import hdf5, { initHdf5 } from '@ioai/hdf5';

await initHdf5();
hdf5.FS.mkdirTree('/work');
hdf5.FS.writeFile('/work/file.h5', uint8Array);
const file = new hdf5.File('/work/file.h5', 'r');
try {
  console.log(file.keys());
} finally {
  file.close();
}

hdf5.ready remains available for the same lazy default initialization:

await hdf5.ready;

npm run build in this package requires a prior WASM build under dist/wasm/ (see Building WASM). The build copies glue into src/wasm/ for TypeScript, runs tsc, then restores the untouched Emscripten glue into dist/wasm/.

Loading modes

Default SPA / bundler

For normal Vite/Webpack/Rollup apps, call initHdf5() with no options. The packaged Emscripten glue is imported from the JS bundle and the sibling ioai_hdf5.wasm URL is resolved by the bundler/runtime asset graph.

import { initHdf5 } from '@ioai/hdf5';

const runtime = await initHdf5();

Explicit static asset URLs

For tests or non-Vite hosts, load glue/wasm from arbitrary URLs:

import { initHdf5 } from '@ioai/hdf5';

const runtime = await initHdf5({
  wasmJsUrl: new URL('./path/to/ioai_hdf5.js', import.meta.url),
  wasmBinaryUrl: new URL('./path/to/ioai_hdf5.wasm', import.meta.url),
});

You can also host both files from a common directory:

const runtime = await initHdf5({
  wasmJsUrl: '/assets/ioai_hdf5.js',
  assetBaseUrl: '/assets/',
});

Inline/blob workers

Inline workers run from a blob: URL, so they should not resolve the .wasm relative to import.meta.url. Fetch the wasm bytes from the parent realm (or otherwise provide bytes) and pass them into the worker:

// main thread
import wasmUrl from '@ioai/hdf5/wasm/ioai_hdf5.wasm?url';

const wasmBinary = await fetch(wasmUrl).then((r) => r.arrayBuffer());
worker.postMessage({ wasmBinary }, [wasmBinary]);

// worker
import { initHdf5 } from '@ioai/hdf5';

const runtime = await initHdf5({ wasmBinary });

Building WASM

Production (default for CI/npm): Docker + HDF Group official HDF5 source + official zlib — see scripts/README-build.md (npm run build:wasm:full:docker). Default build enables gzip/DEFLATE; SZIP/AEC is off unless you maintain a separate build profile.

Stub (no HDF5 C library): npm run build:wasm:stub:docker — produces runnable glue for CI/typecheck; npm run smoke:wasm skips HDF5 fixture checks unless built with IOAI_HDF5_HAVE_LIB.

From scratch: Emscripten, CMake, and your own HDF5-for-Emscripten tree — see native/README.md and scripts/README-build.md.

Publishing (maintainers)

1. Bump version in package.json and merge to main.
2. Create an annotated tag that matches that version, e.g. git tag -a v0.1.3 -m "Release 0.1.3" && git push origin v0.1.3.
3. GitHub Actions Publish npm runs the same Docker WASM + tests as CI, then runs npm publish.
   • Token in repository secrets: add NPM_TOKEN (npm automation or granular publish token for @ioai).
   • Token in an Environment secret: create an environment (e.g. npm) under Settings → Environments, add secret NPM_TOKEN, and ensure Deployment branches and tags allows your release tags (e.g. v*.*.*). The workflow sets jobs.publish.environment: npm — change that value if your environment uses a different name.

License

MIT — see LICENSE. Third-party components are listed in THIRD_PARTY_NOTICES.md.