feat(loader): add EXT_lights_image_based support#396
Conversation
Add the glTF EXT_lights_image_based extension as a self-registering, lazy, side-effect-free GltfFeature module. It installs a document-level image-based light (irradiance SH9 + prefiltered specular mip cubemaps + intensity/rotation) referenced by the active scene and drives PBR IBL through Lite's existing environment path. Reuse (KISS / maximum reuse per GUIDANCE): - Specular mips are RGBD PNGs (same encoding as .env faces) -> shared uploadCubemapRGBD decoder. - BRDF split-sum LUT generated on-GPU via the shared generateBrdfLut. - Final EnvironmentTextures assembled with assembleEnvironmentTextures and wired onto the scene exactly like loadEnvironment (scene._envTextures + registerEnvSceneUniforms + tone-mapping defaults). - glTF irradianceCoefficients -> SphericalPolynomial conversion reproduces Babylon's SH -> intensity -> Lambertian (x1/pi) chain so the shared polynomialToPreScaledHarmonics yields matching coefficients. Wiring: a lazy AssetContainer._sceneSetup closure (invoked by addToScene) installs the environment, keeping the core loader/scene feature-agnostic. Fixes the Khronos "Environment Test" (glTF-IBL) model; parity scene264 renders at MAD 0.461 (limit 0.5), peer-consistent with the existing PBR-IBL scenes. NOTE: reusing the shared IBL modules (env-helpers, hdr-ibl-pipeline, samplers) makes them second consumers in every gltf scene's rollup graph, hoisting those previously-inlined leaves into standalone chunks (+0.5-1.1 KB per gltf scene). This needs a bundle-size ceiling decision — see PR description. No ceilings or golden references changed in this commit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
Give the EXT_lights_image_based feature a private RGBD cubemap uploader (ibl-cubemap-upload.ts) instead of importing uploadCubemapRGBD from the shared loader-env/rgbd-decode.js. rgbd-decode is a static dependency of the .env loader, so a second (extension) consumer pinned the otherwise tree-shaken cubemap path into every .env/DDS scene's bundle (~1 KB). The private copy is reachable only from this lazily-imported feature chunk, so non-IBL scenes pay zero bytes for it and only EXT_lights_image_based scenes fetch it. scene264 stays under its ceiling (80.1/82 KB); parity MAD 0.461. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Duplicate the shared IBL helpers (env-texture assembly, BRDF LUT, SH conversion, samplers, resolveImage, env scene-UBO wiring) into a private ibl-env-assembly.ts reachable only from the lazy EXT_lights_image_based feature chunk, and destroy the feature's env textures directly instead of via the gpu-pool refcount path. This stops the feature from becoming a second static consumer of single-consumer leaves, so rollup keeps the 43 non-IBL glTF scenes' inlined copies untouched (zero bundle delta) and the feature's byte cost lands only on the scene that uses it. Deliberate, user-approved exception to the max-reuse mandate; documented in both modules with canonical-source mapping. Bundle result: 209/211 scenes at/under ceiling; only scene41 and scene47 drift +0.1 KB (the irreducible per-scene gltf-feature-registry entry). scene264 (EnvironmentTest) at 79.7 KB, under its 82 KB ceiling. Parity unchanged (scene264 MAD 0.461). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…y cost The EXT_lights_image_based (PR5) lazy feature adds a ~90-byte entry to every glTF scene's gltf-feature-registry chunk. Three scenes sit at ~zero ceiling headroom, so the exact-byte bundle-size check tips over: - scene12 103.5 -> 103.6 (was over by 7 bytes) - scene41 93.2 -> 93.4 - scene47 92.5 -> 92.7 These are the final COMBINED ceilings covering BOTH the sparse (PR4 #394) and IBL (PR5) registry entries on the same zero-headroom scenes, so once both features coexist no further bump is needed. The ~90-byte registry entry is irreducible (any new lazy glTF feature requires it). scene264 stays under its existing 82 KB ceiling. No golden changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds glTF EXT_lights_image_based support to the Babylon Lite glTF loader as a lazily-loaded feature module, wiring the extension’s document-level IBL (irradiance SH + prefiltered specular cubemap mips) into Lite’s existing environment/scene-UBO path. This introduces a new parity scene (scene264) for the Khronos EnvironmentTest model and updates bundle-size ceilings/manifests to keep CI green.
Changes:
- Add a new lazy glTF feature module for
EXT_lights_image_based, including private (bundle-isolated) IBL helper copies and RGBD cubemap upload. - Add a deferred
AssetContainer._sceneSetuphook and invoke it duringaddToScene()so loader features can perform scene-specific wiring without coupling core code to extensions. - Add scene264 parity + lab entry points, plus scene-config ceiling updates and refreshed bundle manifests.
Reviewed changes
Copilot reviewed 67 out of 69 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/lite/parity/scenes/scene264-environment-test.spec.ts | Adds parity coverage for the new EnvironmentTest scene that exercises EXT_lights_image_based. |
| scene-config.json | Adds scene264 configuration and adjusts bundle-size ceilings for scenes 12/41/47. |
| packages/babylon-lite/src/scene/scene-core.ts | Invokes AssetContainer._sceneSetup during addToScene() when unpacking a container. |
| packages/babylon-lite/src/loader-gltf/ibl-env-assembly.ts | Introduces private IBL assembly helpers used only by the extension’s lazy chunk to avoid shared-chunk hoisting. |
| packages/babylon-lite/src/loader-gltf/ibl-cubemap-upload.ts | Adds a private RGBD cubemap decode/upload path for the extension’s specular mip cubemaps. |
| packages/babylon-lite/src/loader-gltf/gltf-feature-registry.ts | Registers the new EXT_lights_image_based trigger row for lazy loading. |
| packages/babylon-lite/src/loader-gltf/gltf-ext-lights-image-based.ts | Implements the EXT_lights_image_based feature: loads images, uploads cubemap, builds BRDF LUT, and defers scene wiring via _sceneSetup. |
| packages/babylon-lite/src/asset-container.ts | Adds internal _sceneSetup hook to AssetContainer for deferred scene wiring. |
| lab/public/bundle/manifest/scene99.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene73.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene7.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene5.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene47.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene41.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene39.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene37.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene35.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene34.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene33.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene32.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene31.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene30.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene29.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene28.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene27.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene264.json | Adds bundle manifest for the new scene264. |
| lab/public/bundle/manifest/scene260.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene257.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene255.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene254.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene253.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene251.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene247.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene246.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene245.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene244.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene243.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene242.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene241.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene240.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene219.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene218.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene212.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene211.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene210.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene178.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene176.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene164.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene158.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene157.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene152.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene149.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene148.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene147.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene144.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene12.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene115.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene112.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene11.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene105.json | Refreshes bundle manifest after registry/scene changes. |
| lab/public/bundle/manifest/scene104.json | Refreshes bundle manifest after registry/scene changes. |
| lab/lite/src/lite/scene264.ts | Adds Lite lab entry point for scene264 loading the glTF model that carries IBL via the extension. |
| lab/lite/src/bjs/scene264.ts | Adds Babylon.js reference scene entry point for parity/oracle capture. |
| lab/lite/scene264.html | Adds dev (non-bundled) lab HTML entry for scene264 (Lite). |
| lab/lite/bundle-scene264.html | Adds bundled lab HTML entry for scene264 (Lite). |
| lab/lite/bundle-bjs-scene264.html | Adds bundled lab HTML entry for scene264 (Babylon.js reference). |
| lab/lite/babylon-ref-scene264.html | Adds dev (non-bundled) Babylon.js reference HTML entry for scene264. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…n with master NPE scene264 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve conflicts from master's 12 new commits: - Scene ID collision: master added its own scene264 (NPE Particles: Change - Size). Renumbered our EnvironmentTest scene 264->265 (all lab files, spec, golden dir, manifest, thumbnail, scene-config entry). Kept master's scene264. - scene-config.json: kept both scene entries; preserved EXT_lights_image_based ceilings. - Bundle-size: master's new features consumed all per-scene headroom (scenes now sit exactly at ceiling). The feature's irreducible shared cost (~90-byte glTF registry entry + ~20-byte addToScene _sceneSetup hook) tips 20 zero- headroom scenes over by <=0.1 KB. Bumped those 20 ceilings by the exact measured cost (scene260 needed +0.2 for its 80.1025 KB exact size). All are exact-cost, not buffer. Regenerated 78 feature-affected per-scene manifests plus scene265; reverted machine-noise-only manifests to master. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
These three scenes' entry-file bytes shifted sub-KB after merging master, crossing a gzip rounding boundary (18.4->18.5, 28.4->28.5, 31.4->31.5 KB). Chunk content-hashes are unchanged; raw KB is effectively identical. Committing the fresh values so validate-bundle-manifest.ts matches a fresh build of the merged branch in CI. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…image-based # Conflicts: # lab/public/bundle/manifest/scene11.json # lab/public/bundle/manifest/scene152.json # lab/public/bundle/manifest/scene157.json # lab/public/bundle/manifest/scene176.json # lab/public/bundle/manifest/scene177.json # lab/public/bundle/manifest/scene212.json # lab/public/bundle/manifest/scene240.json # lab/public/bundle/manifest/scene242.json # lab/public/bundle/manifest/scene244.json # lab/public/bundle/manifest/scene246.json # lab/public/bundle/manifest/scene247.json # lab/public/bundle/manifest/scene253.json # lab/public/bundle/manifest/scene254.json # lab/public/bundle/manifest/scene255.json # lab/public/bundle/manifest/scene28.json # lab/public/bundle/manifest/scene29.json # lab/public/bundle/manifest/scene30.json # lab/public/bundle/manifest/scene32.json # lab/public/bundle/manifest/scene33.json # lab/public/bundle/manifest/scene34.json # lab/public/bundle/manifest/scene35.json # lab/public/bundle/manifest/scene39.json # lab/public/bundle/manifest/scene41.json # lab/public/bundle/manifest/scene48.json # lab/public/bundle/manifest/scene5.json # lab/public/bundle/manifest/scene53.json # lab/public/bundle/manifest/scene73.json
…feature Fresh build of the merged branch (master's managed-storage-buffer feat! + our EXT_lights_image_based registry entry) shifts 30 scene manifests: - gltf-feature-registry chunk hash changes on glTF scenes (our lazy entry) - raw/gzip rounding shifts on scenes touched by master's storage-buffer change Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bundle Size ChangesIncreases
Sizes rounded to nearest KB. Run |
Lab - Static SiteBuild 20260713.2 - merge @ 079a4de |
What this does
Adds the glTF
EXT_lights_image_basedextension as a self-registering, lazy, side‑effect‑freeGltfFeaturemodule (gltf-ext-lights-image-based.ts), registered by a single trigger row ingltf-feature-registry.ts.The extension installs a document‑level image‑based light — diffuse irradiance SH9 + a set of prefiltered specular mip cubemaps, plus intensity/rotation — referenced by the active scene, and drives PBR IBL through Lite's existing environment path. This fixes the Khronos “Environment Test” (glTF‑IBL) model (#2 in the cx20/gltf-test triage), which previously rendered wrong because the extension was ignored.
How it reuses the existing IBL path (KISS / maximum reuse)
No parallel environment representation was invented — the glTF light is mapped onto Lite's existing
EnvironmentTextures+ scene‑UBO SH machinery:.envfaces) → uploaded through an RGBD cubemap decoder.assembleEnvironmentTextureslogic and wired onto the scene exactly likeloadEnvironment(scene._envTextures+registerEnvSceneUniforms+ tone‑mapping defaults).irradianceCoefficients→SphericalPolynomialconversion reproduces Babylon'sSH → intensity → Lambertian (×1/π)chain, sopolynomialToPreScaledHarmonicsyields matching render coefficients.scene.envRotationY; LOD scale fit to the mip count exactly as Babylon does.Scene wiring uses a lazy
AssetContainer._sceneSetupclosure invoked byaddToScene(), keeping the core loader/scene fully feature‑agnostic (two tiny type‑only/one‑line core hooks).Parity
EnvironmentTest.gltf) renders at MAD 0.461 (limit 0.5), peer‑consistent with the existing PBR‑IBL scenes. Golden + manifest + thumbnail included.maxRawKB: 82ceiling.Bundle‑size — resolved (Option B, user‑approved)
This is the largest of the 5 gltf‑extension fixes and did pressure bundle size. It is now fully green (all 211 bundle‑size checks pass).
Root cause: the feature registry statically lists
() => import("./gltf-ext-lights-image-based.js"), so the extension's dependency graph participates in every gltf scene's rollup build. The shared IBL leaves (env-helpers,hdr-ibl-pipeline, samplers, rgbd decode,resolveImage, env scene‑UBO wiring) were previously inlined into single‑consumer scene chunks. The extension becoming a second static consumer forced rollup to hoist them into shared chunks that all 44 gltf scenes then paid for (~32 KB total).Fix — isolate the feature's cost to its own chunk (
ibl-env-assembly.ts+ibl-cubemap-upload.ts):device.createSamplerinstead ofgpu-pool.getOrCreateSampler.texture.destroy()) instead of via thegpu-poolrefcount path — un‑pinninggpu-poolfrom every gltf scene's main chunk.Result: the only recurring cost is the ~90‑byte
["EXT_lights_image_based", () => import(...)]registry entry that every gltf scene's registry chunk must carry — unavoidable for any new lazy gltf feature. See the post‑merge Update section below for the exact ceiling table after rebasing on the latestmaster.Files
packages/babylon-lite/src/loader-gltf/gltf-ext-lights-image-based.ts(new feature module)packages/babylon-lite/src/loader-gltf/ibl-env-assembly.ts(new — private IBL helper copies; documented exception)packages/babylon-lite/src/loader-gltf/ibl-cubemap-upload.ts(new — private RGBD cubemap uploader)packages/babylon-lite/src/loader-gltf/gltf-feature-registry.ts(one trigger row)packages/babylon-lite/src/asset-container.ts,scene/scene-core.ts(lazy_sceneSetuphook)scene-config.json(scene265 entry + post-merge ceiling bumps — see Update section)Update — merged latest
master(conflict resolution)masteradvanced 12 commits and independently added its ownscene264("NPE Particles: Change — Size"), colliding with this PR'sscene264(EnvironmentTest).Resolution:
264 → 265(all lab entry points, parity spec, golden dir, manifest, thumbnail). Master's NPEscene264is kept intact; ours is nowscene265.scene-config.jsonto keep both scenes and merged the ceiling tables.Bundle size after merge. Master's 12 commits consumed the remaining ceiling headroom on many scenes, so this PR's genuine +0.1 KB feature cost (the ~90‑byte lazy registry entry on every glTF scene + a ~20‑byte inert
_sceneSetuphook in the sharedaddToScene) now tips a set of previously zero‑headroom scenes over. Proven to be a real, exact +0.1 KB delta via an identical‑environment A/B build of this branch vsmaster(master sits exactly at its ceilings). User‑approved exact‑cost bumps (not buffers) were applied:No golden references were changed.
scene265(EnvironmentTest) sits at 80.0 KB / 82 KB ceiling.Three glTF scene manifests (
scene123,scene177,scene217) were refreshed: merging master shifted their scene‑entry bytes sub‑KB across a gzip rounding boundary (chunk content‑hashes unchanged), whichvalidate-bundle-manifest.tsflags.Validation after merge (
LAB_TEST_PORT=5203 REUSE_BROWSER=true):validate-bundle-manifest.ts: 214 scenes up to date against a fresh build of the merged branch.scene265renders correctly against the Babylon.js oracle (MAD 0.461 in a quiet run; ~0.52 under heavy concurrent‑session load). The broad parity noise observed locally affects scenes this PR never touches (e.g. the non‑glTFscene2‑sphere), i.e. live‑oracle flakiness under machine contention — not a regression from this PR, whose diff is 7 localized files.