Add native Draco and meshopt codec plugins - #1797
Open
bkaradzic-microsoft wants to merge 3 commits into
Open
Conversation
The WASM Draco decoder fails to instantiate in the full validation suite
(order-dependent LinkError after ~139 prior tests). Port the decoder to a
native C++ plugin per the project's no-WASM policy.
NativeDraco exposes _native.decodeDracoMesh(dataView, attributes?), a
synchronous decoder built on google/draco 1.5.7 (CMake FetchContent). It
emits the same { indices, attributes[], totalVertices } shape as the WASM
worker path (de-interleaved, tightly packed per-point values).
draco marks its include paths PRIVATE, so re-export them SYSTEM PUBLIC on
the combined target (draco/draco_static), including CMAKE_BINARY_DIR for the
generated draco/draco_features.h.
Un-excludes 7 Draco decode tests (202-206, 232, 233), all pass. Full
regression 373/373. Encoder-dependent tests (207, 217) stay WASM/excluded.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 60c2ec68-6de1-445d-9fc9-b699db737eae
Extends the NativeDraco plugin with _native.encodeDracoMesh(attributes,
indices, options), a synchronous native replacement for the WASM Draco
encoder. Draco is now fully native (decode + encode); no Draco test
depends on WASM.
Faithfully replicates draco's emscripten encoder wrapper: builds a
draco::Mesh (SetNumFaces/SetFace), adds attributes via a templated
AddAttributeToMesh<T> (PointAttribute::Init + per-point SetAttributeValue),
applies quantization/method/speed options, then DeduplicateAttributeValues
+ DeduplicatePointIds + Encoder::EncodeMeshToBuffer. Returns { data,
attributeIds }. The returned attribute id equals draco's unique_id
(PointCloud::SetAttribute -> set_unique_id), so the decoder's
GetAttributeByUniqueId round-trips.
Un-excludes idx 207 (Decoder/Encoder roundtrip) and 217 (glTF serializer
KHR draco), both pass. Full regression 375/375.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 60c2ec68-6de1-445d-9fc9-b699db737eae
Native replacement for the WASM meshopt decoder (zeux/meshoptimizer), exposing _native.decodeMeshopt(source, count, stride, mode, filter?). Mirrors the reference meshopt_decoder.js decode() helper: rounds count up to a multiple of 4, decodes via meshopt_decodeVertexBuffer/IndexBuffer/ IndexSequence, applies the optional Oct/Quat/Exp filter in-place, and returns the first count*stride bytes. FetchContent meshoptimizer v0.22. Un-excludes idx 234 (GLTF Buggy with Meshopt Compression). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 60c2ec68-6de1-445d-9fc9-b699db737eae
Contributor
There was a problem hiding this comment.
Pull request overview
Adds two new native codec plugins (NativeDraco, NativeMeshopt) to replace the current WASM-based Draco/meshopt mesh decoding path in Babylon Native, wiring them into the Embedding runtime and build system via new FetchContent dependencies.
Changes:
- Introduces C++ N-API implementations for
_native.decodeDracoMesh,_native.encodeDracoMesh, and_native.decodeMeshopt. - Adds CMake targets for the new plugins and wires them into
Embeddinginitialization. - Adds
dracoandmeshoptimizeras FetchContent dependencies and configures their builds for use as libraries.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| Plugins/NativeMeshopt/Source/NativeMeshopt.cpp | Implements native meshopt decode entry point exposed to JS. |
| Plugins/NativeMeshopt/Include/Babylon/Plugins/NativeMeshopt.h | Declares NativeMeshopt initialization API. |
| Plugins/NativeMeshopt/CMakeLists.txt | Builds/link-wires the NativeMeshopt plugin. |
| Plugins/NativeDraco/Source/NativeDraco.cpp | Implements native Draco decode + encode entry points exposed to JS. |
| Plugins/NativeDraco/Include/Babylon/Plugins/NativeDraco.h | Declares NativeDraco initialization API. |
| Plugins/NativeDraco/CMakeLists.txt | Builds/link-wires the NativeDraco plugin. |
| Plugins/CMakeLists.txt | Registers the new plugin subdirectories in the plugin build. |
| Embedding/Source/Runtime.cpp | Initializes the new plugins during runtime setup. |
| Embedding/CMakeLists.txt | Links the new plugins into the Embedding target. |
| Dependencies/CMakeLists.txt | Fetches/configures draco + meshoptimizer dependency targets. |
| CMakeLists.txt | Declares new FetchContent sources for draco and meshoptimizer. |
Comment on lines
+28
to
+46
| if (info.Length() < 4 || !info[0].IsTypedArray()) | ||
| { | ||
| throw Napi::TypeError::New(env, "Meshopt: decodeMeshopt(source, count, stride, mode, filter?) requires a source typed array."); | ||
| } | ||
|
|
||
| const auto sourceArray = info[0].As<Napi::TypedArray>(); | ||
| const auto* source = static_cast<const unsigned char*>(sourceArray.ArrayBuffer().Data()) + sourceArray.ByteOffset(); | ||
| const size_t sourceSize = sourceArray.ByteLength(); | ||
|
|
||
| const size_t count = static_cast<size_t>(info[1].As<Napi::Number>().Int64Value()); | ||
| const size_t stride = static_cast<size_t>(info[2].As<Napi::Number>().Int64Value()); | ||
| const std::string mode = info[3].As<Napi::String>().Utf8Value(); | ||
|
|
||
| // Round count up to a multiple of 4 (the reference decoder over-allocates | ||
| // and runs the vertex filter over count4 elements). | ||
| const size_t count4 = (count + 3) & ~static_cast<size_t>(3); | ||
|
|
||
| std::vector<unsigned char> temp(count4 * stride, 0); | ||
|
|
Comment on lines
+297
to
+305
| const auto attr = attributesIn.Get(i).As<Napi::Object>(); | ||
| if (attr.Get("dracoName").As<Napi::String>().Utf8Value() == "POSITION") | ||
| { | ||
| const auto data = attr.Get("data").As<Napi::TypedArray>(); | ||
| const int8_t size = static_cast<int8_t>(attr.Get("size").As<Napi::Number>().Int32Value()); | ||
| positionVerticesCount = static_cast<long>(data.ElementLength()) / size; | ||
| hasPosition = true; | ||
| break; | ||
| } |
Comment on lines
+312
to
+326
| // Indices: use the provided buffer, or synthesize an identity list for unindexed meshes. | ||
| std::vector<int> indices; | ||
| if (info.Length() > 1 && info[1].IsTypedArray()) | ||
| { | ||
| indices = ReadIndices(info[1].As<Napi::TypedArray>()); | ||
| } | ||
| else | ||
| { | ||
| indices.resize(positionVerticesCount); | ||
| for (long i = 0; i < positionVerticesCount; ++i) { indices[i] = static_cast<int>(i); } | ||
| } | ||
|
|
||
| draco::Mesh mesh; | ||
| const long numFaces = static_cast<long>(indices.size()) / 3; | ||
| mesh.SetNumFaces(numFaces); |
Comment on lines
+344
to
+352
| const auto attr = attributesIn.Get(i).As<Napi::Object>(); | ||
| const std::string kind = attr.Get("kind").As<Napi::String>().Utf8Value(); | ||
| const std::string dracoName = attr.Get("dracoName").As<Napi::String>().Utf8Value(); | ||
| const int8_t size = static_cast<int8_t>(attr.Get("size").As<Napi::Number>().Int32Value()); | ||
| const auto data = attr.Get("data").As<Napi::TypedArray>(); | ||
| const draco::GeometryAttribute::Type type = DracoAttributeTypeFromName(dracoName); | ||
|
|
||
| const int attId = AddTypedAttributeToMesh(env, mesh, type, data, size); | ||
| if (attId < 0) |
Comment on lines
+21
to
+22
| add_subdirectory(NativeDraco) | ||
| add_subdirectory(NativeMeshopt) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds native C++ implementations of the Draco and meshopt mesh decoders as two
new optional plugins,
NativeDracoandNativeMeshopt, plus a native Dracoencoder.
Today Babylon Native runs these codecs through the same WASM modules the web
uses. That pulls a WASM runtime into a native app purely to decompress meshes,
costs a module instantiation per session, and copies buffers across the JS/WASM
boundary. These plugins let the native host do the work directly.
What's here
NativeDraco_native.decodeDracoMesh(data, attributeIds?)draco_decoder.wasmNativeDraco_native.encodeDracoMesh(attributes, indices?, options?)draco_encoder.wasmNativeMeshopt_native.decodeMeshopt(source, count, stride, mode, filter?)meshopt_decoder.wasmdecodeDracoMeshsupports both call shapes the Babylon loaders use: the glTFpath, where the caller passes the
EXT/KHR_draco_mesh_compressionmap ofBabylon vertex-buffer kind to Draco unique id, and the standalone
.drcpath,where no map is supplied and the plugin probes the standard named attributes.
Attribute data is de-interleaved and tightly packed per point, mirroring
emscripten's
GetAttributeDataArrayForAllPointsthat the WASM decoder relieson, so the returned buffers are drop-in equivalents.
Dependencies
Two new
FetchContentdependencies, both pinned to release tags:google/draco@1.5.7zeux/meshoptimizer@v0.22Both are declared
EXCLUDE_FROM_ALLand built with tests, install, executablesand JS glue disabled. draco adds its include directories as
PRIVATE, soDependencies/CMakeLists.txtre-exports them asPUBLICon the combinedtarget; the target is named
dracounder MSVC anddraco_staticelsewhere andboth spellings are handled.
Scope
The change is additive only: +657 / -0 across 11 files. Nothing existing is
modified except the CMake wiring that registers the new plugins and four lines
in
Embedding/Source/Runtime.cppthat initialize them. No existing plugin,engine path, or test configuration is touched, and
Apps/Playground/Scripts/config.jsonis deliberately left alone.
On validation
These entry points are not reachable from the currently pinned
babylonjsnpm package (9.15.0) — it contains no
decodeDracoMesh/encodeDracoMesh/decodeMeshoptrouting, so the plugins are inert until the correspondingBabylon.js side lands and the loaders start feature-detecting them. That means
the validation suite cannot exercise this code, so I verified it directly
instead, driving the exported functions from a script in the Playground.
Draco, encode -> decode round trip. A 4-vertex / 2-triangle mesh with
positionandnormalattributes encodes to 143 bytes and decodes back to4 vertices, 2 triangles, both attributes intact. Draco reorders vertices, so
the check compares canonicalized vertex and triangle sets rather than raw
index order. Verified through both decode paths — with the encoder's
attributeIdsmap (glTF), and with no map (named probe).meshopt decode, cross-implementation. A vertex buffer (8 vertices x 12
byte stride) encoded by the reference
meshoptimizer0.22.0 JS encoder —the same version this PR pins the native decoder to — decodes to a byte-exact
match of the original 96 bytes.
Worth noting for anyone testing this later: meshoptimizer's vertex codec is
versioned in the first header byte. Encoders from 0.23.0 onward emit format
version 1 (
0xa1), which a v0.22 decoder correctly rejects with-1. Theencoder used to produce the fixture was version-matched deliberately.
No regression. Full validation suite on this branch:
ran=300 passed=300 failed=0 missingRef=0 skipped=420, identical to theupstream
masterbaseline.