Voxelizing splats to OpenVDB #279
Description
OpenVDB is a popular format for volumetric rendering of sparse 3D grids. There is a good deal of tooling around it, e.g. Blender 5 supports it natively, so you can import a VDB cloud and render it with a state of the art renderer. There are a few active projects right now that are going to render VDB on WebGPU.
It would be great if Spark supported this format. It could provide an API to voxelize splats on GPU into the nanovdb/picovdb format at a certain level-of-detail (LoD). Your existing rasterizer can technically do this by rendering splats into many thin 3d slices. If we think of gaussian splats as a 3D SVG, then voxelizing it is similar to pixelizing an SVG.
Here is a motivating example. I used Spark to generate the following scene with 16 million splats. Spark renders them at a good framerate, but there is no way to add realistic lights. So I voxelized the splats into a 2000x2000x2000 grid with ~750M non-zero voxels and used Blender to render it:
Besides rendering, the VDB format can store vector fields and apply common operators to them: gradient, divergence, curl, advection and so on. The latter makes it easy to advect a 3d density field with a cloud using another 3d velocity vector field, effectively transforming the cloud in a realistic manner. Spark could theoretically import the velocity field from VDB and apply them to a scene with gaussian splats.