Implementation of the RenderingDevice renderer

[mr11stein]

I've implemented a renderer based on the RenderingDevice, which should vastly improve the performance especially on mobile devices. Please give it a try and compare the performance.

There are still some minor tweaks necessary:

1. How to switch between the default renderer and the RenderingDevice (RD) renderer. It should be possible to switch them during runtime, since they use the same interface. Switching renderers is a Project Setting
2. How to handle rendering for the user. The best way in my opinion is to render to a buffer that has the same resolution as the Godot framebuffer. If you want to move the model around you should change the transform inside the model, not the transform of the model. The renderer needs to be notified when the window is resized so that the render buffer can be resized.
3. There's a message at the end about leaked resources which need to be cleaned up when the plugin is unloaded which comes from the InternalCubismPipelineMangerRD. Mostly done but switching models doesn't work yet.
4. Basic code formatting.

[erodozer]

Have you been able to test the performance to get measurements against the latest versions of the extension. Could you add screenshots to the PR that show the performance monitors in Godot's debugger to visualize the differences in your test cases?

I'm curious about the scale of impact of this kind of approach after the changes I made. At face value, the actual function of it does not appear to be that much different than what Godot's own renderer does out of the box for 2D surfaces. Many of the optimizations that were missing, such as proper culling bounds and tighter mask textures, have since been implemented, so 0.9.1 is a lot different than how things were with 0.8.

Things that stick out to me

• Isn't creating all these surfaces for offscreen rendering effectively going to be the same as just using Godot's Subviewport system. For high resolution models, creating texture surfaces that encapsulate the entire model at full resolution is going to be incredibly detrimental to memory, which is why we sought to move away from this approach in Direct rendering #129
• The largest difference I see here is bypassing Godot's Mesh resources, however with the optimizations done in improve performance under modern renderer #140 and Godot's own efficient management of resources, it's unclear how much is really saved by avoiding this overhead. Without the mesh nodes, you also lose the ability to use Godot's modulation system supported in update shaders to support godot's canvasitem modulation #143 on a per mesh bases (which is a use-case I personally have for character customization)

[mr11stein]

I haven't measured time yet, but comparing the rendering stats for the Mao Model gives
Direct vs. RD

• Total Objects drawn: 333 vs. 168
• Total Primitives Drawn: 12336 vs. 992
• Total Draw Calls: 333 vs. 168

This is taken from the viewer, so there's minimal UI overhead. I'm sure the RD renderer can still be optimized, as this is the first implementation. The draw calls itself should also run faster than Godot meshes, since the shaders are much simpler.

The last time I checked the memory for both the Direct/RD renderer was about the same. But you should be able to render directly to the screen buffer, saving memory for the output texture which I didn't bother to do yet. I wanted to create the PR so I can gather feedback how to incorporate this renderer.

If you want to modulate the meshes you either need to modify the renderer/shaders or use the Direct renderer.

As for masking I used the implementation provided by Live2D explained here: https://docs.live2d.com/en/cubism-sdk-manual/ow-sdk-mask-premake/

[erodozer]

Glad you could get that mask packing technique working with this approach. I was wishing I could do something similar with the direct rendering, but Godot's inability to allow SubViewports to render to a shared target limited how far I could take it. Best that I could achieve with just RenderServer was #147.

I think if modulation colors of each mesh are mirrored into the model's properties, similar to the part opacity, that would satisfy my use-case.

I'm expecting the swapping rendering approaches on a loaded model at runtime to be both tricky and not exactly desirable. Maybe putting the choice of renderer behind a Project Setting, similar to rendering driver, will be enough.

My effort in #136 is to also remove the ability to swap a model on an instance in favor of a more Godot approach of managing models as resources. I'm thinking about how to make this work with that effort, along with looking to what it'll take to make things more functional/stateless.

[mr11stein]

Yes I think Project Settings is the best place for this setting. Unfortunately, you can't add descriptions yet (see here), but you will need to restart the Editor for the change to apply.
I added a project setting which defaults to the Direct Renderer because the Rendering Device is not available for all Godot Renderer backends.

I would rename InternalCubismRenderer2D to InternalCubismRenderer2D_Direct or something and use InternalCubismRenderer2D as the base class for both renderers that supplies a common interface.
Also, can you clarify what you mean by modulation? Per-mesh base/screen/multiply color is already supported by this implementation (as I basically copied the shaders provided by the Live2D Vulkan implementation). Custom shaders would be more difficult since the RenderingDevice doesn't use the Godot shading language. I think it's perfectly reasonable to modify the code if you want more customization.

[erodozer]

Color modulations in Godot is a standard feature of Canvas Items which allows you to apply for own overriding multiplier in order to tint the output. In the default render pipeline and shaders it comes through as vertex colors. In the case of Live2D, it's not an override of the colors set by the SDK, but an additional value you would want to apply.

Being able to tint individual meshes or the model as a whole is a useful feature, as it helps enable developers to more efficiently implement things like character customization and simple lighting effects such as day/night systems.

With the direct renderer, all meshes are canvas items in the scene tree, so they all have modulation values that pass into their shaders. The user model node itself is also a canvas item and thus can be tinted, which the effects will cascade to child nodes

[mr11stein]

I see. I think that's possible but I would address this feature in a later PR. Maybe both renderers should be different nodes, since it doesn't make sense to expose the shader properties for the RD renderer, since the RD renderer can't use GD shaders and instead expose modulation properties directly.

[MizunagiKB]

As my knowledge of low-level systems is insufficient, I find it difficult to accurately assess the implementation of this pull request. However, the content is extremely intriguing, and the numerical results provided seem to indicate favorable outcomes.

Processes that contribute to reducing workload are certainly valuable to many people. Yet, when crafted to this high level of sophistication, I personally feel it may be somewhat excessive for my own needs.

I sense that GDCubism has gained wider acceptance than I initially expected, but I personally value the enjoyment of coding as a hobby. Consequently, accepting something beyond my capabilities raises concerns about whether I would be able to maintain and manage the code effectively on my own.

Of course, I understand the perspective that technically superior solutions should be adopted. For those who prioritize product value and technical excellence, my viewpoint may differ from their approach.

While I wish there were a way to connect your efforts with those who truly need them, I prefer to maintain a state that aligns with my own capacity at present. Therefore, I regret to inform you that I have decided not to merge this pull request.

I deeply apologize for any inconvenience.