[FEATURE] Add ImGui overlay for interactive joint control and visualization#2541
Draft
YilingQiao wants to merge 4 commits intoGenesis-Embodied-AI:mainfrom
Draft
[FEATURE] Add ImGui overlay for interactive joint control and visualization#2541YilingQiao wants to merge 4 commits intoGenesis-Embodied-AI:mainfrom
YilingQiao wants to merge 4 commits intoGenesis-Embodied-AI:mainfrom
Conversation
YilingQiao
changed the title
ImGui Overlay (genesis/ext/pyrender/imgui_overlay.py): - ViewerPlugin-based ImGui overlay for the native pyrender viewer - Joint control panel with per-DOF sliders, quaternion groups, free joints - Entity vis mode switching (visual/collision/wireframe), contact viz toggle - Camera control, shadow/frame/frustum toggles, render flag overlays - Example: examples/gui/imgui_joint_control.py Shared module (genesis/vis/scene_ops.py): - Extracted shared visualization helpers used by the overlay - refresh_visual_transforms, switch_entity_vis_mode, set_entity_wireframe - build_entity_joint_data with free/spherical joint handling Tests: - tests/test_imgui_overlay.py: 5 tests covering imports, entity data, overlay Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YilingQiao
force-pushed
the
yiling/260313_imgui_only
branch
from
e202708 to
9f9eff8
Compare
duburcqa
reviewed
Mar 14, 2026
duburcqa
reviewed
Mar 14, 2026
duburcqa
reviewed
Mar 14, 2026
duburcqa
reviewed
Mar 14, 2026
duburcqa
reviewed
Mar 14, 2026
duburcqa
reviewed
Mar 14, 2026
Collaborator
duburcqa
left a comment
duburcqa
left a comment
There was a problem hiding this comment.
Please add a new entrypoint 'gs play' or similar to expose this nice interactive viewer / playground directly to the user and installing genesis via pip.
…lay entrypoint, CI integration - Reformat scene_ops.py docstrings to 120-char width and remove section comments - Delete mock-based imgui overlay tests - Add PYTEST_VERSION detection to imgui example for CI (5-frame exit) - Add gui/**/*.py to CI example test discovery - Add 'gs play' CLI entrypoint for interactive ImGui viewer with optional asset file
duburcqa
reviewed
Mar 20, 2026
Replaces mock-based tests with a single integration test that builds a real scene with Panda robot and verifies plugin state, should_step logic, entity cache, and qpos updates.
Not part of the GUI feature scope.
duburcqa
reviewed
Mar 20, 2026
| """Test ImGuiOverlayPlugin core logic: initial state, should_step(), _cache_entity_data().""" | ||
| from genesis.ext.pyrender.imgui_overlay import ImGuiOverlayPlugin | ||
|
|
||
| plugin = ImGuiOverlayPlugin() |
Collaborator
There was a problem hiding this comment.
This is broken. It is not how pyrender plugins works.
duburcqa
reviewed
Mar 20, 2026
|
|
||
|
|
||
| @pytest.mark.required | ||
| def test_imgui_overlay_plugin(show_viewer): |
Collaborator
There was a problem hiding this comment.
This test is not checking anything relevant. Remove it.
duburcqa
reviewed
Mar 20, 2026
Comment on lines
+66
to
+67
| plugin = ImGuiOverlayPlugin(rebuild_fn=rebuild_scene) | ||
| scene.viewer._pyrender_viewer.register_plugin(plugin) |
Collaborator
There was a problem hiding this comment.
This is not how plugins are being registered: scene.viewer.add_plugin
duburcqa
reviewed
Mar 20, 2026
| scene.build() | ||
|
|
||
| # Re-register plugin on new viewer | ||
| scene.viewer._pyrender_viewer.register_plugin(plugin) |
Collaborator
There was a problem hiding this comment.
This is not how plugins are being registered
duburcqa
requested changes
Mar 23, 2026
Collaborator
duburcqa
left a comment
duburcqa
left a comment
There was a problem hiding this comment.
I cannot be merged without at least one screenshot-based "unit test" (actually integration test) based on playwright.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ImGui Overlay (genesis/ext/pyrender/imgui_overlay.py):
Shared module (genesis/vis/scene_ops.py):
Rasterizer (genesis/vis/rasterizer.py):
Tests:
gui_test.mp4
Related Issue
Resolves Genesis-Embodied-AI/Genesis#
Motivation and Context
How Has This Been / Can This Be Tested?
Screenshots (if appropriate):
Checklist:
Submitting Code Changessection of CONTRIBUTING document.