Camera Extrinsics Vs Camera Transform

[offjangir]

In genesis documentation I see that there is camera.extrinsics and camera.transform What is diffrence between 2 frames. I have camera extrinsics how to initiate camera using them ? The frame of pos etc. in add_camera is diffrent then frame of extrinsic.

[gxy-1111]

Hi, have you figured out the difference between camera.extrinsics and camera.transform? When I want to compute the transformer from pixel to world, which parmer I need to use?

[Sologala]

Same question of the differences of "extrinsics" and "transform"

print("transform", camera.transform)
print("extrinsics", camera.extrinsics)

transform [[ 0.    0.67 -0.74 -3.88]
 [-1.    0.    0.    0.26]
 [ 0.    0.74  0.67  9.83]
 [ 0.    0.    0.    1.  ]]
extrinsics [[-0.   -1.   -0.   -0.  ]
 [-0.67 -0.   -0.74  0.  ]
 [ 0.74  0.   -0.67 13.45]
 [ 0.    0.    0.    1.  ]]

The printed value are totally different.

[duburcqa]

• camera.transform stores the pose of the camera in the world. This may be changing dynamically between every timesteps.
• camera.extrinsics fully specify the transform from pixel to world. It is supposed to be constant for a given camera.

[Sologala]

@duburcqa Ok , Got it.
I try out the way to project point in world frame to camera frame by using the transform. It seems like the transform be the extrinsics in common 3dvision geometry. 😕

            T_wc = camera.transform
            t_wc = T_wc[:3, 3]
            R_wc = T_wc[:3, :3]
            R_cw = R_wc.T
            t_cw = -R_cw @ t_wc

            #  project some point in world to camrea frame
            boat_pos_cam = R_cw @ boat_pos + t_cw

[duburcqa]

Please have a look at this implementations:

• Genesis/genesis/vis/camera.py

  Line 533 in 5b1fca1

  def render_pointcloud(self, world_frame=True):

• Genesis/tests/test_render.py

  Line 851 in 5b1fca1

  def test_point_cloud(renderer_type, renderer, show_viewer):