Add spherical_reprojected mode to panorama_sfm example

[shuta-ochiai]

Summary

This adds a new mode, spherical_reprojected, to python/examples/panorama_sfm.py, as a middle ground between the existing perspective_* and spherical modes. It implements the hybrid flow proposed in #4406, where @chpatrick agreed this is the right approach:

1. Render the panoramas into perspective virtual camera images and extract features on them (as in the perspective modes, using the same overlapping layout and masks) — so the feature quality does not suffer from the equirectangular distortion, especially towards the poles.
2. Reproject the keypoints onto the panoramas (reusing the same spherical mapping as convert_to_equirectangular) and inject them, together with their descriptors and GPS pose priors, into a panorama database with a single native EQUIRECTANGULAR camera.
3. Match and reconstruct directly on the panoramas (as in the spherical mode), with one image per panorama instead of a rig — so matching and mapping cost stays low (the overlapping rig instead feeds 12 images per panorama into matching and bundle adjustment).

Compared to the flow sketched in #4406, the one difference is that matching runs on the panoramas after reprojecting the keypoints and descriptors, instead of matching on the faces and reprojecting the matches. This is simpler (no match/two-view-geometry rewriting) and lighter: each panorama pair costs a single matcher invocation and a single geometric verification, whereas matching on the faces needs up to N×N face-pair matches and verifications per panorama pair to catch correspondences that cross face boundaries (12×12 for the overlapping layout), with correspondingly more scheduling overhead and database rows. Cross-face correspondences are matched naturally, since all reprojected features of a panorama live in one image.

Implementation notes

• The keypoint reprojection math is shared with convert_to_equirectangular via a new PanoProcessor.pano_xy_from_cam_xy helper; rendering + feature extraction is shared with the perspective modes via a new render_and_extract_features helper.
• The panorama database skeleton (camera, images, trivial rigs/frames) is created directly via the Database bindings and apply_rig_config([], db), avoiding a throwaway feature extraction pass on the panoramas.
• GPS pose priors imported from the renderings' EXIF tags are copied over to the panorama images, so --matcher spatial also works in this mode.
• Only the keypoint centers are reprojected; the scale/orientation columns stay in the rendered image frame, which is fine since matching only compares descriptors and mapping only uses the centers.

This mirrors a pipeline we run in production on long 360° video walkthroughs, where the perspective rig mode is accurate but matching/mapping-heavy and the spherical mode loses feature quality; extracting on the renderings and mapping on the panoramas gave us perspective-level feature quality at spherical-level cost.

Closes #4406.

🤖 Generated with Claude Code

[shuta-ochiai]

A note on feature/matcher support of the reprojection design (the example itself currently wires up the default SIFT + nearest-neighbor matching only):

• SIFT + NN matching (default): supported. The NN matchers only compare descriptors, and mapping only uses the keypoint centers, so reprojecting the centers is sufficient.
• ALIKED + LightGlue: should work as-is (untested in this mode). The injection preserves the descriptor dtype/type from the database, and the ALIKED LightGlue model consumes only keypoint positions + descriptors — the reprojected positions are the true panorama coordinates, so its inputs stay consistent. The usual caveat of running LightGlue on equirectangular position distributions applies equally to the spherical mode.
• SIFT + LightGlue: not supported yet, future work. The SIFT LightGlue model additionally consumes per-keypoint scale/orientation, which currently stay in the rendered image frame while the positions are in panorama coordinates — i.e. inconsistent inputs. Supporting this would require propagating the local Jacobian of the face→panorama mapping to the shape columns (see the code comment in reproject_features_to_panos).

🤖 Generated with Claude Code

[sarlinpe]

Thank you for your contribution! I have a few concerns, which to me are for now sufficiently critical to not merge this PR.

Match and reconstruct directly on the panoramas (as in the spherical mode), with one image per panorama instead of a rig — so matching and mapping cost stays low (the overlapping rig instead feeds 12 images per panorama into matching and bundle adjustment).

Could you please provide evidence of this? The speed of matching and BA scales mostly with the number of features, which is unchanged when projecting features from perspective to spherical. BA already considers all observations in the rig at once.

the reprojected positions are the true panorama coordinates, so its inputs stay consistent. The usual caveat of running LightGlue on equirectangular position distributions applies equally to the spherical mode.

Could you please quantify this? I highly doubt that LightGlue performs satisfactorily with such inputs.