Question about reproducing the Replica tracking results in Table II

[JackmanYuan]

Hello, and thank you for releasing 2DGS-SLAM.

I am trying to reproduce the Replica tracking results reported in Table II and would appreciate some guidance on the intended experimental setup.

Setup

• Repository commit: b46da105aa6d3899816e05cbf118be96819b8c6c

• Dataset: Replica, organized according to configs/replica/

• Command:

  python slam.py --config configs/replica/<scene>.yaml

• Full sequence with the released frame_step=2
• Loop closure enabled according to the released configuration
• No GUI or web visualizer
• No post-SLAM map refinement for trajectory evaluation

My results do not align with those reported in Table II, and I was wondering whether I might be missing part of the intended setup.

Scene	Table II ATE (cm)	Reproduced ATE (cm)
room0	0.06	43.96
room1	0.08	14.52
room2	0.10	15.01
office4	0.09	13.01

All sequences completed without runtime errors and processed 1,000 frames.

I also noticed that several values in the Replica configuration differ from those reported in Table I:

Parameter	Table I	Replica configuration
Keyframe distance threshold	15 cm	0.4 m
Covisibility threshold	0.9	0.925
Mapping depth weight	0.5	5.0
Mapping normal weight	0.02	0.05
Active/inactive training frames	3 / 2	5 / 3

Could you please clarify:

1. Which commit, tag, or release was used to produce the results in Table II?
2. Should the Table I parameters be used for reproducing Table II, or are the released Replica configuration values the intended settings?
3. How exactly was ATE computed for Table II, including the evaluated frames and trajectory alignment protocol?

I would be happy to provide the complete logs, environment information, configuration files, evaluation commands, and saved trajectories if helpful.

Thank you very much for your time and for releasing the project.

[YuePanEdward]

Thanks for your interest in our project.
I reran the experiments on the Replica dataset with the current code and didn't get your reported results. Could you please paste the logs here, like the following?

SLAM: init the system
SLAM: 100%|█████████████████████████████████████████████| 999/999 [1:12:33<00:00, 4.11s/frame, frame=2000, kf=186]
Eval: Final ATE on all keyframes (n=186)
Eval: RMSE ATE [m] (final_keyframes): 0.0065
Eval: Final ATE on all tracked frames (n=1000)
Eval: RMSE ATE [m] (final_all_tracked): 0.0050

[JackmanYuan]

Hi! Thank you for rerunning the experiment.

I created a clean worktree directly from commit
b46da10 and reran Office0
on an otherwise idle server using:

python slam.py --config configs/replica/office0.yaml

The tracked source tree was identical to origin/main. I obtained:

SLAM: init the system
SLAM: 100%|██████████| 999/999 [1:01:20<00:00, 4.31s/frame, frame=2000, kf=190]
Eval: Final ATE on all keyframes (n=190)
Eval: RMSE ATE [m] (final_keyframes): 0.1220
Eval: Final ATE on all tracked frames (n=1000)
Eval: RMSE ATE [m] (final_all_tracked): 0.1318

Environment:

• NVIDIA RTX 6000 Ada
• Python 3.10.20
• PyTorch 2.5.1+cu121
• torchvision 0.20.1+cu121
• Open3D 0.19.0
• GTSAM 4.2

To help identify the difference, could you please share the exact commit hash, PyTorch/CUDA versions, and Replica dataset source used for your rerun?

For comparison, the SHA256 of my office0/traj.txt is:

6fdf15ff9a85626e0fd2f5f65ade95cd403d8043054701f3398aaf277d5cf602

I can also provide the complete log and configuration files if helpful.

[YuePanEdward]

Hi, sorry for the late reply.
My environment setup for reproducing the experiment results is:

PyTorch: 2.5.1+cu121
CUDA (toolkit): 12.1
cuDNN: 9.1.0 (90100)
NVIDIA driver: 535.230.02
GPU: NVIDIA RTX A5000
Python: 3.10.20

However, I have also recently run experiments on A100 GPU and got similar results to yours. We will try to figure out the root cause of the mismatch due to hardware and environments ASAP.