Training problems

[panxiong1994]

Hello! This is a very impressive work, and we have encountered some issues during training and would like to consult you.

1. During the training phase, we encountered an issue where gt_mask does not exist. We downloaded the checkpoints you provided, but the number of files inside does not match your provided JSON file—some images appear to be missing.

2. Regarding pseudo-mask generation: Your paper describes a Pseudo Mask Generation method. Could you please explain how to implement your approach using GroundingSAM?

3. Where exactly in the dataset should we place the point3d.ply file generated through Gaussian Splatting training? and the datasets link in the hugingface is closed

Thank you very much for sharing your work with us.

[panxiong1994]

When I downloaded the mask data you provided and started training, I encountered the following problem2:

[panxiong1994]

Hello, I followed the steps as outlined below:

1. Used the 3DGS code to generate chkpnt30000.pth and input.ply.
2. Ran the command python train.py -s ./data/Ref-lerf/ramen -m ./output/ramen to generate the model chkpnt_cbasetea2514.pth.
3. Executed python render.py -s /home/x/RIS/data/Ref-lerf/ramen -m ./output/ramen/ to generate the rendering results.
4. Ran the test_miou.py script to evaluate the ground truth (gt) and rendered outputs from Step 3.

During this process, Step 2 attempts to load gt_mask; however, using your provided train/test split files (train_json and test_json) causes errors during execution, because the masks are loaded according to the image’s corresponding position.

To resolve this issue, we merged both train_json and test_json together. However, this means that during testing, evaluation is no longer performed on the specific images you originally designated. Therefore, could you please provide fully runnable code specifically for Step 2?

thank you very much.

[panxiong1994]

we use the gt_mask and train the model, we only get 15.44. but use your pth can get 35.97. What is cause this problem?
thank you very much.

[eR3R3]

its absolutely not just 15.44, I trained it in the original code and get a 21 something, but it's far from 35

[eR3R3]

but I am just talking about the ramen dataset

[heshuting555]

Thank you for your interest in our work, and apologies for the delayed reply.

Yes, the train and test JSON files need to be merged. We’ll update the code soon to make it more complete. Thank you for pointing this out.

As for the performance issue, results may vary across epochs. Overfitting can occur quickly on a small dataset, so I suggest checking the results from the third epoch.