forked from IIPCVLAB/LCCNet
-
Notifications
You must be signed in to change notification settings - Fork 0
KITTI odometry evaluation
Jayadeep Karnati edited this page Aug 2, 2023
·
2 revisions
LCCNet (7473067) Evaluation metrics on KITTI odometry dataset:
| Iteration | Mean Translation Error (cm) | Mean Rotation Error (°) | Median Translation Error (cm) | Median Rotation Error (°) | Std. Translation Error (cm) | Std. Rotation Error (°) |
|---|---|---|---|---|---|---|
| 0 | 142.4449 | 19.1410 | 146.6212 | 19.6021 | 41.7970 | 5.5942 |
| 1 | 15.9710 | 1.2324 | 11.8625 | 0.8350 | 15.9951 | 1.6224 |
| 2 | 4.6279 | 0.5653 | 4.1592 | 0.3667 | 2.9313 | 1.0704 |
| 3 | 2.3157 | 0.4226 | 1.8792 | 0.2798 | 1.7609 | 1.0206 |
| 4 | 1.5184 | 0.3125 | 0.8859 | 0.1992 | 1.9753 | 0.9960 |
| 5 | 1.3158 | 0.2015 | 0.6122 | 0.1159 | 2.0315 | 0.9914 |
Took 12 minutes to run on Nvidia RTX 2060.
Logs
$ python evaluate_calib.py
WARNING - LCCNet-evaluate-iterative - No observers have been added to this run
INFO - LCCNet-evaluate-iterative - Running command 'main'
INFO - LCCNet-evaluate-iterative - Started
Ground truth poses are not avaialble for sequence 11.
Ground truth poses are not avaialble for sequence 12.
Ground truth poses are not avaialble for sequence 13.
Ground truth poses are not avaialble for sequence 14.
Ground truth poses are not avaialble for sequence 15.
Ground truth poses are not avaialble for sequence 16.
Ground truth poses are not avaialble for sequence 17.
Ground truth poses are not avaialble for sequence 18.
Ground truth poses are not avaialble for sequence 19.
Ground truth poses are not avaialble for sequence 20.
Ground truth poses are not avaialble for sequence 21.
VAL SET: Using this file: /home/jay/datasets/KITTI/odometry_color/sequences/val_RT_left_seq00_20.00_1.50.csv
4541
0%| | 0/4541 [00:00<?, ?it/s]Init worker 0 with seed 339816598
Init worker 1 with seed 339816599
Init worker 2 with seed 339816600
Init worker 3 with seed 339816601
Init worker 4 with seed 339816602
Init worker 5 with seed 339816603
/home/jay/mambaforge/envs/lcc/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=405 error=11 : invalid argument
100%|█████████████████████████████████████████████████████████████| 4541/4541 [12:22<00:00, 6.11it/s]
Iterative refinement:
Iteration 0: Mean Translation Error: 142.4449 cm Mean Rotation Error: 19.1410 °
Iteration 0: Median Translation Error: 146.6212 cm Median Rotation Error: 19.6021 °
Iteration 0: Std. Translation Error: 41.7970 cm Std. Rotation Error: 5.5942 °
Iteration 0: Mean Translation X Error: 124.1440 cm Median Translation X Error: 132.1708 cm Std. Translation X Error: 18.7156 cm
Iteration 0: Mean Translation Y Error: 88.8781 cm Median Translation Y Error: 65.1481 cm Std. Translation Y Error: 47.2362 cm
Iteration 0: Mean Translation Z Error: 60.6971 cm Median Translation Z Error: 25.3637 cm Std. Translation Z Error: 63.0298 cm
Iteration 0: Mean Rotation Roll Error: 15.5884 ° Median Rotation Roll Error: 15.1797 ° Std. Rotation Roll Error: 3.3502 °
Iteration 0: Mean Rotation Pitch Error: 14.5941 ° Median Rotation Pitch Error: 16.6809 ° Std. Rotation Pitch Error: 6.4290 °
Iteration 0: Mean Rotation Yaw Error: 12.9661 ° Median Rotation Yaw Error: 13.3254 ° Std. Rotation Yaw Error: 4.4223 °
Iteration 1: Mean Translation Error: 15.9710 cm Mean Rotation Error: 1.2324 °
Iteration 1: Median Translation Error: 11.8625 cm Median Rotation Error: 0.8350 °
Iteration 1: Std. Translation Error: 15.9951 cm Std. Rotation Error: 1.6224 °
Iteration 1: Mean Translation X Error: 24.9094 cm Median Translation X Error: 17.4307 cm Std. Translation X Error: 14.5424 cm
Iteration 1: Mean Translation Y Error: 6.6866 cm Median Translation Y Error: 4.5194 cm Std. Translation Y Error: 5.3829 cm
Iteration 1: Mean Translation Z Error: 4.8419 cm Median Translation Z Error: 3.2328 cm Std. Translation Z Error: 3.7389 cm
Iteration 1: Mean Rotation Roll Error: 0.8375 ° Median Rotation Roll Error: 1.0245 ° Std. Rotation Roll Error: 0.5651 °
Iteration 1: Mean Rotation Pitch Error: 0.4014 ° Median Rotation Pitch Error: 0.1515 ° Std. Rotation Pitch Error: 0.5200 °
Iteration 1: Mean Rotation Yaw Error: 0.5895 ° Median Rotation Yaw Error: 0.1826 ° Std. Rotation Yaw Error: 0.8330 °
Iteration 2: Mean Translation Error: 4.6279 cm Mean Rotation Error: 0.5653 °
Iteration 2: Median Translation Error: 4.1592 cm Median Rotation Error: 0.3667 °
Iteration 2: Std. Translation Error: 2.9313 cm Std. Rotation Error: 1.0704 °
Iteration 2: Mean Translation X Error: 1.4049 cm Median Translation X Error: 0.9402 cm Std. Translation X Error: 1.0910 cm
Iteration 2: Mean Translation Y Error: 3.6883 cm Median Translation Y Error: 3.1790 cm Std. Translation Y Error: 2.0065 cm
Iteration 2: Mean Translation Z Error: 1.1210 cm Median Translation Z Error: 1.3836 cm Std. Translation Z Error: 0.8185 cm
Iteration 2: Mean Rotation Roll Error: 0.4047 ° Median Rotation Roll Error: 0.0673 ° Std. Rotation Roll Error: 0.6406 °
Iteration 2: Mean Rotation Pitch Error: 0.4911 ° Median Rotation Pitch Error: 0.0837 ° Std. Rotation Pitch Error: 0.7444 °
Iteration 2: Mean Rotation Yaw Error: 0.3735 ° Median Rotation Yaw Error: 0.0559 ° Std. Rotation Yaw Error: 0.5966 °
Iteration 3: Mean Translation Error: 2.3157 cm Mean Rotation Error: 0.4226 °
Iteration 3: Median Translation Error: 1.8792 cm Median Rotation Error: 0.2798 °
Iteration 3: Std. Translation Error: 1.7609 cm Std. Rotation Error: 1.0206 °
Iteration 3: Mean Translation X Error: 1.6656 cm Median Translation X Error: 1.2080 cm Std. Translation X Error: 1.3287 cm
Iteration 3: Mean Translation Y Error: 1.5005 cm Median Translation Y Error: 2.0992 cm Std. Translation Y Error: 1.2466 cm
Iteration 3: Mean Translation Z Error: 1.5560 cm Median Translation Z Error: 1.5598 cm Std. Translation Z Error: 0.7202 cm
Iteration 3: Mean Rotation Roll Error: 0.1885 ° Median Rotation Roll Error: 0.0254 ° Std. Rotation Roll Error: 0.3016 °
Iteration 3: Mean Rotation Pitch Error: 0.2603 ° Median Rotation Pitch Error: 0.0367 ° Std. Rotation Pitch Error: 0.4077 °
Iteration 3: Mean Rotation Yaw Error: 0.2358 ° Median Rotation Yaw Error: 0.0389 ° Std. Rotation Yaw Error: 0.3493 °
Iteration 4: Mean Translation Error: 1.5184 cm Mean Rotation Error: 0.3125 °
Iteration 4: Median Translation Error: 0.8859 cm Median Rotation Error: 0.1992 °
Iteration 4: Std. Translation Error: 1.9753 cm Std. Rotation Error: 0.9960 °
Iteration 4: Mean Translation X Error: 0.2836 cm Median Translation X Error: 0.1875 cm Std. Translation X Error: 0.2766 cm
Iteration 4: Mean Translation Y Error: 0.2686 cm Median Translation Y Error: 0.3587 cm Std. Translation Y Error: 0.2239 cm
Iteration 4: Mean Translation Z Error: 0.2484 cm Median Translation Z Error: 0.1613 cm Std. Translation Z Error: 0.2934 cm
Iteration 4: Mean Rotation Roll Error: 0.1399 ° Median Rotation Roll Error: 0.0244 ° Std. Rotation Roll Error: 0.2048 °
Iteration 4: Mean Rotation Pitch Error: 0.1274 ° Median Rotation Pitch Error: 0.0269 ° Std. Rotation Pitch Error: 0.1823 °
Iteration 4: Mean Rotation Yaw Error: 0.1109 ° Median Rotation Yaw Error: 0.0323 ° Std. Rotation Yaw Error: 0.1372 °
Iteration 5: Mean Translation Error: 1.3158 cm Mean Rotation Error: 0.2015 °
Iteration 5: Median Translation Error: 0.6122 cm Median Rotation Error: 0.1159 °
Iteration 5: Std. Translation Error: 2.0315 cm Std. Rotation Error: 0.9914 °
Iteration 5: Mean Translation X Error: 0.2511 cm Median Translation X Error: 0.3297 cm Std. Translation X Error: 0.2130 cm
Iteration 5: Mean Translation Y Error: 0.1414 cm Median Translation Y Error: 0.1302 cm Std. Translation Y Error: 0.0840 cm
Iteration 5: Mean Translation Z Error: 0.3964 cm Median Translation Z Error: 0.4185 cm Std. Translation Z Error: 0.3474 cm
Iteration 5: Mean Rotation Roll Error: 0.0648 ° Median Rotation Roll Error: 0.0118 ° Std. Rotation Roll Error: 0.0953 °
Iteration 5: Mean Rotation Pitch Error: 0.0567 ° Median Rotation Pitch Error: 0.0134 ° Std. Rotation Pitch Error: 0.0824 °
Iteration 5: Mean Rotation Yaw Error: 0.0414 ° Median Rotation Yaw Error: 0.0077 ° Std. Rotation Yaw Error: 0.0603 °
average runing time on 5 iteration: 0.00343108801977195 s
End!
INFO - LCCNet-evaluate-iterative - Completed after 0:12:32