|
5 | 5 | <a href="https://xingezhu.me">Xinge Zhu</a>, |
6 | 6 | <a href="https://tai-wang.github.io">Tai Wang</a>, |
7 | 7 | <a href="https://yuexinma.me/aboutme.html">Yuexin Ma</a> |
8 | | - |
9 | | - |
10 | 8 | </p> |
11 | | - <h3 align="center"><a href="https://arxiv.org/pdf/2312.03774.pdf">Paper</a> <a </h3> |
12 | | - <div align="center"></div> |
| 9 | + <p align="center"> |
| 10 | + <a href="https://arxiv.org/pdf/2312.03774.pdf"><strong>Paper</strong></a> |
| 11 | + </p> |
| 12 | +</p> |
13 | 13 |
|
14 | 14 | ## Main Idea |
| 15 | + |
15 | 16 | <p align="center"> |
16 | | - <a href=""> |
17 | | - <img src="assets/teaser.png" alt="Relit" width="75%"> |
18 | | - </a> |
19 | | -</p> |
20 | | -<p align="center"> |
21 | | - OctreeOcc employs octree queries to offer varying granularity for distinct semantic regions, thereby diminishing the requisite number of queries for modeling and mitigating the issue of low information density in 3D space. |
| 17 | + <img src="assets/teaser.png" alt="OctreeOcc Teaser" width="80%"> |
22 | 18 | </p> |
23 | 19 |
|
24 | | -## Architecture overview |
| 20 | +OctreeOcc employs octree queries to offer varying granularity for distinct semantic regions, thereby diminishing the requisite number of queries for modeling and mitigating the issue of low information density in 3D space. |
| 21 | + |
| 22 | +## Architecture Overview |
25 | 23 |
|
26 | 24 | <p align="center"> |
27 | | - <a href=""> |
28 | | - <img src="./assets/pipeline.png" alt="Pipeline" width="99%"> |
29 | | - </a> |
30 | | -</p> |
31 | | -<p align="center"> |
32 | | - Given multi-view images, we extract multi-scale image features utilizing an image backbone. Subsequently, the initial octree structure is derived through image segmentation priors, and the transformation of dense queries into octree |
33 | | -queries is effected. Following this, we concomitantly refine octree queries and rectify the octree structure through the octree encoder. Finally, we decode from the octree query and obtain occupancy prediction outcomes for this frame. For better visualisation, the diagram of Iterative Structure Rectification module shows octree query and mask in 2D form(quadtree). |
| 25 | + <img src="./assets/pipeline.png" alt="Pipeline" width="100%"> |
34 | 26 | </p> |
35 | 27 |
|
| 28 | +Given multi-view images, we extract multi-scale image features utilizing an image backbone. Subsequently, the initial octree structure is derived through image segmentation priors, and the transformation of dense queries into octree queries is effected. Following this, we concomitantly refine octree queries and rectify the octree structure through the octree encoder. Finally, we decode from the octree query and obtain occupancy prediction outcomes for this frame. For better visualisation, the diagram of Iterative Structure Rectification module shows octree query and mask in 2D form (quadtree). |
| 29 | + |
36 | 30 | ## Performance |
37 | 31 |
|
38 | 32 | <p align="center"> |
39 | | - <a href=""> |
40 | | - <img src="./assets/exp_1.png" alt="main_res" width="99%"> |
41 | | - <img src="./assets/exp_2.png" alt="eff_res" width="75%"> |
42 | | - </a> |
| 33 | + <img src="./assets/exp_1.png" alt="Main Results" width="100%"> |
43 | 34 | </p> |
| 35 | + |
44 | 36 | <p align="center"> |
45 | | - Experiments conducted on the Occ3D-nuScenes dataset demonstrate that our approach enhances performance while substantially decreasing computational overhead (even when compared to 2D modeling approaches). |
| 37 | + <img src="./assets/exp_2.png" alt="Efficiency Results" width="80%"> |
46 | 38 | </p> |
47 | 39 |
|
| 40 | +Experiments conducted on the Occ3D-nuScenes dataset demonstrate that our approach enhances performance while substantially decreasing computational overhead (even when compared to 2D modeling approaches). |
| 41 | + |
48 | 42 | ## Visualization |
49 | 43 |
|
50 | 44 | <p align="center"> |
51 | | - <a href=""> |
52 | | - <img src="./assets/vis.png" alt="vis" width="99%"> |
53 | | - </a> |
54 | | -</p> |
55 | | -<p align="center"> |
56 | | - Qualitative results on Occ3D-nuScenes validation set. The first row displays input multi-view images, while the second row showcases the occupancy prediction results of PanoOcc, FBOCC, our methods, and the ground truth |
| 45 | + <img src="./assets/vis.png" alt="Visualization" width="100%"> |
57 | 46 | </p> |
58 | 47 |
|
59 | | -## Next Step |
| 48 | +Qualitative results on Occ3D-nuScenes validation set. The first row displays input multi-view images, while the second row showcases the occupancy prediction results of PanoOcc, FBOCC, our methods, and the ground truth. |
| 49 | + |
| 50 | +## Coming Soon |
| 51 | + |
| 52 | +- Detailed guidance documents |
| 53 | +- Data generation scripts |
| 54 | + |
| 55 | +Stay tuned for updates. Thank you for your interest in our work! |
60 | 56 |
|
61 | | -The code will be released after the paper is accepted. |
62 | 57 |
|
| 58 | +## Citation |
| 59 | +``` |
| 60 | +@article{lu2024octreeocc, |
| 61 | + title={Octreeocc: Efficient and multi-granularity occupancy prediction using octree queries}, |
| 62 | + author={Lu, Yuhang and Zhu, Xinge and Wang, Tai and Ma, Yuexin}, |
| 63 | + journal={Advances in Neural Information Processing Systems}, |
| 64 | + volume={37}, |
| 65 | + pages={79618--79641}, |
| 66 | + year={2024} |
| 67 | +} |
| 68 | +``` |
0 commit comments