VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection - PyTorch

PyTorch implementation of the paper VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection based on previous work in this repo.

The file todo.sh contains all build instructions, so either run it with ./todo.sh or copy paste the following into your shell.

docker build -t voxelnet .

docker volume create --opt type=none \
                     --opt o=bind \
                     --opt device=. \
                     vxlnt-vol

docker run --gpus all -it \
           -p 5001:6006 \
           --name voxelnet_cnt \
           -v vxlnt-vol:/home/PyTorch-YOLOv3/:rw \
           voxelnet

cd VoxelNet-pytorch

python demo.py

Introduction

This is an unofficial implementation of VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection in pytorch. A large part of this project is based on the work here

Dependencies

• python3.5+
• pytorch (tested on 0.3.1)
• opencv
• shapely
• mayavi

Installation

1. Clone this repository.
2. Compile the Cython module for box_overlaps

$ python3 setup.py build_ext --inplace

3. Compile the nms model

$ python3 nms/build.py

Data Preparation

1. Download the 3D KITTI detection dataset from here. Data to download include:

   • Velodyne point clouds (29 GB): input data to VoxelNet
   • Training labels of object data set (5 MB): input label to VoxelNet
   • Camera calibration matrices of object data set (16 MB): for visualization of predictions
   • Left color images of object data set (12 GB): for visualization of predictions

2. In this project, the cropped point cloud data for training and validation. Point clouds outside the image coordinates are removed.

$ python3 data/crop.py

3. Split the training set into training and validation set according to the protocol here.

└── DATA_DIR
       ├── training   <-- training data
       |   ├── image_2
       |   ├── label_2
       |   ├── velodyne
       |   └── crop
       └── testing  <--- testing data
       |   ├── image_2
       |   ├── label_2
       |   ├── velodyne
       |   └── crop

Train

$ python3 train.py

TODO

• training code
• data augmentation
• validation code
• reproduce results for Car, Pedestrian and Cyclist
• multi-gpu support
• improve the performances