Home
Publications
Photo Gallery
Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection
Tianyu Wang, Xiaowei Hu, Zhengzhe Liu, Chi-Wing Fu
NeurIPS · 2022
Research summary
A framework to boost 3D detection by learning to densify point clouds in latent space, improving detection of small and distant objects.
Abstract
LiDAR-produced point clouds are the major source for most state-of-the-art 3D object detectors. Yet, small, distant, and incomplete objects with sparse or few points are often hard to detect. We present Sparse2Dense, a new framework to efficiently boost 3D detection performance by learning to densify point clouds in latent space. Specifically, we first train a dense point 3D detector (DDet) with a dense point cloud as input and design a sparse point 3D detector (SDet) with a regular point cloud as input. Importantly, we formulate the lightweight plug-in S2D module and the point cloud reconstruction module in SDet to densify 3D features and train SDet to produce 3D features, following the dense 3D features in DDet. So, in inference, SDet can simulate dense 3D features from regular (sparse) point cloud inputs without requiring dense inputs. We evaluate our method on the large-scale Waymo Open Dataset and the Waymo Domain Adaptation Dataset, showing its high performance and efficiency over the state of the arts.
Key insight
Learns to densify sparse 3D point cloud features in latent space, significantly improving LiDAR-based detection of small and distant objects without requiring denser sensor input at inference time.
arXiv version linked; publication venue as listed above
Paper & resources
BibTeX
@inproceedings{wang2022sparse2dense,
title={Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection},
author={Wang, Tianyu and Hu, Xiaowei and Liu, Zhengzhe and Fu, Chi-Wing},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
volume={35},
pages={38533--38545},
year={2022}
}