Simplifying Point Cloud Labeling with Contextual Images and Point Cloud Filtering

Annotating point clouds from multi-line 360° LiDAR is exceedingly difficult. Providing context in the form of camera frames and limiting the point cloud to the Field Of View (FOV) of the camera simplifies things. To achieve this, we first had to replace our old, and not so stable LiDAR mount, with a sturdier one capable of also holding a camera.

With the LiDAR and the camera closer together, the next step was to synchronize and store the data coming from both sensors. The collected data was huge, so a simple Python script was created to allow for the selection of sequences of interest. Once the sequences were visually selected, they were saved in a format that the annotation tool CVAT can understand.

It was noted that although now CVAT provided the camera frames as context to annotate the LiDAR point clouds, the point clouds were still too large (360° horizontally, 32° vertically). It was not easy to know which part of the cloud corresponded to the camera frame (visual context), and many objects were still hard to identify.

To solve the issue, a C++ program using the Point Cloud Library (PCL) was created. PCL’s FrustumCulling filter was used for this purpose. The filter lets the user define a vertical and horizontal Field Of View (FOV), as well as the near and far plane distance. After some testing, the best parameters were defined to approximate the FOV of the camera. The points of the input cloud that fall outside of the approximated FOV are filtered out, and the points of the output closely match what the camera sees. This greatly facilitates the annotation of objects in the point cloud. Watch the above video for more exciting details.