The automatic labelling of point clouds from highway scans

Up till now the classification of point clouds was time-consuming as the work was done mainly manually and huge amounts of data need to be labelled. Furthermore, traditional methods lack precision or granularity. Nowadays, modern techniques make it possible to accelerate and digitise this process to a new level of precision. The goal of this infrastructure project was to fully automate point cloud labelling by using deep learning models.

For this task, high-resolution mobile mapping LiDAR scans of two German highways were provided by Cloud-Vermessung + Planung GmbH, which surveyed the road network for the Bavarian State Construction Administration. The used vehicle was a conventional MB V class. The measuring unit was the Trimble MX9 with built-in Riegl scanners. With this setup very high accuracies (+/- 1 cm) and very dense point, clouds were created. Also, the scanner captured buildings such as bridges around the highway and delivers point clouds including intensity values and other additional data that can be used for the deep learning task.

The classification of 3D point clouds from laser scans of highways is a manual and laborious activity. To make the process scalable and economically efficient it should be automated with the help of AI. One of the biggest challenges was the clear definition of object classes to collect suitable training data for the AI. There are 3 requirement criteria for this definition: Complete: Each data point must be assigned to a class Explicit: Object classes must be clearly distinguishable Representative: Classes should be represented as equally as possible No class should be seriously underrepresented In this project, the most important classes were street, road marking, ground, vegetation, rails, signs and vehicles. However, it was a demanding task to exactly distinguish a road from other asphalt surfaces and determine the beginning and end of particular objects (e.g. components of bridge structures).

Firstly, the provided datasets were labelled with Pointly by in-house employees. Here it is beneficial that Pointly lets you create customised classes. Like this, all the classes needed for the customer could be classified. Several kilometres of sampled highway sections were completely classified for the model training. With the generated training data, our pretrained 3D neural network was trained and finetuned to detect the different classes in the LiDAR Scans. Eventually, the trained network is used to classify new point clouds automatically with high precision corresponding to the custom object classes. All provided point clouds were classified within a few hours as they were parallel processed in a cloud environment.

The trained neural network works with a strong and convincing precision and speeds up the classification process tremendously. The existing point clouds were automatically classified by the AI and delivered to the customer. Upcoming scans can be processed almost without any manual labour within hours. Also, the model can be enhanced anytime by feeding it additional training data. In a follow-up project with the Bavarian State Construction Administration, the classified point clouds were used furthermore to automatically extract road outlines as 3D polylines and to create CAD models from these. Soon the classification models will be made available for everyone on the Pointly platform so that they can be easily used on demand.