Modern AI technologies develop quickly. Recently, there have been many advances in the field of deep learning on 3D point clouds. In many industries new beneficial possibilities arise – one of them is the road and highway sector. This Use Case describes a follow up project of ‘The automated labeling of point clouds from highway scans’ with the Bavarian State Construction Administration which regularly carries out LiDAR surveys of the Bavarian road network. This serves the process of planning, construction, and operation of road infrastructure.
Tasks of processing and information extraction of those LiDAR surveys usually require lots of labour work and expensive infrastructure to handle the large amounts of data. One of the most time-consuming tasks is the generation of CAD models for curb stones, road markings and many other structural elements from the laser scans.
The goal of our this project was to automate the generation of CAD models from point clouds with the help of deep learning.
For this task, high resolution mobile mapping LiDAR scans of two German highways and manually generated CAD models, including 3D polylines were provided by Cloud-Vermessung + Planung GmbH. The used vehicle is a conventional MB V class. The Measure unit was the Trimble MX9 with built-in Riegl scanners. All the acquired data was then imported into AutoDesk Civil 3D and merged into one 3D vector model.
The main challenge was related to object classes resulting from the semantic context. In a common CAD mapping, objects are represented in different ways and enriched by human prior knowledge. For example, the roadway is marked by outer lines. Based on the viewer’s knowledge of the relationship between the roadway and road markings, a distinction is made between left, middle and right lanes. Such distinctions, which only result from the context of different objects, cannot be directly taught to a neural network. It is also important to mention that CAD annotations do not work as training data directly. The neural network is a map function that projects each data point from the input distribution into an N-dimensional space, where N is the number of classes to be recognized. An object class is assigned to each laser scan point. However, the neural network cannot recognize which points make up a CAD relevant line. In the picture on the right the lines that are relevant for a CAD model are visible.
CAD lines of a highway section done by Cloud-Vermessung + Planung GmbH
In the beginning, the provided manually generated CAD outlines needed to be pre-processed, so that the neural network could recognize them as training data. Using our inhouse pre-trained 3D neural network, we finetuned the model to detect the desired road outlines from LiDAR highway scans. The trained AI model was able to detect road outlines and generate CAD models in unseen highway scans with a lower than centimetre precision.
The whole training and inference pipeline was modelled in a Microsoft Azure cloud environment. The pipeline is built completely modular and can be scaled by adding additional workers. Like this the processing of hundreds of kilometres of highway can be achieved in parallel for upcoming highway surveys.
Our trained 3D neural network can now automatically detect road outlines and generate CAD models from highway scans with sub-centimetre accuracy. As seen in the screenshots the ground truth and prediction are precisely overlayed with only minimal deviations. Any small deviations might also be human errors in the ground truth.
The speedup and cost savings compared to the manual process are tremendous for the Bavarian State Construction Administration while also minimizing human error in the process. In that way, future processing of surveys on the highway can be conducted automatically with minimal manual effort and time, which allows for a more precise and efficient planning, construction, and maintenance of the road infrastructure.
For now, we can trigger the processing and CAD model generation manually. For the future we are working on a direct deployment of these models to the Pointly platform. This way, the models would be available to all users and can be applied to their scans on demand.