Classifying point clouds from highway scans automatically: from weeks of manual work to hours

Mobile mapping delivers as-built data at an accuracy that used to require a survey in the field. The bottleneck comes afterwards: a point cloud is initially a set of coordinates without meaning. Only classification turns points into road, marking, sign or vegetation – and thereby into a basis for as-built plans, condition assessment and planning.

Making that assignment by hand is a project in itself for several kilometres of highway. And it is a project that repeats with every new scan: the same work comes up again on the next section of route.

High-resolution mobile mapping LiDAR scans of two German highways, provided by Cloud-Vermessung + Planung GmbH, which surveyed the road network for the Bavarian State Construction Administration. Capture was done with a Trimble MX9 with built-in Riegl scanners; capture accuracy is ±1 cm, and alongside the geometry there are intensity values and the surrounding structures.

The training data was created in Pointly: several kilometres of highway were classified there by hand to give the network a reference.

The class definition decides the outcome, not the network. Before a model can be trained, the class catalogue has to meet three conditions: completeness – every point must be assignable to a class, with no remainder. Unambiguity – no point may fit two classes. Representativeness – every class must occur often enough in the training material. A catalogue that violates one of these conditions produces a model that is unusable at scale, and the error only surfaces after training.

The classes defined were road, road marking, ground, vegetation, rails, signs and vehicles.

A road is not the same as asphalt. The hardest distinction was between the carriageway and other asphalted surfaces – such as car parks, access roads, hard shoulders. Geometrically and in reflection intensity they are barely separable. What distinguishes them is their function, and that is not in the point but in the context.

Classify manually where it counts most. Instead of annotating the entire route, several kilometres were classified by hand in Pointly – selected so that all classes are sufficiently represented. The platform's selection tools cut this step from weeks to days.

Fine-tune a pretrained 3D network. We did not train from scratch but adapted a pretrained 3D neural network to the data at hand and the defined class catalogue. That lowers the need for annotated data considerably.

Parallel processing in the cloud. Classification runs in parallel in a cloud environment. Only that makes it possible to evaluate several kilometres in hours – on a single machine it remains a matter of days.

The trained network classifies existing point clouds within a few hours at high precision. For future scans of the same kind, manual work is almost entirely eliminated – the effort shifts from classification to review.

The real outcome is therefore not the classified point cloud but a reusable model: the next section of route does not start from zero again. The models are being made available through the Pointly platform.

In a follow-up project with the Bavarian State Construction Administration, the classified point clouds were also used to extract road outlines as 3D polylines and to automatically generate CAD models from them.

The procedure – define a class catalogue by completeness, unambiguity and representativeness, annotate a small part deliberately, fine-tune a pretrained network, evaluate in parallel in the cloud – carries beyond the highway: railways, utility corridors, interiors, industrial facilities, vegetation monitoring along infrastructure. Wherever new scans of the same type keep coming in, the model is the investment, not the individual evaluation. Today we deliver projects of this kind together with our sister company Pointly as our technology partner.

Nothing would change about the basic structure, quite a bit about the building blocks. On the architecture side, transformer-based point cloud models and sparse convolution methods take the place of older 3D networks; they capture spatial context across larger distances, which helps precisely with distinguishing carriageway from other asphalt. For annotation, promptable pre-segmentation is used: a model proposes objects, the domain expert corrects – which shortens the manual part considerably once more. And we would set up data storage cloud-natively, with Cloud-Optimized Point Cloud and PDAL, so that a scan no longer has to be loaded in full to evaluate one section. What has stayed the same: the class catalogue is the real domain work. No model repairs a class definition that is not complete, unambiguous and representative.