Instance segmentation on 3D point clouds: single objects from scenes of up to ten million points

Photogrammetric point clouds from drone flights have long been routine in construction and planning. What is missing is the step from the point cloud to countable information.

At the time of the project, the available methods were largely limited to classifying single, small point clouds – one object, one scan. But for use in construction and planning, the whole scene is needed: a construction site, a street, an area, with tens of millions of points and many objects of the same class.

Photogrammetric point clouds from urban and semi-urban environments, often construction sites, captured through drone flights by our partner FairFleet. Individual scans contain up to ten million points.

The difference between classification and instance segmentation. A model that marks every point as a vehicle does not say how many vehicles stand there. Yet for counting, locating and further processing, that is exactly the information sought – and it requires objects of the same class to be separated from one another.

Scaling as the real problem. Ten million points cannot be processed in one pass. The approach has to decompose the scene and merge the partial results again without objects being counted twice or torn apart at the seams.

Unsupervised clustering combined with supervised training. The unsupervised method uses the spatial structure of the data itself to form coherent objects; the trained network delivers the class assignment. The combination is the core of the approach: the class has to be learned, the object boundary sits in the geometry – forcing both out of one method would be more laborious and harder to verify.

Designed for scalability rather than optimised for one dataset. The approach was intended for large scenes from the start, not for individual objects. That is why it remains applicable to new flights.

For individual object classes, accuracies of up to 93% were reached. The output is an instance segmentation: points are not only classified but separated into individual instances per class. That makes automatic counting and locating of objects possible.

The results can be transferred into geodatabases, map layers and AutoCAD models – the tools construction and planning already work in.

Urban infrastructure inventory · suitability analysis for solar installations · construction site monitoring · creation of 3D models

Instance segmentation on large point clouds is the foundation for everything that is to be counted, measured or tracked over time. We applied the same principle – classify first, then separate, then attribute – to trees in the automated forest inventory and to transport infrastructure in the highway scans. Today we deliver projects of this kind together with our sister company Pointly as our technology partner.

The approach was ahead of its time, and the tools have caught up. For instance segmentation on point clouds there are now methods that solve class assignment and object separation in one model, as well as promptable models that isolate objects without class-specific training. Among architectures, transformer-based point cloud models and sparse convolution methods have taken the place of the older networks. Scaling has changed the most: with Cloud-Optimized Point Cloud and PDAL, parts of a scene can be loaded and processed without opening the entire scan – the decomposition that had to be purpose-built back then is now a property of the data format. What has stayed the same: the object boundary sits in the geometry, not in the label. A method that leaves the separation to the network alone needs considerably more training data for the same result.