Estimating crowd density instead of detecting people: 330,165 training examples

Tracking crowd density matters for the safety and management of cities and events: access routes, choke points, capacity limits. The information is valuable when it is available promptly – an analysis after the event is documentation, not a basis for control.

The obvious route – detecting and counting every person individually – fails on two counts. At high density it is technically unreliable, because people occlude each other. And it collects more information than the task requires.

1,198 images of crowds, in which 330,165 heads are marked as training labels – on average around 275 people per image.

The marking is a head position, not an outline of the person. That is already part of the method: a density can be derived from points without a person ever being modelled as an object.

Everything varies at once. Appearance, perspective, lighting, density and distribution differ from image to image. A shot from head height and one from a rooftop show the same crowd completely differently – in the first, people occlude each other; in the second, they shrink to a few pixels.

Very many objects per image. With several hundred people per shot, individual object detection is not just difficult but the wrong approach: every error multiplies with the count.

Estimate density instead of counting objects. We implemented a density-based approach with the CSRNet architecture, working per pixel: the model predicts a person density for every pixel, and the sum across the image yields the count. No individual person has to be delineated – the approach sidesteps the occlusion problem instead of solving it.

The method is also the privacy-friendly variant. A density model produces no person detection and no identifying features. It delivers the number needed for crowd management, and nothing more. For an application in public space, that is the precondition for deploying it at all.

The tool estimates crowds in images and enables monitoring in near real time.

The transferable value showed quickly outside the original purpose: the same method counts plants for stand density, and cells or bacteria in microscopy images. The task is structurally identical – very many small, partly occluded objects in one image.

Density estimation is the right approach whenever the count is needed and the individual objects are too small, too many or too heavily occluded to delineate one by one: stand density in agriculture, cell and colony counting in the lab, animal counts in remote sensing, piece counts in manufacturing. Exactly this closeness makes the use case evidence for our image-based analysis in screening and diagnostics application field too.

The architecture would look different today – transformer-based models estimate density more reliably across changing perspectives, and for counting tasks there are methods that transfer to a new object class with a few examples. For the laboratory applications, specialised models are now established, and we work with them in image analysis. What has changed markedly is the legal framework, and for this application it is decisive. The EU AI Act regulates remote biometric identification in public spaces strictly. A density model does not fall under it, because it identifies no persons – but that has to be documented and demonstrable in the architecture, not merely asserted. Which is exactly why the decision back then for density estimation over person detection is worth more today than it was: it was justified on technical grounds, and it has since become the variant that stands up in regulatory terms as well.