A global platform for weed control trials

Weeds impede the growth of cultivated plants and so damage yield directly. Agronomic practices combined with chemical herbicides can reduce that damage effectively – but which combination works under which regional conditions can only be answered through global field trials.

The goal was a full assessment of the cost and efficacy of the practices studied, from which recommendations could be derived. The route there ran through a data problem: the assessment data previously recorded was captured very heterogeneously across the different sites.

Assessment data from regional field trials in several countries, each in its own accumulated format – plus the cost data without which no economic assessment is possible.

Introduce a global format without losing the regions. A central platform required a uniform data format to be defined and the regional formats adapted to it. That is less a technical than an organisational problem: the format has to be built so that each region recognises its own recording practice within it.

Reports that adapt. The dashboards had to be adjustable by users themselves so that specific reports could be created and exported – rather than a fixed analysis that no longer fits the next question.

An adaptable capture template instead of a migration project. We designed a largely adaptable data capture template, migrated the existing data into it, and in doing so also settled how future assessments would be documented. The decisive point: the standardisation works forwards, not only backwards.

Automatic transformation in R. The data transformation pipeline was built in R with the wrangling package dplyr.

A central platform in Tibco Spotfire. It consolidates the various regional platforms and evaluates cost efficiency across regions. Dashboards and reports are accessible through a web browser.

Measurement results from several countries and their associated field trials were successfully imported into the central platform; further countries and future experiments are being added continuously. Existing charts and filters extend automatically as new data arrives – the platform does not have to be touched with every addition.

Farmers get a rapid economic assessment of individual weed control practices and of the use of specific products.

The further-reaching effect lies in the standardisation itself: because the data is captured uniformly and processed automatically, it can be analysed more deeply with machine learning methods and merged with other field trial results, weather data or more detailed product information. That opens up analyses which were previously impossible.

Bringing heterogeneous recording practice towards a common structure and placing a self-extending analysis on top of it is the task in every multi-centre study – in agriculture as in clinical research.

What we called a "global homogeneous data format" in 2021 is now called a data contract and is an established concept with its own tooling: schemas are versioned like code, and automated tests reject violations at the point of delivery rather than letting them surface in the analysis. Today we would build the transformation pipeline in dbt, with tests attached directly to the transformations, and store the data in a lakehouse format that serves several analysis tools at once – rather than tying it to one BI product. Capture itself would be validated at the point of origin, through mobile entry with plausibility checks in the field. And the promise of being able to merge the data with other sources later is no longer a prospect but the normal state of such a platform.