<\/div><\/div><\/span>Automated and user-friendly Data Analysis Pipeline for Agricultural Field Trials<\/h1><\/span>
Automated and user-friendly Data Analysis Pipeline for Agricultural Field Trials<\/h1><\/span><\/div><\/div><\/div>a Supper & Supper Use Case<\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div>
![](\"https:\/\/supperundsupper.com\/wp-content\/uploads\/2020\/07\/Harvester_560x420.jpg\" "\\"Harvester_560x420\\"")
<\/span><\/div><\/i><\/i><\/span>Project Goal<\/span><\/a><\/h4><\/div>\nThe goal of this project was to develop a tool incorporating an all-in-one automatic pipeline for agricultural data analysis, that integrates different database systems, visualization tools, training of machine learning models and statistical inference. The tool enables people from a non-technical background to initialize new analyses and help them to make business decisions easily.<\/div>\n<\/div><\/div><\/div><\/i><\/i><\/span>Challenges<\/span><\/a><\/h4><\/div>\nThe major challenge was to combine the diverse set of data processing tools. The realization required broad knowledge of data storage, data extraction, data loading and data analysis techniques. The second challenge was the interface design. The tool was designed primarily for non-tech users. Therefore the tool needed to be well documented and required an intuitive user-friendly interface.<\/p>\n<\/div><\/div><\/div>
<\/i><\/i><\/span>Applied Methods (Implementation)<\/span><\/a><\/h4><\/div>\n\n- Interface Design: The user interface was built in R-shiny which is an interactive web application (app) straight from R. Through the web interface users can explore the data visually and produce graphical and statistical output for different use cases without writing a single line of code.<\/li>\n
- Data Extraction and Wrangling: As the raw data is stored in a structured way, the developed tool can fetch the needed data automatically, from connected relational database manage systems, a Neo4j knowledge graph and different web services. Data wrangling and merging in R was realized using packages like dplyr, tidyr and stringr.<\/li>\n
- Data Visualization: With the integration of a JavaScript library, the tool generates interactive maps showing the trial locations and related information, such as weather data, soil information and more meta data.<\/li>\n
- Statistical Modeling: Random forest and XGBoosting are used for the modeling. The user gets an overview of the model performance (R-square), the importance of different factors and influences of individual factors as well as the interaction strength between factors.<\/li>\n<\/ul>\n
![](\"https:\/\/supperundsupper.com\/wp-content\/uploads\/2020\/07\/angewandte-Methoden-800x554-1.jpg\")
<\/a><\/span><\/div>\n<\/div><\/div><\/div>
<\/div><\/div><\/div><\/span><\/div><\/i><\/i><\/span>Project Goal<\/span><\/a><\/h4><\/div>
a Supper & Supper Use Case<\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div>
<\/span><\/div><\/i><\/i><\/span>Project Goal<\/span><\/a><\/h4><\/div>\nThe goal of this project was to develop a tool incorporating an all-in-one automatic pipeline for agricultural data analysis, that integrates different database systems, visualization tools, training of machine learning models and statistical inference. The tool enables people from a non-technical background to initialize new analyses and help them to make business decisions easily.<\/div>\n<\/div><\/div><\/div><\/i><\/i><\/span>Challenges<\/span><\/a><\/h4><\/div>\nThe major challenge was to combine the diverse set of data processing tools. The realization required broad knowledge of data storage, data extraction, data loading and data analysis techniques. The second challenge was the interface design. The tool was designed primarily for non-tech users. Therefore the tool needed to be well documented and required an intuitive user-friendly interface.<\/p>\n<\/div><\/div><\/div>
<\/i><\/i><\/span>Applied Methods (Implementation)<\/span><\/a><\/h4><\/div>\n\n- Interface Design: The user interface was built in R-shiny which is an interactive web application (app) straight from R. Through the web interface users can explore the data visually and produce graphical and statistical output for different use cases without writing a single line of code.<\/li>\n
- Data Extraction and Wrangling: As the raw data is stored in a structured way, the developed tool can fetch the needed data automatically, from connected relational database manage systems, a Neo4j knowledge graph and different web services. Data wrangling and merging in R was realized using packages like dplyr, tidyr and stringr.<\/li>\n
- Data Visualization: With the integration of a JavaScript library, the tool generates interactive maps showing the trial locations and related information, such as weather data, soil information and more meta data.<\/li>\n
- Statistical Modeling: Random forest and XGBoosting are used for the modeling. The user gets an overview of the model performance (R-square), the importance of different factors and influences of individual factors as well as the interaction strength between factors.<\/li>\n<\/ul>\n<\/a><\/span><\/div>\n<\/div><\/div><\/div>
\n
The goal of this project was to develop a tool incorporating an all-in-one automatic pipeline for agricultural data analysis, that integrates different database systems, visualization tools, training of machine learning models and statistical inference. The tool enables people from a non-technical background to initialize new analyses and help them to make business decisions easily.<\/div>\n<\/div><\/div><\/div><\/i><\/i><\/span>Challenges<\/span><\/a><\/h4><\/div>
<\/i><\/i><\/span>Challenges<\/span><\/a><\/h4><\/div>\nThe major challenge was to combine the diverse set of data processing tools. The realization required broad knowledge of data storage, data extraction, data loading and data analysis techniques. The second challenge was the interface design. The tool was designed primarily for non-tech users. Therefore the tool needed to be well documented and required an intuitive user-friendly interface.<\/p>\n<\/div><\/div><\/div>
<\/i><\/i><\/span>Applied Methods (Implementation)<\/span><\/a><\/h4><\/div>\n\n- Interface Design: The user interface was built in R-shiny which is an interactive web application (app) straight from R. Through the web interface users can explore the data visually and produce graphical and statistical output for different use cases without writing a single line of code.<\/li>\n
- Data Extraction and Wrangling: As the raw data is stored in a structured way, the developed tool can fetch the needed data automatically, from connected relational database manage systems, a Neo4j knowledge graph and different web services. Data wrangling and merging in R was realized using packages like dplyr, tidyr and stringr.<\/li>\n
- Data Visualization: With the integration of a JavaScript library, the tool generates interactive maps showing the trial locations and related information, such as weather data, soil information and more meta data.<\/li>\n
- Statistical Modeling: Random forest and XGBoosting are used for the modeling. The user gets an overview of the model performance (R-square), the importance of different factors and influences of individual factors as well as the interaction strength between factors.<\/li>\n<\/ul>\n<\/a><\/span><\/div>\n<\/div><\/div><\/div>
\n<\/i><\/i><\/span>Applied Methods (Implementation)<\/span><\/a><\/h4><\/div>
The major challenge was to combine the diverse set of data processing tools. The realization required broad knowledge of data storage, data extraction, data loading and data analysis techniques. The second challenge was the interface design. The tool was designed primarily for non-tech users. Therefore the tool needed to be well documented and required an intuitive user-friendly interface.<\/p>\n<\/div><\/div><\/div>
<\/i><\/i><\/span>Applied Methods (Implementation)<\/span><\/a><\/h4><\/div>\n\n- Interface Design: The user interface was built in R-shiny which is an interactive web application (app) straight from R. Through the web interface users can explore the data visually and produce graphical and statistical output for different use cases without writing a single line of code.<\/li>\n
- Data Extraction and Wrangling: As the raw data is stored in a structured way, the developed tool can fetch the needed data automatically, from connected relational database manage systems, a Neo4j knowledge graph and different web services. Data wrangling and merging in R was realized using packages like dplyr, tidyr and stringr.<\/li>\n
- Data Visualization: With the integration of a JavaScript library, the tool generates interactive maps showing the trial locations and related information, such as weather data, soil information and more meta data.<\/li>\n
- Statistical Modeling: Random forest and XGBoosting are used for the modeling. The user gets an overview of the model performance (R-square), the importance of different factors and influences of individual factors as well as the interaction strength between factors.<\/li>\n<\/ul>\n<\/a><\/span><\/div>\n<\/div><\/div><\/div>
\n
- \n
- Interface Design: The user interface was built in R-shiny which is an interactive web application (app) straight from R. Through the web interface users can explore the data visually and produce graphical and statistical output for different use cases without writing a single line of code.<\/li>\n
- Data Extraction and Wrangling: As the raw data is stored in a structured way, the developed tool can fetch the needed data automatically, from connected relational database manage systems, a Neo4j knowledge graph and different web services. Data wrangling and merging in R was realized using packages like dplyr, tidyr and stringr.<\/li>\n
- Data Visualization: With the integration of a JavaScript library, the tool generates interactive maps showing the trial locations and related information, such as weather data, soil information and more meta data.<\/li>\n
- Statistical Modeling: Random forest and XGBoosting are used for the modeling. The user gets an overview of the model performance (R-square), the importance of different factors and influences of individual factors as well as the interaction strength between factors.<\/li>\n<\/ul>\n<\/a><\/span><\/div>\n<\/div><\/div><\/div>
![](\"https:\/\/supperundsupper.com\/wp-content\/uploads\/2020\/07\/Trials_info.png\" "\\"Trials_info\\"")
<\/span><\/div>\n<\/div><\/div><\/div><\/div><\/div>