Sensitivity analysis and Bayesian calibration for testing robustness of the BASGRA model in different environments
Introduction
Grassland covers about 70% of the world’s agricultural area (FAO). It has a central role in feeding ruminants and other herbivores, and the growing demand for meat may induce an even more intensive use in the future.
Complex dynamic growth models are increasingly used to simulate the interactions between vegetation and environment. Such models are useful in order to forecast yield, study the effect of climate change on yield, optimize management and to better understand the system. It is common to apply the same model in different regions and for different species and cultivars, and it should work well in all the situations for which it is applied. This requires that it is properly parameterised, and that parameters and output uncertainty are well quantified. Among parameter estimation methods, Bayesian calibration (Berger, 1985) has the advantage that it, in addition to calibrating the parameter values, simultaneously quantifies parameter uncertainty (Campbell, 2006). It achieves this by calculating posterior parameter distributions as a function of the original parameter uncertainty (prior knowledge) and new information incorporated through the conditional probability distribution of the collected data (likelihood function). The method is still rarely used for complex models, but its application has been increasing in recent years (Gouache et al., 2013, Minunno et al., 2013, Thorsen and Höglind, 2010, van Oijen et al., 2005a, van Oijen et al., 2005b, Kennedy and O'Hagan, 2001).
To estimate all the parameters of complex, parameter rich models simultaneously is often challenging. A major problem is the large computational effort required to investigate a high dimensional parameter space. As a result, predictive performance may be poor suggesting a need for model simplification (Cox et al., 2006). A study by Crout et al. (2014) identified several redundant variables in the Sirius wheat model. Here we focus on a different form of model simplification: reducing the number of free parameters in the model. Sensitivity analysis, or parameter screening, is a useful tool for model reduction that can make it easier and less time requiring to parameterise models by detecting the least sensitive parameters. These are parameters that can be fixed within their prior parameter boundaries without strongly affecting model robustness. Robustness is here referred to as the extent the model results are affected with when reducing the number of free parameters, where model results include the uncertainty in model outputs caused by parameter uncertainty. A simplification of a model by fixing the poorly sensitive parameters to nominal values will increase the efficiency of model calibration, but also result in underestimation of parameter uncertainty, since the parameter values that are fixed are not known for certain. A combination of sensitivity analysis and Bayesian calibration of a complex model was given by Raj et al. (2016), whereas the effect of model reduction on model uncertainty was not covered.
Study of the mismatch (error term) between observed and simulated model output is a widely used procedure for model evaluation. A detailed analysis of the error term, decomposing it into the three components of bias, variance error and phase shift, was proposed by Kobayashi and Salam (2000). Their method is still rarely used (but see van Oijen et al., 2011, Ewert et al., 2002), yet it adds valuable information about model behaviour.
The process-based BASGRA (BASic GRAssland) model is used in this study. It is a model that simulates growth of Phleum pratense (L.) (Höglind et al., 2001, Thorsen et al., 2010, van Oijen et al., 2005a). BASGRA contains 66 parameters and is driven by the environmental variables air temperature, precipitation, relative humidity, global radiation and wind-speed at a daily resolution. It calculates 23 state variables of which 13 quantify the state of the plant and 10 represent the above- and belowground environment. Only one output variable, aboveground biomass, is the focus of this study. This is one of the most often measured variables in grassland research.
The general objective of this study was to examine the robustness of aboveground biomass predictions by the grassland model BASGRA. The impact of parameter screening and subsequent parameter reduction on aboveground biomass predictions were quantified in order to allow efficient quantification of output uncertainty. The specific objective of this study was to identify a minimum number of parameters required for the BASGRA model in order to estimate both the value of aboveground biomass and its uncertainty with sufficient accuracy, consistent between sites and species.
Four sets of data were used: (1) total aboveground biomass of Phleum pratense (P. pratense) grown at Særheim, Norway, observed at intervals of 1–2 weeks throughout the growing season including at the agricultural harvests, (2) observations (two per year) of biomass yield from the same experiment, (3) observations (three per year) of biomass yield from a mixed sward dominated by P. pratense grown at Rengen, Germany and (4) observations (two per year) of biomass yield from a mixed sward dominated by Arrhenatherum elatius (A. elatius) grown at Rengen, Germany. Model performance had been tested thoroughly for P. pratense growth at Særheim by (van Oijen et al., 2005a) and the full dataset of that study was used here for model training. The datasets from Rengen were further split up into one training and one test dataset.
Section snippets
Grassland growth model
The BASGRA (BASic GRAssland) model simulates the growth of grassland swards for any period of time (a short growing cycle, a sequence of growing cycles, a winter period, a sequence of whole years etc.). The model is based on the LINGRA model (Schapendonk et al., 1998), but differs in that it simulates the dynamics of both vegetative and reproductive tillers (Höglind et al., 2001, van Oijen et al., 2005a) and that it includes processes which occur during winter (Thorsen and Höglind, 2010,
Sensitivity analysis
The sensitivity analysis explored the space within the prior parameter boundaries (Table S2), and was performed using the Morris method with 2000 trajectories and 4 levels. It was applied separately to the dataset at Særheim and Rengen.
Discussion
Process-based growth models, as the BASGRA model, are usually parameter rich. Satisfactory simplification of such models has previously been shown (Oomen et al., 2016, Raj et al., 2016). Based on the sensitivity analysis performed in this study, reduction of the number of model parameters seems possible for the BASGRA model as well. Results from the analysis showed large differences between the impact of parameters on model output, which is consistent with similar studies of other parameter
Conclusion
The objective of this study was to examine the impact of parameter screening and subsequent parameter reduction on aboveground biomass predictions by the grassland model BASGRA, in order to efficiently be able to include uncertainty in model outputs. According to this study, a reduction of model parameters from 66 to 45 was possible. The error term, for both the 45 and the fully parameterised model was characterised by the timing (phase shift) when considering the training data, while no
Acknowledgements
This work was funded by the Research Council of Norway and was conducted within the framework of the Modelling European Agriculture with Climate Change for Food Security (MACSUR) knowledge hub within the Joint Programming Initiative for Agriculture, Climate Change, and Food Security (FACCE-JPI).
Jürgen Schellberg acknowledges funding from the SPACES project “Limpopo Living Landscapes” (01LLL1304C) funded by Federal Ministry of Education and Research.
References (37)
Statistical calibration of computer simulations
Reliabil. Eng. Syst. Safety
(2006)- et al.
An effective screening design for sensitivity analysis of large models
Environ. Modell. Softw.
(2007) - et al.
Comparison of sensitivity analysis techniques: a case study with the rice model WARM
Ecol. Modell.
(2010) - et al.
Sensitivity analysis of the rice model WARM in Europe: exploring the effects of different locations, climates and methods of analysis on model sensitivity to crop parameters
Environ. Modell. Softw.
(2010) Monte Carlo based sensitivity analysis of two crop simulators and considerations on model balance
Eur. J. Agron.
(2010)- et al.
Toward the systematic simplification of mechanistic models
Ecol. Modell.
(2006) - et al.
An objective apporach to model reduction: application to the Sirius wheat model
Agric. Forest Meteorol.
(2014) - et al.
Mapping grassland leaf area index with airborne hyperspectral imagery: a comparison study of statistical approaches and inversion of radiative transfer models
ISPR J. Photogr. Remote Sensing
(2011) - et al.
Effects of elevated CO2 and drought on wheat: testing crop simulation models for different experimental and climatic conditions
Agric. Ecosyst. Environ.
(2002) - et al.
Modelling climate change impact on Septoria tritici blotch (STB) in France: accounting for climate model and disease model uncertainty
Agric. Forest Meteorol.
(2013)