Elsevier

Ecological Modelling

Volume 359, 10 September 2017, Pages 80-91
Ecological Modelling

Sensitivity analysis and Bayesian calibration for testing robustness of the BASGRA model in different environments

https://doi.org/10.1016/j.ecolmodel.2017.05.015Get rights and content

Highlights

  • The parameters to be fixed were consistent across sites.

  • Model calibration must be performed separately for each specific case.

  • Possible to reduce model parameters from 66 to 45.

  • Strong model reductions must be avoided.

  • The error term for the training data were characterised by timing (phase shift).

Abstract

Proper parameterisation and quantification of model uncertainty are two essential tasks in improvement and assessment of model performance. Bayesian calibration is a method that combines both tasks by quantifying probability distributions for model parameters and outputs. However, the method is rarely applied to complex models because of its high computational demand when used with high-dimensional parameter spaces. We therefore combined Bayesian calibration with sensitivity analysis, using the screening method by Morris (1991), in order to reduce model complexity by fixing parameters to which model output was only weakly sensitive to a nominal value. Further, the robustness of the model with respect to reduction in the number of free parameters were examined according to model discrepancy and output uncertainty. The process-based grassland model BASGRA was examined in the present study on two sites in Norway and in Germany, for two grass species (Phleum pratense and Arrhenatherum elatius). According to this study, a reduction of free model parameters from 66 to 45 was possible. The sensitivity analysis showed that the parameters to be fixed were consistent across sites (which differed in climate and soil conditions), while model calibration had to be performed separately for each combination of site and species. The output uncertainty decreased slightly, but still covered the field observations of aboveground biomass. Considering the training data, the mean square error for both the 66 and the 45 parameter model was dominated by errors in timing (phase shift), whereas no general pattern was found in errors when using the validation data. Stronger model reduction should be avoided, as the error term increased and output uncertainty was underestimated.

Introduction

Grassland covers about 70% of the world’s agricultural area (FAO). It has a central role in feeding ruminants and other herbivores, and the growing demand for meat may induce an even more intensive use in the future.

Complex dynamic growth models are increasingly used to simulate the interactions between vegetation and environment. Such models are useful in order to forecast yield, study the effect of climate change on yield, optimize management and to better understand the system. It is common to apply the same model in different regions and for different species and cultivars, and it should work well in all the situations for which it is applied. This requires that it is properly parameterised, and that parameters and output uncertainty are well quantified. Among parameter estimation methods, Bayesian calibration (Berger, 1985) has the advantage that it, in addition to calibrating the parameter values, simultaneously quantifies parameter uncertainty (Campbell, 2006). It achieves this by calculating posterior parameter distributions as a function of the original parameter uncertainty (prior knowledge) and new information incorporated through the conditional probability distribution of the collected data (likelihood function). The method is still rarely used for complex models, but its application has been increasing in recent years (Gouache et al., 2013, Minunno et al., 2013, Thorsen and Höglind, 2010, van Oijen et al., 2005a, van Oijen et al., 2005b, Kennedy and O'Hagan, 2001).

To estimate all the parameters of complex, parameter rich models simultaneously is often challenging. A major problem is the large computational effort required to investigate a high dimensional parameter space. As a result, predictive performance may be poor suggesting a need for model simplification (Cox et al., 2006). A study by Crout et al. (2014) identified several redundant variables in the Sirius wheat model. Here we focus on a different form of model simplification: reducing the number of free parameters in the model. Sensitivity analysis, or parameter screening, is a useful tool for model reduction that can make it easier and less time requiring to parameterise models by detecting the least sensitive parameters. These are parameters that can be fixed within their prior parameter boundaries without strongly affecting model robustness. Robustness is here referred to as the extent the model results are affected with when reducing the number of free parameters, where model results include the uncertainty in model outputs caused by parameter uncertainty. A simplification of a model by fixing the poorly sensitive parameters to nominal values will increase the efficiency of model calibration, but also result in underestimation of parameter uncertainty, since the parameter values that are fixed are not known for certain. A combination of sensitivity analysis and Bayesian calibration of a complex model was given by Raj et al. (2016), whereas the effect of model reduction on model uncertainty was not covered.

Study of the mismatch (error term) between observed and simulated model output is a widely used procedure for model evaluation. A detailed analysis of the error term, decomposing it into the three components of bias, variance error and phase shift, was proposed by Kobayashi and Salam (2000). Their method is still rarely used (but see van Oijen et al., 2011, Ewert et al., 2002), yet it adds valuable information about model behaviour.

The process-based BASGRA (BASic GRAssland) model is used in this study. It is a model that simulates growth of Phleum pratense (L.) (Höglind et al., 2001, Thorsen et al., 2010, van Oijen et al., 2005a). BASGRA contains 66 parameters and is driven by the environmental variables air temperature, precipitation, relative humidity, global radiation and wind-speed at a daily resolution. It calculates 23 state variables of which 13 quantify the state of the plant and 10 represent the above- and belowground environment. Only one output variable, aboveground biomass, is the focus of this study. This is one of the most often measured variables in grassland research.

The general objective of this study was to examine the robustness of aboveground biomass predictions by the grassland model BASGRA. The impact of parameter screening and subsequent parameter reduction on aboveground biomass predictions were quantified in order to allow efficient quantification of output uncertainty. The specific objective of this study was to identify a minimum number of parameters required for the BASGRA model in order to estimate both the value of aboveground biomass and its uncertainty with sufficient accuracy, consistent between sites and species.

Four sets of data were used: (1) total aboveground biomass of Phleum pratense (P. pratense) grown at Særheim, Norway, observed at intervals of 1–2 weeks throughout the growing season including at the agricultural harvests, (2) observations (two per year) of biomass yield from the same experiment, (3) observations (three per year) of biomass yield from a mixed sward dominated by P. pratense grown at Rengen, Germany and (4) observations (two per year) of biomass yield from a mixed sward dominated by Arrhenatherum elatius (A. elatius) grown at Rengen, Germany. Model performance had been tested thoroughly for P. pratense growth at Særheim by (van Oijen et al., 2005a) and the full dataset of that study was used here for model training. The datasets from Rengen were further split up into one training and one test dataset.

Section snippets

Grassland growth model

The BASGRA (BASic GRAssland) model simulates the growth of grassland swards for any period of time (a short growing cycle, a sequence of growing cycles, a winter period, a sequence of whole years etc.). The model is based on the LINGRA model (Schapendonk et al., 1998), but differs in that it simulates the dynamics of both vegetative and reproductive tillers (Höglind et al., 2001, van Oijen et al., 2005a) and that it includes processes which occur during winter (Thorsen and Höglind, 2010,

Sensitivity analysis

The sensitivity analysis explored the space within the prior parameter boundaries (Table S2), and was performed using the Morris method with 2000 trajectories and 4 levels. It was applied separately to the dataset at Særheim and Rengen.

Discussion

Process-based growth models, as the BASGRA model, are usually parameter rich. Satisfactory simplification of such models has previously been shown (Oomen et al., 2016, Raj et al., 2016). Based on the sensitivity analysis performed in this study, reduction of the number of model parameters seems possible for the BASGRA model as well. Results from the analysis showed large differences between the impact of parameters on model output, which is consistent with similar studies of other parameter

Conclusion

The objective of this study was to examine the impact of parameter screening and subsequent parameter reduction on aboveground biomass predictions by the grassland model BASGRA, in order to efficiently be able to include uncertainty in model outputs. According to this study, a reduction of model parameters from 66 to 45 was possible. The error term, for both the 45 and the fully parameterised model was characterised by the timing (phase shift) when considering the training data, while no

Acknowledgements

This work was funded by the Research Council of Norway and was conducted within the framework of the Modelling European Agriculture with Climate Change for Food Security (MACSUR) knowledge hub within the Joint Programming Initiative for Agriculture, Climate Change, and Food Security (FACCE-JPI).

Jürgen Schellberg acknowledges funding from the SPACES project “Limpopo Living Landscapes” (01LLL1304C) funded by Federal Ministry of Education and Research.

References (37)

Cited by (0)

View full text