Uncertainty analysis: An evaluation metric for synthesis science

The methods for conducting reductionist ecological science are well known and widely used.
In contrast, those used in the synthesis of ecological science (i.e., synthesis science) are still being
developed, vary widely, and often lack the rigor of reductionist approaches. This is unfortunate because the
synthesis of ecological parts into a greater whole is critical to understanding many of the environmental
challenges faced by society. To help address this imbalance in approaches, we examine how the rigor of
ecological synthesis science might be increased by using uncertainty as an evaluation metric—as a parallel
to methods used in reductionist science. To estimate and understand uncertainty we propose that it be
divided into four general classes: (1) measurement uncertainty (i.e., experimental error) as defined by
precision and accuracy, (2) sampling uncertainty that reflects natural variation in space and time as
quantified by classical statistical moments (e.g., mean and variance), (3) model prediction uncertainty
which relates to the transformation of measurements into other variables of interest (e.g., plant dimensions
to biomass), and (4) model selection uncertainty which relates to uncertainty about the form of the
relationships used in models. Of these sources of uncertainty, model selection is the least understood and
potentially, the most important, because it is integral to how components of a system are combined and it
reflects imperfect knowledge about these relationships. To demonstrate uncertainty in synthesis science, we
examine each source of uncertainty in an analysis that estimates the live tree biomass of a forest and how
knowledge of each source can improve future estimates. By quantifying sources of uncertainty in synthesis
science, it should be possible to make rigorous comparisons among results, to judge whether they differ
within the bounds of measurement and knowledge, and to assess the degree to which scientific progress is
being made. However, to be accepted as a standard method, best practices analogous to those used in
reductionist science need to be developed and implemented.
Key words: aboveground biomass; H. J. Andrews Experimental Forest; measurement uncertainty; model parameter
uncertainty; model selection uncertainty; Oregon, USA; sampling uncertainty; Special Feature: Uncertainty Analysis.