Exploring an Ensemble-Based Approach to Atmospheric Climate Modeling and Testing at Scale
In this paper, the utility of a ensembles based testing framework is demonstrated; it was shown that running many short simulations is more efficient than running a single long simulation. It was also demonstrated that aggressive optimizations can lead to simulations with a statistically distinct model state. Also, averaged behavior of many single year runs of the atmosphere are statistically different than one long run, demonstrating atmosphere low-frequency variability.
Ensembles based testing provides faster verification to model developers when developing at scale. With this testing, developers can now more effectively use multicore computing systems when performing development model simulations.
A strict throughput requirement has placed a cap on the degree to which we can depend on the execution of single, long, fine spatial grid simulations to explore global atmospheric climate behavior. Whereas, running an ensemble of short simulations is computationally more efficient. We test the null hypothesis that the climate statistics of a full-complexity atmospheric model derived from an ensemble of independent short simulation is equivalent to that from an equilibrated long simulation. The climate of short simulation ensembles is statistically distinguishable from that of a long simulation in terms of the distribution of global annual means, largely due to the presence of low-frequency atmospheric intrinsic variability in the long simulation. We also find that model climate statistics of the simulation ensemble are sensitive to the choice of compiler optimizations. While some answer-changing optimization choices do not effect the climate state in terms of mean, variability and extremes, aggressive optimizations can result in significantly different climate states.