A More Powerful Reality Test for Climate Models
This new analysis package is designed to provide objective summaries that compare climate models with observations. It produces well-established statistical comparions that are frequently used in many aspects of climate modeling and research, and will now be available for hundreds of simulations that are publically available.
This analysis package will be applied to all models that are included in the Coupled Model Intercomparison Project (CMIP). The resulting statistical summaries will be available to modeling groups soon after their simulations are made public, giving them more rapid feedback for their contribututions to CMIP than previously possible. The results will also be accessible by the broader research community for exploration and make the process of model-data comparisons more transparent and accessible.
This work introduces the PCMDI Metrics Package (PMP v1.1) to the research community. The PMP leverages the vast CMIP data archive and uses well-established statistical error measures to objectively compare results from climate model simulations to observations. It consists of four components: analysis software, an observationally based collection of global or near-global observations, a database of performance metrics computed from all models contributing to CMIP, and usage documentation. The PMP was prototyped with mean climate statistics, as emphasized in the current release. A diverse suite of additional tests are being implemented including summary statistics for sea ice distribution, land surface vegetation characteristics, three-dimensional structure of ocean temperature and salinity, monsoon onset and withdrawal, the diurnal cycle of precipitation, major modes of climate variability, and selected “emergent constraints”. A primary purpose for the PMP is to provide a diverse suite of objective summaries for all CMIP DECK and historical simulations made available through the ESGF. The PMP is well suited for documenting performance summaries of CMIP simulations and making them readily available in support research and model development. The PMP framework is extensible and transparent and will readily enable the CMIP research community to contribute analysis routines. Additionally, several CMIP modeling groups have already incorporated the PMP into the their internal analysis workflow and many others have expressed interest in doing so. This will enable modeling groups to rapidly compare their current model version(s) with CMIP models rather than await feedback from the external analysis community. The information provided by the PMP can help identify deficiencies in a model, which can be useful in setting priorities for further model development work. The PMP uses the Python programming language and DOE supported Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT), a powerful software tool kit that provides cutting-edge diagnostic and visualization capabilities. The PMP is open source, and its entire code is readily available at https://github.com/PCMDI/pcmdi_metrics.