The PCMDI Metrics Package (PMP) is an open-source Python-based capability used to produce and document objective summaries of Earth System Model (ESM) agreement with observations. This is accomplished via a diverse suite of relatively robust high-level summary statistics across space and time scales. These results can be used to quantify model improvements across generations (e.g., CMIP1-CMIP6) and to identify the relative strengths and weaknesses of different models. In this presentation we highlight new metrics that have been incorporated into the PMP, with interest in the possible reduction of systematic biases in the newer generation of (CMIP6) models. Some of these results are based on recent PCMDI research, but a particular emphasis is on collaborations with an assortment of international expert teams. Examples include metrics for extratropical modes of variability, ENSO, MJO, monsoons and high frequency characteristics of simulated precipitation. Leveraging over twenty years of DOE supported Python-based analysis software (including the Community Data Analysis Tools; CDAT), an important objective of the PMP is to create end-to-end provenance to ensure the benchmarking of CMIP class models is fully documented, traceable, and reproducible across all variants of input data and the tools used to analyse them. The PCMDI Metrics Package is available at https://github.com/PCMDI/pcmdi_metrics.

Acknowledgement: This work was supported by the Office of Science (BER), U.S. Department of Energy through Lawrence Livermore National Laboratory contract DE-AC52-07NA27344.