Climate models are frequently used tools for adaptation planning in light of future uncertainty. However, not all climate models are equally trustworthy, and so model biases must be assessed to select models suitable for producing credible projections. Drought is a well-known and high-impact form of extreme weather, and knowledge of its frequency, intensity, and duration key for regional water management plans. Droughts are also difficult to assess in climate datasets, due to the long duration per event, relative to the length of a typical simulation. Therefore, there is a growing need for a standardized suite of metrics addressing how well models capture this phenomenon. In this study, we present a widely applicable set of metrics for evaluating agreement between climate datasets and observations in the context of drought. Two notable advances are made in our evaluation system: First, statistical hypothesis testing is employed for normalization of individual scores against the threshold for statistical significance. And second, within each evaluation region and dataset, principal feature analysis is used to select the most descriptive metrics among 11 metrics that capture essential features of drought. Our metrics package is applied to three characteristically distinct regions in the conterminous US and across several commonly employed climate datasets (CMIP5/6, LOCA and CORDEX). As a result, insights emerge into the underlying drivers of model bias in global climate models, regional climate models, and statistically downscaled models.