Decision-makers today have relatively easy web-based access to climate projections from several different models and downscaled datasets. Yet, there is minimal guidance on the credibility and appropriate use of such models and projections for specific adaptation contexts. The few studies that provide recommendations on model choice are often based on evaluations of broad physical climate metrics (such as temperature averages or extremes) at regional scales, without additional examination of local-scale decision-relevant climatic metrics (such as growing degree days or chill hours) that underpin the adaptation action. While such broad regional skill may be considered necessary for the overall credibility of models, it is not clear whether it is sufficient to ensure good skill for decision applications. This paper evaluates the skill of different Global Circulation Models (GCMs) in predicting the decision-relevant metric of chill hours in Fresno, California, and examines how model selection impacts future projections. We find that good skill in predicting broader physical climate metrics in California does not guarantee skill in prediction of chill hours in Fresno. In fact, the models with good regional climatic skill were mutually exclusive of the ones with good skill for chill hours, which leads to some counterintuitive results for this unique metric. Since many decision-relevant metrics are non-linear derivations of primary physical quantities (like the chill hour metric), more such decision-relevant model evaluations are needed to provide better insights on model credibility and choice for adaptation decisions.