This article describes a software suite that can be used for objective evaluation of tropical cyclones (TCs) in gridded climate data. Using cyclone trajectories derived from 6-hourly data, a comprehensive set of metrics is defined to systematically compare and contrast products with one another. In addition to annual TC climatologies, attention is paid to spatial and temporal patterns of storm occurrence and intensity. Assessment can be performed either on the global scale or for regional domains. Simple-to-visualize “scorecards” allow for rapid credibility assessment. We showcase three key findings enabled by this suite. First, we compare the representation of TCs in seven current-generation global reanalyses and conclude that higher-resolution models and those with TC-specific assimilation contain more accurate storm climatologies. Second, using a free-running Earth system model (ESM) we find that full basin refinement is required in variable-resolution configurations to adequately simulate North Atlantic Ocean TC frequency. Upstream refinement over northern Africa offers little benefit in simulating storm occurrence, but spatial genesis patterns are improved. We also show that TCs simulated by ESMs can be highly sensitive to individual parameterizations in climate models, with North Atlantic TC metrics varying greatly depending on the version of the Morrison–Gettelman microphysics package that is used.