Mesoscale convective systems (MCSs) consisting of an assembly of cumulonimbus clouds are the largest form of deep convective storms. MCSs contribute to 30%–70% of annual rainfall east of the Rocky Mountains in the U.S. and in the global tropics. MCSs are notoriously difficult to simulate in climate models with parameterized convection. Using the FLEXTRKR algorithm, MCSs have been tracked based on observations to develop benchmark datasets of MCS characteristics globally for evaluating simulations of MCSs in climate models. Applying FLEXTRKR to a 25-km resolution simulation produced by the Energy Exascale Earth System Model version 1 (E3SM v1), various aspects of MCSs and their large-scale environments are evaluated over the United States. Results show that the model produces a more realistic simulation of MCSs in spring than summer, although during both seasons MCS rain intensity is notably underpredicted. Biases in simulating MCSs are partly attributed to biases in their large-scale environments, including the upper-level jet and lower-level moisture. With recent developments towards E3SM v2 to improve the representation of deep convection and cloud microphysics, such as inclusion of mesoscale heating, sensitivity experiments are being performed at 25-km grid spacing to evaluate the impacts of the new developments on MCS characteristics. This study highlights the development and use of MCS metrics and diagnostics to quantify and understand model biases in simulating MCSs and their use in informing model development.