Mesoscale convective system (MCS) is an important component of the Earth’s hydrological cycle. MCSs are responsible for most of the tropical precipitation and often produce extreme weather such as flood, hail and wind damage. Properly simulating MCSs in Earth System Models (ESM) is crucial in projecting their future changes and associated extremes. Global convection-permitting models with km-scale horizontal grid spacing is a promising future for ESM. In this work, we systematically examine MCSs simulated by global km-scale models in the DYAMOND project (Stevens et al. 2019) by organizing an international initiative called MCSMIP (MCS tracking Method Intercomparison Project). Results from seven different feature trackers show that DYAMOND models in both phases generally underestimate observed MCS precipitation and their contribution to total precipitation by ~10% (land) and ~30% (ocean) in the tropics. MCS cloud shield evolution is better simulated than precipitation characteristics, though most models reproduce the MCS diurnal cycle well over both land and ocean. Models show a wide range of precipitable water vapor in the tropics compared to reanalysis, but many models overestimate observed MCS precipitation sensitivity to PWV, regardless of tracker formulations. We also found that two versions of SCREAM (v0 vs. v1) produce notably different MCS characteristics and sensitivity of precipitation to PWV. Ongoing work diagnosing the role of MCS on air-sea interactions that may be responsible for tropical PWV biases will be discussed.