Many climate models struggle with a poor simulation of the Madden‐Julian Oscillation (MJO), especially its propagation across the Maritime Continent (MC). This study quantitatively evaluates the robustness of MJO propagation over the MC in climate models that participated in Coupled Model Intercomparison Project Phase 5 (CMIP5) and Phase 6 (CMIP6) with a newly developed MC propagation metric. The results show that the CMIP6 models simulate MJO propagation over the MC more realistically than the CMIP5 models. Lower free‐tropospheric moisture budget analysis highlights that the greater horizontal moisture advection is responsible for the enhanced MJO propagation over the MC. The increase in horizontal moisture advection in the CMIP6 models is mainly attributed to the steeper horizontal mean state moisture gradient around the MC, which is associated with the reduction of the equatorial dry bias.