While urban surface and near-surface air temperatures are known to be often higher than their rural counterparts, a phenomenon labeled as urban heat islands (UHIs), whether urban temperatures are more persistent than rural temperatures at time scales commensurate to heat waves has not been addressed despite the importance for human health. Combining numerical simulations by a global model with a surface energy balance theory, we demonstrate that urban surface and near-surface air temperatures are significantly more persistent than their rural counterparts in cities dominated by materials with large thermal inertia. Further use of these materials will result in even stronger urban temperature persistence, especially for tropical cities. Mitigation strategies that can simultaneously ameliorate the larger magnitude and stronger persistence of urban temperatures are recommended.