I will give an overview of our work developing cloud resolving and superparamterized versions of the E3SM model for DOE's upcoming GPU based Exascale computers. We have completed GPU ports of the atmosphere dynamical core (using C++/Kokks) and the SAM cloud resolving model (Fortran/openACC) used in the super-parameterized E3SM. After normalizing for power consumption, we see that modern GPUs (NVIDIA V100) can obtain moderate (~3x) speedups over the best CPUs for traditional Earth system models. The speedups are significant when compared to the near stagnant improvements in CPU performance in the post-Moore's law era.

These GPU speedups can only be obtained when running with very high workloads per node, such as found in ultra high-resolution and superparameterizated (SP) models. GPU systems will make short cloud resolving simulations affordable, but wont provide the orders of magnitude speedups necessary for multi-century climate simulations. For climate length simulations on GPUs, SP is a more promising approach. It has the potential to obtain the most critical aspects of a cloud resolving model with the high throughput necessary for multi-century simulation campaigns. The initial GPU version of the E3SM-SP atmosphere model, running at 25km global resolution with a 1km super-parameterization obtains close to 1 simulated-year-per-day. As others have shown, SP appears able to capture much of the mesoscale organization necessary for realistic upscaling. This aspect of SP is dramatically improved by E3SM-SP's 25km global resolution.