Use of PETSc and libCEED to achieve algorithmic and hardware portability in developing a river dynamical core for E3SM
Compound flooding (CF) poses significant risks to human and natural systems. It can impair basic infrastructure and threaten lives and livelihood. The drivers of CF, including rain-on-snow, sea-level rise, storm surges, and intensity and frequency of precipitation, are all expected to change in the future climate. However, several shortcomings in the current version of the Department of Energy's (DOE's) Energy Exascale Earth Model (E3SM) limit the use of E3SM to gain a predictive understanding of CF. The lack of portability of the current E3SM's river model on DOE's exascale-class supercomputers, which have heterogeneous computing architectures, poses another challenge. A SciDAC5-funded project aims to overcome the shortcomings of E3SM's river model by developing a rigorously verified and validated river dynamical core (RDycore) for E3SM, which includes efficient and scalable solvers for DOE's exascale-class supercomputers, for the study of CF and its impacts on sediment dynamics and riverine saltwater intrusion in a changing climate. RDycore, which solves 2D shallow water equations using first-order finite volume spatial discretization, uses PETSc and libCEED to achieve algorithmic and hardware portability. PETSc provides multiple time integrators that can be selected at runtime, the combination of PETSc and libCEED ensures RDycore can run efficiently on CPUs and GPUs. An initial integration of RDycore within E3SM has been completed and the coupled model has been successfully run on NVIDIA GPUs (Perlmutter) and AMD GPUs (Frontier). This presentation will provide an overview of the model integration and preliminary computational performance results for NVIDIA and AMD GPUs.