Drought is a persistent and formidable environmental threat across North America. Extensive research has demonstrated that La Niña events favor drought conditions over southwestern North America through well-established teleconnections. Additionally, soil moisture plays a crucial role in shaping these droughts through its intrinsic memory and land-atmosphere interactions. Despite these insights, quantifying the contributions to drought in these regions due to La Niña conditions and soil moisture in the seasons leading up to a La Niña event remains challenging. Furthermore, the potential for anthropogenically-driven soil aridification to drive more frequent high-impact events, such as the 2020-21 western U.S. drought, is insufficiently studied. Using the CESM2 large ensemble, we quantify how summer soil moisture and winter ENSO conditions influence second-summer agricultural droughts. We identified all severe La Niña events with December Nino3.4 values that are 1.5 standard deviations below the mean, and composited these events into five ensembles based on different soil moisture conditions in the first summer for both historical and future periods. Our major findings are: 1) soil moisture memory is longer by several months in drier hydroclimate regimes compared to wetter ones, and similarly so in deeper soil layers relative to shallow layers; 2) wetter antecedent soil in the preceding summer cannot completely counteract La Niña-driven droughts in the second summer over these regions, but it can mitigate their severity; 3) La Niña-induced droughts with drier antecedent soil moisture can be twice as severe as those with wetter antecedent soil moisture in some dry regions, such as Arizona, New Mexico, and western Texas; 4) anthropogenic warming-induced soil aridification increases the likelihood of drier soil conditions in the summer preceding a severe La Niña event, potentially enhancing the possibility of multi-year droughts. Our findings have important implications for agricultural drought prediction with potential lead times of more than one year and for understanding the driving mechanisms of multiyear droughts in southwestern North America in the warming world.