The respective roles of ocean and atmospheric dynamics in driving the Atlantic Multidecadal Variability (AMV) remain an open question, and imply vastly different timescales of predictability for North Atlantic climate. We investigate if AMV-like variability can emerge using a hierarchy of stochastic models that solely consider the local upper ocean response to stochastic atmospheric forcing. The seasonal variability and spatial patterns of the model parameters, including heat flux feedback, mixed-layer depth, and stochastic forcing amplitude, are estimated from long simulations using the Community Earth System Model 1 (CESM). Despite its simplicity, the stochastic model is able to reproduce many key characteristics of sea surface temperature (SST) variability in the North Atlantic compared to CESM with either slab and fully-dynamic ocean components. Our results show how the seasonal mixed-layer cycle’s control on the upper ocean thermal inertia is critical for determining seasonal-to-interannual persistence of SST. In contrast, vertical entrainment and re-emergence of subsurface temperature anomalies enhances SST variability at sub-annual and decadal frequencies, suggesting potential for longer-term predictability where this mechanism is active. We further examined the respective roles of each leading modes of atmospheric variability in generating the AMV pattern. While the centers of action for the canonical AMV pattern are roughly reproduced by the stochastic model, its magnitude remains overestimated in the subtropics and underestimated elsewhere. This not only leaves a role for ocean dynamics in driving AMV, but illuminates regional differences in the dominant dynamics and predictability.