For decades, traditional process-based geoscientific models cannot efficiently absorb information from large datasets, and therefore could not evolve rapidly given increasing data, but they offer advantages in interpretability and physical consistency. Recently, purely data-driven Deep learning (DL) has emerged to support a much more rapid pace of learning from data and thus improved accuracy, but deep networks remain challenging to interpret and do not readily facilitate the formulation and asking of desired science questions. Process-based modeling and DL are often perceived as distinct paradigms, but we argue that their apparent chasm mainly arises from a core differentiator of DL, which critically supports learning complex functional relationships. Here we propose differentiable modeling in geosciences (DG), a paradigm that peels off architectural elements of DL and marries its core elements to geoscientific process descriptions. We demonstrate examples of DG in hydrology, ecosystem, and water quality simulations. Preliminary evidence suggests that differentiable geoscientific models can approach the performance of purely data-driven DL models and even generalize better in space and time, while providing a full suite of observed or unobserved variables and thus a full narrative to stakeholders. With physical principles providing the connective tissues, we can precisely place our question marks at desired places in the modeling system, opening new paths to gaining knowledge