The covariability of temperature (T), precipitation (P) and radiation (R) is an important aspect in understanding the climate influence on crop yields. Here, we analyze county-level corn and soybean yields and observed climate for the period 1983–2012 to understand how growing-season (June, July and August) mean T, P and R influence crop yields jointly and in isolation across the CONterminous United States (CONUS). Results show that nationally averaged corn and soybean yields exhibit large interannual variability of 21% and 22%, of which 35% and 32% can be significantly explained by T and P, respectively. By including R, an additional of 5% in variability can be explained for both crops. Using partial regression analyses, we find that studies that ignore the covariability among T, P, and R can substantially overestimate the sensitivity of crop yields to a single climate factor at the county scale. Further analyses indicate large spatial variation in the relative contributions of different climate variables to the variability of historical corn and soybean yields. The structure of the dominant climate factors did not change substantially over 1983–2012, confirming the robustness of the findings, which have important implications for crop yield prediction and crop model validations.