Although soil erosion plays an important role in the global carbon cycle and affects the well being of aquatic and marine ecosystems, few earth system models include process-based representations of the transport of sediments and particulate organic carbon from land to rivers and streams. Two critical challenges that such development faces are scale and heterogeneity because the dominant processes and environmental factors for soil erosion can differ dramatically among catchments of varying sizes and climatic and geologic zones. Analyzing the observed sediment and particulate organic carbon (POC) yield at several hundred small catchments (< 200 km²) across the globe, we found that for the spatial scale of earth system models, runoff-driven erosion is more dominant than rainfall-driven erosion and soil deposition must be included. Human activities including agriculture and soil conservation can disturb the key soil erosion processes, thus contributing to a large fraction of variability in the observations. More importantly, we find that event-scale climatic parameters (e.g., peak runoff) have much larger correlations with sediment yield than annual-scale climatic parameters. This suggests that the use of event-scale soil erosion model may achieve better performance in earth system modeling. We further find that POC yield is tightly correlated with sediment yield so modeling of POC and sediment yield can be achieved using the same framework. Lastly, using knowledge from our analysis, we developed a global scale regression model for sediment yield based on several significant parameters that can be readily implemented in earth system models.