Satellite cloud observations have become an indispensable tool for evaluating general circulation models (GCMs). To facilitate the satellite and GCM comparisons, the CFMIP (Cloud Feedback Model Inter-comparison Project) Observation Simulator Package (COSP) has been developed and is now increasingly used in GCM evaluations. Real-world clouds and precipitation can have significant sub-grid variations, which, however, are often ignored or oversimplified in the COSP simulation. In this study, we use COSP cloud simulations from the Super-Parameterized Community Atmosphere Model (SPCAM5) and satellite observations from the Moderate Resolution Imaging Spectroradiometer (MODIS) and CloudSat to demonstrate the importance of considering the sub-grid variability of cloud and precipitation when using the COSP to evaluate GCMsimulations. We carry out two sensitivity tests: SPCAM5 COSP and SPCAM5-Homogeneous COSP. In the SPCAM5 COSP run, the sub-grid cloud and precipitation properties from the embedded cloud-resolvingCE1 model (CRM) of SPCAM5 are used to drive the COSP simulation, while in the SPCAM5-Homogeneous COSP run only grid-mean cloud and precipitation properties (i.e., no sub-grid variations) are given to the COSP.We find that the warm rain signatures in the SPCAM5 COSP run agree with the MODIS and CloudSat observations quite well. In contrast, the SPCAM5-Homogeneous COSP run which ignores the sub-grid cloud variations substantially overestimates the radar reflectivity and probability of precipitation compared to the satellite observations, as well as the results from the SPCAM5 COSP run. The significant differences between the two COSP runs demonstrate that it is important to take into account the sub-grid variations of cloud and precipitation when using COSP to evaluate the GCM to avoid confusing and misleading results.