The mineralogy of desert dust is important due to its effect on radiation, clouds and biogeochemical cycling of trace nutrients. This study presents the simulation of dust radiative forcing as a function of both mineral composition and size at the global scale using mineral soil maps for estimating emissions. Externally mixed mineral aerosols in the bulk aerosol module in the Community Atmosphere Model version 4 (CAM4) and internally mixed mineral aerosols in the modal aerosol module in the Community Atmosphere Model version 5.1 (CAM5) embedded in the Community Earth System Model version 1.0.5 (CESM) are speciated into common mineral components in place of total dust. The simulations with mineralogy are compared to available observations of mineral atmospheric distribution and deposition along with observations of clear-sky radiative forcing efficiency. Based on these simulations, we estimate the all-sky direct radiative forcing at the top of the atmosphere as +0.05 Wm-2 for both CAM4 and CAM5 simulations with mineralogy and compare this both with simulations of dust in release versions of CAM4 and CAM5 (+0.08 and +0.17 Wm-2) and of dust with optimized optical properties, wet scavenging and particle size distribution in CAM4 and CAM5, -0.05 and -0.17 Wm-2, respectively. The ability to correctly include the mineralogy of dust in climate models is hindered by its spatial and temporal variability as well as insufficient global in-situ observations, incomplete and uncertain source mineralogies and the uncertainties associated with data retrieved from remote sensing methods.