A new collaborative organization for sea ice model development, the CICE Consortium, has devised quality control procedures to maintain the integrity of its numerical codes’ physical representations, enabling broad participation from the scientific community in the Consortium’s open software development environment. Using output from five coupled and uncoupled configurations of the Los Alamos Sea Ice Model, CICE, we formulate quality control methods that exploit common statistical properties of sea ice thickness, and test for significant changes in model results in a computationally efficient manner. New additions and changes to CICE are graded into four categories, ranging from bit- for-bit amendments to significant, answer-changing upgrades. These modifications are assessed using a critical ice thickness threshold that accounts for the high level of autocorrelation in sea ice time series, along with a quadratic skill metric that searches for hemispheric changes in model answers across an array of different CICE configurations. These metrics also provide objective guidance for assessing new physical representations and code functionality.