Evaluation of the performance of a computer-aided diagnosis (CAD) system based on the quantified color distribution in strain elastography imaging to evaluate the malignancy of breast tumors. The database consisted of 31 malignant and 52 benign lesions. A radiologist blinded to the diagnosis, performed the visual analysis of the lesions. After six months with no eye contact on the breast images, the same radiologist and other two radiologists drew manually the contour of the lesions in B-mode ultrasound, which was masked in the elastography image. In order to measure the amount of hard tissue in a lesion, we developed a CAD system able to identify the amount of hard tissue, represented by red color, and quantify its predominance in a lesion, allowing classifying it as soft, intermediate or hard. The data obtained with the CAD system were compared with the visual analysis. We calculated the sensitivity, specificity and AUC for the classification using the CAD system from the manual delineation of the contour by each radiologist. The performance of the CAD system for the most experienced radiologist achieved sensitivity of 70.97%, specificity of 88.46%, and AUC of 0.853. The system presented better performance compared to his visual diagnosis, whose sensitivity, specificity and AUC were 61.29, 88.46 and 0.829, respectively. The system obtained sensitivity, specificity and AUC of 67.70%, 84.60% and 0.783 for images segmented by Radiologist 2, and 51.60%, 92.30% and 0.771 for those segmented by the Resident. The intra-class correlation coefficient was 0.748. The interobserver agreement of the CAD system with the different contours were good in all comparison. The proposed CAD system can improve the radiologist performance for classifying breast masses, with excellent interobserver agreement. It could be a promising tool for clinical use.