With the development and the application of visible-IR image fusion techniques in color vision night, evaluating the performance of image-fused algorithms is becoming an important aspect. In this study, three basic visual evaluation factors, target detection, details and colorfulness, were presented for evaluating perceptual quality of color fused images, and the degree of correlation between the perceptual quality and the three evaluation factors was investigated. Visual evaluation experiment on the color images fused visible and IR images of ten scenes by four fusion algorithms was conducted. The experimental results show that the fuesd images have good performances in target detection, the correlation coefficient between the perceptual qulity and details is 0.89, and the correlation coefficient between the perceptual qulity and colorfulness is 0.75 . It indicates that details are the main factor and the colorfulness plays an important role when targets are beyong the detected level.