In power systems, to use computer vision and image processing technologies to detect the faults in arresters plays an important role in their safe operation. An arrester fault detection method based on infrared images is proposed. The algorithm firstly preprocesses the input images and uses Scale-Invariant Feature Transform (SIFT) descriptors and K-means++ algorithm to train a vision dictionary to precisely position the arrester. Then, it uses Linear Spectral Clustering (LSC) to segment the area selected. Finally, it implements the detection of arrester fault by analyzing the characteristics in the thermal image of the arrester. The experimental results show that the proposed algorithm can detect the faults in arresters effectively.