In order to realize the FLIR and visual image registration effectively, an automatic registration algorithm based on sensor parameters and the refined CPD algorithm was proposed. Firstly, geometric rectification based on the attitude angle and height parameters was carried out to eliminate the rotation and scale discrepancies between the FLIR and visual images. Then the edges of visual image and rectified infrared image were extracted and a refined CPD algorithm was proposed for point set registration, the similarity transformation was adopted for fine image registration. Finally, the experiments on real FLIR data show that the proposed algorithm can realize the registration of infrared and visual images effectively and the registration precision can be around one pixel.