Because the battlefield environment is complex and changes quickly, a fast detection/localization algorithm against shore island background is proposed. The algorithm can search for the infrared ship targets of interest, such as the ship docked at the port, without prior training. In this method, only a single target image is used to find the similar matching ship in the field of view, without learning the target to be searched for prior. A local regression kernel extracted from the image to be searched for is used as a descriptor for measuring the similarity of a pixel and its neighborhood. Then, the remarkable feature is extracted from the descriptor and is compared with the similar features in the target image. The comparison is conducted by using a cosine similarity matrix. The experimental results show that this algorithm is excellent in performance because it recognizes the ship target successfully against the complex background.