To implement the detection of infrared ship targets in the complex shore island background, an infrared ship target detection and recognition method based on multi-spectral fusion is proposed. Firstly, Nonsubsampled Contourlet Transform (NSCT) multi-level multi-spectral image fusion is performed according to the relationship between different spectral segments. Then, in combination with LSD line segment detection and clustering, shoreline is detected in the fusion image. A selective search algorithm is used to generate the initial target candidate region. The spatial location of the shoreline, the ship target geometric feature and the gray feature constraint are used to eliminate some false target regions. Finally, the feature descriptors in the Histogram of Oriented Gradient (HOG) are extracted in the candidate region. The linear Support Vector Machine (SVM) classifier is used to classify and identify the real ship targets. The experimental results show that compared with the single spectral band infrared ship target detection method, the proposed method has a better detection accuracy.