A novel ship detection method using cascaded convolutional neural network (CCNet) with multispectral image is proposed to achieve high-speed detection. The CCNet employs two cascaded convolutional neural networks (CNN) for extracting regions of interest (ROIs), locating and segmenting ship objects sequentially. Benefit from the abundant details of the multispectral image, CCNet can extract more robust feature for achieving more accurate detection. The efficiency of CCNet has been validated by the experiments on SPOT 6 satellite multispectral images. In comparison with the state-of-the-art deep learning based ship detection algorithms, the experimental results indicate that the proposed ship detection algorithm accelerates the processing by more than 5 times with a high accurate detection performance.