A semantic segmentation network based on deep learning is proposed. The network designs a spatial pyramid module which can extract multi-scale information from images through Atrous convolution. It also explores the influence of Atrous convolution sampling rate and multi-scale branches on the performance of network through extensive experiment. the impact of hyperparameters on network performance during training is discussed. The test results on the SUN RGB-D dataset show that compared with other state-of-the-art semantic segmentation networks, the performance of the network we proposed is outstanding. Finally, the semantic segmentation based on infrared images is explored preliminarily.