Memory Effect (ME) noise is a stripe noise which occurs in infrared opto-mechnical scanning images. It is particularly obvious for the scene with a bright-dark mutation in the image. It may affect the visual effectiveness of the image seriously. The traditional ME noise removal methods usually use a system impulse response function to build a restoration filter for restoring images. The disadvantage is that there should be an accuracy impulse response function. In view of the disadvantages of traditional methods, a new ME noise removal algorithm is proposed. In the algorithm, the radiation information of image self is used, a detection template is used to traverse the whole scene image so as to detect ME noise and an iterative method is used to remove the detected noise. Finally, the IRMSS images from CBERS01 satellite are selected to make experiment. The experimental results show that this method can remove ME noise effectively so that it can improve the radiation quality of images.