Multiple visual sensor fusion provides an effective way to improve the robustness and accuracy of visual surveillance system. This paper gives a review of moving object detection with visual-thermal fusion. Firstly, the problem is analyzed, including its procedure, key problems, advantages, testing data sets. Then the existing algorithms are analyzed according to several different rules: fusion strategy, detection method and fusion structure. By comparing different kinds of methods, we analyzed the advantage, disadvantage and applicability. Finally we conclude with discussion on difficulties, challenges and possible future research directions of this kind of method.