As an important part of a Precision Guided Weapon (PGW) system, the infrared tracking system is mainly used for search, recognition, aiming and tracking on a moving carrier. Because the perturbation of the moving carrier has effect on the performance of the infrared tracking system, it is necessary to isolate the perturbation so as to eliminate its effect on tracking. By studying the operation principle of the infrared imaging system, the factors which might cause infrared images to be perturbed are found. The effect of perturbation on an infrared tracking system is analyzed qualitatively. Finally, some suggestions on how to overcome perturbation when the infrared tracking system is used are made.