For target rotation in IR image sequence, an improved Mean shift tracking method was approached. Mean shift vector was established by Gaussian kernel with bandwidth matrix, and recursive equations were given in EM like manner. Flexibility of algorithm was improved by motion compensation of image sequence. Robustness of algorithm was improved by combining with Kalman filter. Experiments show that our method achieves success in IR target tracking, at same time the method can be adapted to the transformations of rotation and zooming.