To overcome the deficiency of the robustness and real-time performance of traditional Kalman filter used for changeable target tracking, a new Kalman algorithm based on multi-scale feature extraction was proposed. After the feature points of a frame matched ones of the follow-up frame in the target area of image, the latter feature points centroid was took as the center from which the searching area was located so as to avoid traversing the whole image. So the stable signals and residuals of observations were provided to the Kalman filter equations to calculate accurately the posteriori state value. Experiment show that the multi-scale feature extraction technology introduced into the traditional Kalman filter equation as constrained conditions reduced filtering time and restrained divergence. Thus improved filter has good convergence.