An infrared target extraction algorithm based on the energy of a target is proposed. By utilizing the phenomenon that the image may smear when the relative angular velocity of a target exceeds the instantaneous field of view, the algorithm enhances the signal-to-noise ratio of the target and background through energy accumulation. Thus, a remote infrared target with the signal-to-noise ratio less than or equal to 5 can be detected and tracked. The algorithm has a small computation amount and is easy to be implemented on hardware. The experimental result shows that this algorithm can detect and track an infrared target which is 15 kilometers away with a detection probability of 96%.