The level set based active contour model (LSAC) has been proved advantageous for image segmentation. Based on LSAC techniques, a novel algorithm was proposed to overcome the difficulties of image segmentation in infrared human detection systems. It consists of a motion-based LSAC module, a threshold-based LSAC module and a fusion module. The motion-based LSAC, which bridges level set and background-subtraction techniques, conducts foreground segmentation and background estimation simultaneously based on converged level set functions. It works for detecting the moving regions in a sequence. Moreover, its output is regarded as the input of the threshold-based LSAC, which combines level set and thresholding techniques. This threshold-based LSAC module has the ability to extract the image regions having intensities within the range specified by dual thresholds and works for detecting all possible regions that may contain human candidates. Finally, the third module fuses the LSAC outputs and results in faithful segmentation result owing to the morphological open reconstruction. Furthermore, the fast numeric scheme proposed for evolving the LSAC modules and the optimized algorithmic flow improves efficiency. Experimental results demonstrate that the algorithm enjoys better performance in accuracy, efficiency and robustness to camera movement and temporal changes in the scene in comparison with the rival algorithms.