The causes for nonuniformity of infrared imaging systems are theoretically studied in the following three aspects: the response characteristics of detectors, the relative radiation of optical systems and the stray light from system housings. In an Infrared Focal Plane Array (IRFPA), the difference of structures and materials of detection elements may affect their quantum efficiency and dark current and hence affect their response characteristics. The optical system mainly affects the scene flux distribution through its pupil distortion matrix. The contribution of the radiation from the optical system itself to the nonuniformity can be ignored. The stray light from the housing can be divided into the high temperature part and the low temperature part. By ignoring the low temperature part and introducing the concept of System Cold Shield Efficiency (SCSE), a high temperature component radiation model is established. This theoretical research is of a certain referential value to the design and nonuniformity correction of optical systems.