According to the requirement of large data transmission and arithmetic operation, a real-time infrared image processing system based on FPGA and DSP framework is designed. In the system, the DSP is used as the main image processing unit and the FPGA is used to control data acquisition and transmission logically. The implementation of data acquisition, data processing and data transmission in the multi-task mechanism of DSP/BIOS is discussed in detail. The experimental result shows that the design is reasonable and feasible. It is of great value in practical applications.