Infrared and visible image fusion based on edge-preserving and attention generative adversarial network
Received:October 29, 2020  Revised:September 06, 2021  download
Citation:
Hits: 2
Download times: 4
Author NameAffiliationE-mail
ZHU Wen-Qing Shanghai Institute of Technical Physics Chinese Academy of Sciences Shanghai 200083 China
University of Chinese Academy of Sciences Beijing 100049 China
Key Laboratory of Infrared System Detection and Imaging Technology Chinese Academy of Sciences Shanghai 200083 China 
fyzhuwenqing@163.com 
TANG Xin-Yi Shanghai Institute of Technical Physics Chinese Academy of Sciences Shanghai 200083 China
Key Laboratory of Infrared System Detection and Imaging Technology Chinese Academy of Sciences Shanghai 200083 China 
gq227@mail.sitp.ac.cn 
ZHANG Rui Shanghai Institute of Technical Physics Chinese Academy of Sciences Shanghai 200083 China
University of Chinese Academy of Sciences Beijing 100049 China
Key Laboratory of Infrared System Detection and Imaging Technology Chinese Academy of Sciences Shanghai 200083 China 
 
CHEN Xiao Shanghai Institute of Technical Physics Chinese Academy of Sciences Shanghai 200083 China
University of Chinese Academy of Sciences Beijing 100049 China
Key Laboratory of Infrared System Detection and Imaging Technology Chinese Academy of Sciences Shanghai 200083 China 
 
MIAO Zhuang Shanghai Institute of Technical Physics Chinese Academy of Sciences Shanghai 200083 China
University of Chinese Academy of Sciences Beijing 100049 China
Key Laboratory of Infrared System Detection and Imaging Technology Chinese Academy of Sciences Shanghai 200083 China 
 
Abstract:Infrared and visible image features are quite different, and there are no ideal fused images supervise neural networks to learn the mapping relationship between the source images and the fused images. Thus, the application of deep learning is limited to the field of image fusion. To solve this problem, a generative adversarial network framework based on attention mechanism and edge loss is proposed, which is applied to the infrared and visible image fusion. Derived from the thoughts of attention mechanism and adversarial training, the fusion problem is regarded as an adversarial game between the source images and the fused images, and combining channel attention and spatial attention mechanism can learn nonlinear relationship between channel domain features and spatial domain features, which enhances the expression of salient target features. At the same time, an edge-based loss function is proposed, which converts the mapping relationship between the source image pixels and the fused image pixels into the mapping relationship between the edges. Experimental results on multiple datasets demonstrate that the proposed method can effectively fuse infrared target and visible texture information, sharpen image edges, and significantly improve image clarity and contrast.
keywords:image fusion  generative adversarial network  edge-based loss function  attention mechanism
View Full Text  HTML  View/Add Comment  Download reader

《Journal of Infrared And Millimeter Waves》