基于边缘保持和注意力生成对抗网络的红外与可见光图像融合
投稿时间:2020-10-29  修订日期:2020-12-05  点此下载全文
引用本文:
摘要点击次数: 149
全文下载次数: 0
作者单位E-mail
朱雯青 中国科学院上海技术物理研究所
中国科学院大学
中国科学院红外探测与成像技术重点实验室 
fyzhuwenqing@163.com 
汤心溢 中国科学院上海技术物理研究所
中国科学院红外探测与成像技术重点实验室 
gq227@mail.sitp.ac.cn 
张瑞 中国科学院上海技术物理研究所
中国科学院大学
中国科学院红外探测与成像技术重点实验室 
 
陈潇 中国科学院上海技术物理研究所
中国科学院大学
中国科学院红外探测与成像技术重点实验室 
 
苗壮 中国科学院上海技术物理研究所
中国科学院大学
中国科学院红外探测与成像技术重点实验室 
 
中文摘要:由于红外与可见光图像特征差异大,并且不存在理想的融合图像监督网络学习源图像与融合图像之间的映射关系,深度学习在图像融合领域的应用受到了限制。针对此问题,提出了一个基于注意力机制和边缘损失函数的生成对抗网络框架,应用于红外与可见光图像融合。通过引入对抗训练和注意力机制的思想,将融合问题视为源图像和融合图像对抗的关系,并结合了通道注意力和空间注意力机制学习特征通道域和空间域的非线性关系,增强了显著性目标特征表达。同时提出了一种基于边缘的损失函数,将源图像与融合图像像素之间的映射关系转化为图像边缘之间的映射关系。多个数据集的测试结果表明,该方法能有效融合红外目标和可见光纹理信息,锐化图像边缘,显著提高图像清晰度和对比度。
中文关键词:图像融合  生成对抗网络  边缘损失  通道注意力  空间注意力
 
Infrared and visible image fusion based on edge-preserving and attention generative adversarial network
Abstract:Infrared and visible image features are quite different, and there are no ideal fused images supervise neural networks to learn the mapping relationship between the source images and the fused images, thus, the application of deep learning is limited to the field of image fusion. To solve this problem, a generative adversarial network framework based on attention mechanism and edge loss is proposed, which is applied to the infrared and visible image fusion. Derived from the thoughts of attention mechanism and adversarial training, the fusion problem is regarded as an adversarial game between the source images and the fused images, and combining channel attention and spatial attention mechanism can learn the nonlinear relationship between channel domain features and spatial domain features, which enhances the expression of salient target features. At the same time, an edge-based loss function is proposed, which converts the mapping relationship between the source image pixels and the fused image pixels into the mapping relationship between the image edges. Experiments on multiple datasets demonstrate that this method can effectively fuse infrared target and visible texture information, sharpen image edges, and significantly improve image clarity and contrast.
keywords:image fusion, generative adversarial network, edge-based loss function, channel attention, spatial attention
  HTML  查看/发表评论  下载PDF阅读器

版权所有:《红外与毫米波学报》编辑部