基于边缘保持和注意力生成对抗网络的红外与可见光图像融合
作者:
作者单位:

1.中国科学院上海技术物理研究所,上海 200083;2.中国科学院大学,北京 100049;3.中国科学院红外探测与成像技术重点实验室,上海 200083

作者简介:

通讯作者:

中图分类号:

TP391.41

基金项目:


Infrared and visible image fusion based on edge-preserving and attention generative adversarial network
Author:
Affiliation:

1.Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China;2.University of Chinese Academy of Sciences, Beijing 100049, China;3.Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    由于红外与可见光图像特征差异大,并且不存在理想的融合图像监督网络学习源图像与融合图像之间的映射关系,深度学习在图像融合领域的应用受到了限制。针对此问题,提出了一个基于注意力机制和边缘损失函数的生成对抗网络框架,应用于红外与可见光图像融合。通过引入对抗训练和注意力机制的思想,将融合问题视为源图像和融合图像对抗的关系,并结合了通道注意力和空间注意力机制学习特征通道域和空间域的非线性关系,增强了显著性目标特征表达。同时提出了一种边缘损失函数,将源图像与融合图像像素之间的映射关系转化为边缘之间的映射关系。多个数据集的测试结果表明,该方法能有效融合红外目标和可见光纹理信息,锐化图像边缘,显著提高图像清晰度和对比度。

    Abstract:

    Infrared and visible image features are quite different, and there are no ideal fused images supervise neural networks to learn the mapping relationship between the source images and the fused images. Thus, the application of deep learning is limited to the field of image fusion. To solve this problem, a generative adversarial network framework based on attention mechanism and edge loss is proposed, which is applied to the infrared and visible image fusion. Derived from the thoughts of attention mechanism and adversarial training, the fusion problem is regarded as an adversarial game between the source images and the fused images, and combining channel attention and spatial attention mechanism can learn nonlinear relationship between channel domain features and spatial domain features, which enhances the expression of salient target features. At the same time, an edge-based loss function is proposed, which converts the mapping relationship between the source image pixels and the fused image pixels into the mapping relationship between the edges. Experimental results on multiple datasets demonstrate that the proposed method can effectively fuse infrared target and visible texture information, sharpen image edges, and significantly improve image clarity and contrast.

    参考文献
    相似文献
    引证文献
引用本文

朱雯青,汤心溢,张瑞,陈潇,苗壮.基于边缘保持和注意力生成对抗网络的红外与可见光图像融合[J].红外与毫米波学报,2021,40(5):696~708]. ZHU Wen-Qing, TANG Xin-Yi, ZHANG Rui, CHEN Xiao, MIAO Zhuang. Infrared and visible image fusion based on edge-preserving and attention generative adversarial network[J]. J. Infrared Millim. Waves,2021,40(5):696~708.]

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-10-29
  • 最后修改日期:2021-09-06
  • 录用日期:2020-12-16
  • 在线发布日期: 2021-09-06
  • 出版日期:
文章二维码