Infrared-NeRF:一种基于NeRF的低分辨率热红外光场3D重建方法
DOI:
作者:
作者单位:

南京大学

作者简介:

通讯作者:

中图分类号:

基金项目:

中央高校基本科研业务费(融合创新类 2024300443);国家自然科学基金(青年基金项目 No 62405131)


Infrared-NeRF: a low resolution thermal infrared light field 3D reconstruction method based on NeRF
Author:
Affiliation:

Nanjing University

Fund Project:

The Fundamental Research Funds for the Central Universities(Integrated Innovation Category 2024300443);the National Natural Science Foundation of China (NSFC) Young Scientists Fund (Grants No 62405131)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    本文针对低分辨率热红外场景提出了一种基于神经辐射场(NeRF)的三维光场重建方法Infrared-NeRF。结合低分辨率热红外成像特点,面向热红外三维重建的速度和精度提升问题开展了多方面优化。首先,受玻尔兹曼热辐射定律启发,首次将距离这一因素纳入NeRF模型中,使得单光线的传播呈非线性变化,更为准确地描述了红外辐射强度随距离增大而递减的物理性质。其次,在推理速度提升方面,基于红外图像中前景和背景呈高低频分布现象,提出了多光线非均匀光线合成策略,使得模型更关注场景中的前景物体,减少背景中光线的分布,在不降低精度的情况下大幅减少训练时间。此外,相较于可见光场景红外图像仅有单通道,因此只需要较少的网络参数。经过同一训练数据和数据筛选方法进行实验表明,相较原始NeRF,改进后的网络PSNR和SSIM平均提升了13.8%和4.62%,LPIPS平均降低了46%。并且得益于网络层数的优化和数据甄别方法,训练仅需耗费约原方法25%的时间即可达到收敛。最后,针对背景暗弱的场景,本文通过限制模型查询区间的方法使得模型的推理速度相较于原始NeRF提升了4-6倍。

    Abstract:

    This article proposes a three-dimensional light field reconstruction method based on neural radiation field (NeRF) called Infrared NeRF for low resolution thermal infrared scenes. Based on the characteristics of low resolution thermal infrared imaging, various optimizations have been carried out to improve the speed and accuracy of thermal infrared 3D reconstruction. Firstly, inspired by Boltzmann""s law of thermal radiation, distance was incorporated into the NeRF model for the first time, resulting in a nonlinear propagation of a single ray and a more accurate description of the physical property that infrared radiation intensity decreases with increasing distance. Secondly, in terms of improving inference speed, based on the phenomenon of high and low frequency distribution of foreground and background in infrared images, a multi ray non-uniform light synthesis strategy is proposed to make the model pay more attention to foreground objects in the scene, reduce the distribution of light in the background, and significantly reduce training time without reducing accuracy. In addition, compared to visible light scenes, infrared images only have a single channel, so fewer network parameters are required. Experiments using the same training data and data filtering method showed that compared to the original NeRF, the improved network achieved an average improvement of 13.8% and 4.62% in PSNR and SSIM, respectively, while an average decrease of 46% in LPIPS. And thanks to the optimization of network layers and data filtering methods, training only takes about 25% of the original method""s time to achieve convergence. Finally, for scenes with weak backgrounds, this article improves the inference speed of the model by 4-6 times compared to the original NeRF by limiting the query interval of the model.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-10-21
  • 最后修改日期:2024-12-09
  • 录用日期:2024-12-31
  • 在线发布日期:
  • 出版日期:
文章二维码