基于红外双目视觉的同步目标检测与匹配网络
作者:
作者单位:

1.中国科学院上海技术物理研究所,上海 200083;2.中国科学院大学,北京 100049

作者简介:

通讯作者:

中图分类号:

基金项目:

国家重点研发计划(2023YFB3905400)


Synchronous Object Detection and Matching Network Based on Infrared Binocular Vision
Author:
Affiliation:

1.Shanghai Institute of Technical Physics of the Chinese Academy of Sciences, Shanghai 200083, China;2.University of Chinese Academy of Sciences, Beijing 100049, China

Fund Project:

Supported by the National Key Research and Development Program of China (2023YFB3905400)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    特殊环境下道路目标的三维感知对汽车的全天候、全气候自动驾驶具有重要意义,红外双目视觉模仿人眼实现微光/无光等特殊环境下目标的立体感知,目标检测与匹配是双目视觉立体感知的关键技术。针对当前分步实现目标检测与目标匹配的过程冗杂问题,提出了一个可以同步检测与匹配红外目标的深度学习网络SODMNet(Synchronous Object Detection and Matching Network)。SODMNet创新的融合了目标检测网络和目标匹配模块,以目标检测网络为主要架构,取其分类与回归分支深层特征为目标匹配模块的输入,与特征图相对位置编码拼接后通过卷积网络输出左右图像特征描述子,根据特征描述子之间的欧式距离得到目标匹配结果,实现双目视觉目标检测与匹配。与此同时,采集并制作了一个包含人、车辆等标注目标的夜间红外双目数据集。实验表明,SODMNet在该红外双目数据集上的目标检测精度mAP(Mean Average Precision)提升84.9%以上,同时目标匹配精度AP(Average Precision)达到0.5777。结果证明,SODMNet能够高精度地同步实现红外双目目标检测与匹配。

    Abstract:

    The three-dimensional perception of road objects in challenging environments is crucial for the development of autonomous vehicles capable of operating in all conditions, at all hours. Infrared binocular vision mimics the human binocular system, facilitating stereoscopic perception of objects in challenging conditions such as dim or zero-light environments. The core technology for stereoscopic perception in binocular vision systems lies in accurate object detection and matching. To streamline the complex sequence of object detection and matching procedures, a synchronous object detection and matching network (SODMNet) is proposed, which is capable of synchronous detection and matching of infrared objects. SODMNet innovatively combines a object detection network with a object matching module, leveraging the deep features from the classification and regression branches as inputs for the object matching module. By concatenating these features with relative position encodings from the feature maps and processing them through a convolutional network, the network generates feature descriptors for the left and right images. The object matching is then achieved by calculating the Euclidean distances between these descriptors, thus facilitating synchronous object detection and matching in binocular vision. In addition, a novel nighttime infrared binocular dataset, annotated with targets such as pedestrians and vehicles, is created to support the development and evaluation of the proposed network. Experimental results indicate that SODMNet achieves a significant improvement of over 84.9% in object detection mean average precision (mAP) on this dataset, with a object matching average precision (AP) of 0.5777. These results demonstrate that SODMNet is capable of high-precision, synchronized object detection and matching in infrared binocular vision, marking a significant advancement in the field.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-04-17
  • 最后修改日期:2024-09-27
  • 录用日期:2024-05-21
  • 在线发布日期: 2024-09-25
  • 出版日期:
文章二维码