Camera-LiDAR Fusion With Latent Correlation for Cross-Scene Place Recognition

As a crucial issue for outdoor mobile robot navigation, place recognition (PR) remains challenging in long-term cross-scene applications, necessitating enhanced robustness of PR algorithms. In this article, a novel multimodal PR approach is proposed, which deeply fuses the camera and LiDAR to effect...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on industrial electronics (1982) 2024-09, p.1-9
Main Authors: Pan, Yan, Xie, Jiapeng, Wu, Jiajie, Zhou, Bo
Format: Article
Language:English
Subjects:
Attention mechanisms
Cameras
Correlation
Cross-scene
Feature extraction
Laser radar
multimodal deep fusion
place recognition (PR)
Point cloud compression
Transformers
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As a crucial issue for outdoor mobile robot navigation, place recognition (PR) remains challenging in long-term cross-scene applications, necessitating enhanced robustness of PR algorithms. In this article, a novel multimodal PR approach is proposed, which deeply fuses the camera and LiDAR to effectively compensate for their respective limitations. The introduced place descriptor consists of three branches: image-based, point cloud-based, and fusion-based. Specifically, the fusion-based branch employs a dual-stage pipeline, leveraging the latent correlation between the two modalities for information interaction and conducting channel-level fusion. Moreover, implicit alignment in the fusion branch, coupled with information supplementation from the two single-modal branches, better addresses the information loss caused by limited overlap in their field-of-view (FoV). In extensive experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI), North Campus Long-Term (NCLT), USVInland datasets, and the real-world campus test, the proposed method stands out in precision-recall curves with an average maximum F1 score of 0.949, demonstrating superior robustness and generalization compared with other state-of-the-art methods.
ISSN:0278-0046
1557-9948
DOI:10.1109/TIE.2024.3440470