Dual-view Snapshot Compressive Imaging via Optical Flow Aided Recurrent Neural Network

Dual-view snapshot compressive imaging (SCI) aims to capture videos from two field-of-views (FoVs) using a 2D sensor (detector) in a single snapshot, achieving joint FoV and temporal compressive sensing, and thus enjoying the advantages of low-bandwidth, low-power and low-cost. However, it is challe...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision 2021-12, Vol.129 (12), p.3279-3298
Main Authors: Lu, Ruiying, Chen, Bo, Liu, Guanliang, Cheng, Ziheng, Qiao, Mu, Yuan, Xin
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Artificial neural networks
Computer Imaging
Computer Science
Decoding
Image Processing and Computer Vision
Neural networks
Optical flow (image analysis)
Pattern Recognition
Pattern Recognition and Graphics
Recurrent neural networks
Run time (computers)
Sensors
Special Issue on 3D Computer Vision
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Dual-view snapshot compressive imaging (SCI) aims to capture videos from two field-of-views (FoVs) using a 2D sensor (detector) in a single snapshot, achieving joint FoV and temporal compressive sensing, and thus enjoying the advantages of low-bandwidth, low-power and low-cost. However, it is challenging for existing model-based decoding algorithms to reconstruct each individual scene, which usually require exhaustive parameter tuning with extremely long running time for large scale data. In this paper, we propose an optical flow-aided recurrent neural network for dual video SCI systems, which provides high-quality decoding in seconds. Firstly, we develop a diversity amplification method to enlarge the differences between scenes of two FoVs, and design a deep convolutional neural network with dual branches to separate different scenes from the single measurement. Secondly, we integrate the bidirectional optical flow extracted from adjacent frames with the recurrent neural network to jointly reconstruct each video in a sequential manner. Extensive results on both simulation and real data demonstrate the superior performance of our proposed model in short inference time. The code and data are available at https://github.com/RuiyingLu/OFaNet-for-Dual-view-SCI .
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-021-01532-1