HPGAN: Hyperspectral Pansharpening Using 3-D Generative Adversarial Networks

Hyperspectral (HS) pansharpening, as a special case of the superresolution (SR) problem, is to obtain a high-resolution (HR) image from the fusion of an HR panchromatic (PAN) image and a low-resolution (LR) HS image. Though HS pansharpening based on deep learning has gained rapid development in rece...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2021-01, Vol.59 (1), p.463-477
Main Authors: Xie, Weiying, Cui, Yuhang, Li, Yunsong, Lei, Jie, Du, Qian, Li, Jiaojiao
Format: Article
Language:English
Subjects:
3-D high-frequency block
Bayes methods
Computer applications
Data models
Dimensions
Gallium nitride
Generative adversarial networks
Generators
global constraint
hyperspectral pansharpening
Image enhancement
Image reconstruction
Image resolution
Resolution
Sensitivity
Sensors
Spatial discrimination
Spatial resolution
Spectra
spectral–spatial constraints
Successive discrimination learning
Training
Uniqueness
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Hyperspectral (HS) pansharpening, as a special case of the superresolution (SR) problem, is to obtain a high-resolution (HR) image from the fusion of an HR panchromatic (PAN) image and a low-resolution (LR) HS image. Though HS pansharpening based on deep learning has gained rapid development in recent years, it is still a challenging task because of the following requirements: 1) a unique model with the goal of fusing two images with different dimensions should enhance spatial resolution while preserving spectral information; 2) all the parameters should be adaptively trained without manual adjustment; and 3) a model with good generalization should overcome the sensitivity to different sensor data in reasonable computational complexity. To meet such requirements, we propose a unique HS pansharpening framework based on a 3-D generative adversarial network (HPGAN) in this article. The HPGAN induces the 3-D spectral-spatial generator network to reconstruct the HR HS image from the newly constructed 3-D PAN cube and the LR HS image. It searches for an optimal HR HS image by successive adversarial learning to fool the introduced PAN discriminator network. The loss function is specifically designed to comprehensively consider global constraint, spectral constraint, and spatial constraint. Besides, the proposed 3-D training in the high-frequency domain reduces the sensitivity to different sensor data and extends the generalization of HPGAN. Experimental results on data sets captured by different sensors illustrate that the proposed method can successfully enhance spatial resolution and preserve spectral information.
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2020.2994238