TRustworthy Uncertainty Propagation for Sequential Time-Series Analysis in RNNs

The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is part...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering 2024-02, Vol.36 (2), p.1-13
Main Authors: Dera, Dimah, Ahmed, Sabeen, Bouaynaya, Nidhal C., Rasool, Ghulam
Format: Article
Language:English
Subjects:
Bayes methods
Bayesian analysis
Computational modeling
gated recurrent units and long short-term memory networks
Internet of Things
Lower bounds
Mathematical models
Normal distribution
Parameter estimation
Power consumption
Predictive models
Recurrent neural networks
Self assessment
Statistical inference
Time series
Time series analysis
Trustworthiness
Uncertainty
uncertainty propagation
variational inference
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is particularly critical in heterogeneous and noisy environments. Bayesian inference allows reasoning about predictive uncertainty by estimating the posterior distribution of the parameters. The challenge remains in propagating the high-dimensional distribution through the sequential, non-linear layers of RNNs, resulting in mode collapse leading to erroneous uncertainty estimation and exacerbating the gradient explosion problem. This paper proposes a TRustworthy Uncertainty propagation for Sequential Time-series analysis (TRUST) in RNNs by introducing a Gaussian prior over network parameters and estimating the first two moments of the Gaussian variational distribution using the evidence lower bound. We propagate the variational moments through the sequential, non-linear layers of RNNs using the first-order Taylor approximation. The propagated covariance of the predictive distribution captures uncertainty in the output decision. The extensive experiments using ECG5000 and PeMS-SF classification and weather and power consumption prediction tasks demonstrate 1) significant robustness of TRUST-RNNs against noise and adversarial attacks and 2) self-assessment through the uncertainty that increases significantly with increasing noise.
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2023.3288628