BTFormer: A BNN-Based Trend-Aware Time-Series Prediction Model for Industrial Intelligence

Prediction of industrial time-series is crucial for various Industrial Internet of Things applications. Despite the high accuracy of deep learning methods for time-series prediction, the significant memory requirements of deep learning models pose a challenge for the limited computational resources...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on industrial informatics 2024-09, p.1-10
Main Authors: Ren, Lei, Li, Shixiang, Wang, Xiaokang, Wang, Haiteng, Laili, Yuanjun
Format: Article
Language:English
Subjects:
Adaptation models
Binary neural network (BNN)
Computational modeling
Degradation
industrial time-series prediction
model quantization
Optimization
Predictive models
transformer
Transformers
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Prediction of industrial time-series is crucial for various Industrial Internet of Things applications. Despite the high accuracy of deep learning methods for time-series prediction, the significant memory requirements of deep learning models pose a challenge for the limited computational resources of industrial edge devices. To address this issue, this work proposes BTFormer, which achieves a high compression rate while maintaining competitive performance. First, a binary adaptive attention module is proposed to mitigate the loss of attention information caused by binarization. Second, a trend information soft-link is proposed to propagate trend information between layers and improve the representation ability of the model. Finally, a distribution-guided distillation strategy is proposed to optimize the training process. The experiments demonstrate that BTFormer effectively reduces model memory usage by 31.0 times and improves computational efficiency by 32.8 times while maintaining competitive performance.
ISSN:1551-3203
1941-0050
DOI:10.1109/TII.2024.3452181