Comparison of threshold tuning methods for predictive monitoring

Predictive monitoring techniques produce signals in case of a high predicted probability of an undesirable event, such as mortality, heart attacks, or machine failure. When using these predicted probabilities to classify the unknown outcome, a decision threshold needs to be chosen in statistical and...

Full description

Saved in:
Bibliographic Details
Published in:Quality and reliability engineering international 2024-02, Vol.40 (1), p.499-512
Main Authors: von Stackelberg, Paulina, Goedhart, Rob, Birbil, Ş. İlker, Does, Ronald J. M. M.
Format: Article
Language:English
Subjects:
false alarm rate
False alarms
Machine learning
Performance measurement
predictive monitoring
Signal monitoring
Statistical analysis
threshold tuning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Predictive monitoring techniques produce signals in case of a high predicted probability of an undesirable event, such as mortality, heart attacks, or machine failure. When using these predicted probabilities to classify the unknown outcome, a decision threshold needs to be chosen in statistical and machine learning models. In many cases, this is set to 0.5 by default. However, this may not lead to an acceptable model performance. To mitigate this issue, different threshold optimization approaches have been proposed in the literature. In this paper, we compare existing thresholding techniques to achieve a desired false alarm rate, and also evaluate the corresponding precision and recall performance metrics. A simulation study is conducted and a real‐world example on a medical dataset is provided.
ISSN:0748-8017
1099-1638
DOI:10.1002/qre.3436