BTM: Black-Box Testing for DNN Based on Meta-Learning

Deep learning is widely used in security fields like autonomous driving, but testing deep learning models poses challenges due to low generation efficiency and limited error detection. Current white-box test case generation methods rely on neuron coverage, but black-box testing is crucial when model...

Full description

Saved in:
Bibliographic Details
Main Authors: Zhang, Zhuangyu, Zhang, Zhiyi, Wang, Ziyuan, Chen, Fang, Huang, Zhiqiu
Format: Conference Proceeding
Language:English
Subjects:
black-box testing
Closed box
Deep learning
Life estimation
meta-learning
Metalearning
Neurons
Perturbation methods
Software reliability
test case generation
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning is widely used in security fields like autonomous driving, but testing deep learning models poses challenges due to low generation efficiency and limited error detection. Current white-box test case generation methods rely on neuron coverage, but black-box testing is crucial when model details cannot be accessed. Prior methods also needed more consideration for error diversity under lower time resources. In this paper, we propose a novel approach based on the intuition that two models trained on the same classification task learn similar rules at a coarse-grained level. We employ meta-learning to train universal meta perturbation on a surrogate model, which increases neuron coverage on the surrogate model while inducing misclassifications in the original model (thus enhancing the adequacy of the initial tests). Subsequently, we apply data initialization to discover a wider range of errors faster. To accelerate test case generation and reduce resource consumption, we introduce a set of acceleration techniques based on image prediction probabilities, minimizing the time spent exploring irrelevant regions in images. Finally, we evaluate our approach on a well-known dataset and several well-known models, demonstrating its effectiveness.
ISSN:2693-9177
DOI:10.1109/QRS60937.2023.00063