What is it about?
To fix the incorrect behaviors of DNN-driven software, developers often need rich labeled data for the testing and optimization of DNN models. However, collecting diverse data from application scenarios and labeling them is often a time-consuming task.To alleviate this problem, ATS is designed to select a subset of the massive unlabeled dataset with diverse tests.
Featured Image
Photo by John Schnobrich on Unsplash
Why is it important?
ATS has the following advantages. First, ATS could adaptively determine which test in the candidate set is more suitable to be labeled manually. Next, ATS could select a test set with enough and diverse faults. Finally, ATS could optimize the DNN model with a much lighter labeling cost. We experiment ATS with four well-designed DNN models and four widely-used datasets in comparison with various kinds of neuron coverage. The results demonstrate that ATS can significantly outperform all test selection methods in assessing both fault detection and model improvement capability of test suites. It is promising to save the data labeling and model retraining costs for deep neural network
Perspectives
I hope this article will provide a new perspective for identifying and selecting the tests. Different from current test selection methods, ATS is guided by fault pattern design and candidate fitness metric for test selection of Deep Neural Networks. We want the ATS to be able to select enough and diverse errors, which can efficiently improve the system.
xinyu gao
Nanjing University
Read the Original
This page is a summary of: Adaptive test selection for deep neural networks, May 2022, ACM (Association for Computing Machinery),
DOI: 10.1145/3510003.3510232.
You can read the full text:
Resources
Contributors
The following have contributed to this page
