Write a Blog >>
ICST 2020
Sat 24 - Wed 28 October 2020 Porto, Portugal
Tue 27 Oct 2020 10:00 - 10:15 at Infante - Testing tools Chair(s): Andy Zaidman, René Just
Tue 27 Oct 2020 21:00 - 21:15 at Infante - Testing tools Chair(s): René Just, Andy Zaidman

Machine Learning (ML) has seen massive progress in the last decade and as a result, there is a pressing need for validating ML-based systems. To this end, we propose, design and evaluate CALLISTO- a novel test generation and data quality assessment framework. To the best of our knowledge, CALLISTO is the first black box framework to leverage the uncertainty in the prediction and systematically generate new test cases for ML classifiers. Our evaluation of CALLISTO on four real world data sets reveals thousands of errors. We also show that leveraging the uncertainty in prediction can increase the number of erroneous test cases up to a factor of 20, as compared to when no such knowledge is used for testing.CALLISTO has the capability to detect low quality data in the datasets that may contain mislabelled data. We conduct and present an extensive user study to validate the results of CALLISTO on identifying low quality data from four state-of-the-art real world datasets.

Tue 27 Oct
Times are displayed in time zone: Greenwich Mean Time : Lisbon change

10:00 - 11:00: Testing toolsTesting Tools Track at Infante +11h
Chair(s): Andy ZaidmanDelft University of Technology, René JustUniversity of Washington, USA
10:00 - 10:15
Talk
Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems
Testing Tools Track
Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design
Link to publication DOI
10:15 - 10:30
Talk
Run Java Applications and Test Them In-Vivo Meantime
Testing Tools Track
Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI)
Link to publication DOI
10:30 - 10:45
Talk
AFLNet: A Greybox Fuzzer for Network Protocols
Testing Tools Track
Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore
Link to publication DOI
10:45 - 11:00
Talk
ct-fuzz: Fuzzing for Timing Leaks
Testing Tools Track
Shaobo HeUniversity of Utah, Michael EmmiAmazon Web Services, Gabriela CiocarlieSRI International
Link to publication DOI
21:00 - 22:00: Testing toolsTesting Tools Track at Infante
Chair(s): René JustUniversity of Washington, USA, Andy ZaidmanDelft University of Technology
21:00 - 21:15
Talk
Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems
Testing Tools Track
Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design
Link to publication DOI
21:15 - 21:30
Talk
Run Java Applications and Test Them In-Vivo Meantime
Testing Tools Track
Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI)
Link to publication DOI
21:30 - 21:45
Talk
AFLNet: A Greybox Fuzzer for Network Protocols
Testing Tools Track
Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore
Link to publication DOI
21:45 - 22:00
Talk
ct-fuzz: Fuzzing for Timing Leaks
Testing Tools Track
Shaobo HeUniversity of Utah, Michael EmmiAmazon Web Services, Gabriela CiocarlieSRI International
Link to publication DOI