The Tools Track seeks to bridge the gap between research and practice, focusing on software testing, verification, and validation. Its main goal is to advance both the state of the art and the state of the practice.
Sun 25 Oct Times are displayed in time zone: Lisbon change
09:00 - 09:15 | Intro & WelcomeResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room +11h Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | ||
09:00 15mDay opening | Intro & Welcome Research Papers |
09:15 - 10:45 | RT1 - Fault Localization and Debugging InfanteResearch Papers at Infante +11h Chair(s): Andreas ZellerCISPA, Germany | ||
09:15 30mTalk | Can We Predict the Quality of Spectrum-based Fault Localization? Research Papers Mojdeh GolaghaTechnical University of Munich, Alexander PretschnerTechnical University of Munich, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
09:45 30mTalk | Substate Profiling for Enhanced Fault Detection and Localization: An Empirical Study Research Papers Rawad Abou AssiAmerican University of Beirut, Wes MasriAmerican University of Beirut, Chadi TradAmerican University of Beirut Link to publication DOI | ||
10:15 30mTalk | More Accurate Dynamic Slicing for Better Supporting Software Debugging Research Papers Link to publication DOI |
11:00 - 12:30 | RT3 - Testing Deep Learning and Robotic SystemsResearch Papers at Farfetch (D. Maria) +11h Chair(s): Antonio FilieriImperial College London | ||
11:00 30mTalk | An Empirical Evaluation of Mutation Operators for Deep Learning Systems Research Papers Link to publication DOI | ||
11:30 30mTalk | Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study Research Papers Fitash Ul HaqUniversity of Luxembourg, Donghwan ShinUniversity of Luxembourg, Shiva NejatiUniversity of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
12:00 30mTalk | A Study on Challenges of Testing Robotic Systems Research Papers Afsoon AfzalCarnegie Mellon University, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Christopher Steven TimperleyCarnegie Mellon University Link to publication DOI |
11:00 - 12:30 | RT2 - Test Generation InfanteResearch Papers at Infante +11h Chair(s): Marcelo d'AmorimFederal University of Pernambuco | ||
11:00 30mTalk | Language-Agnostic Generation of Compilable Test Programs Research Papers Patrick KreutzerFriedrich-Alexander University Erlangen-Nürnberg, Stefan KrausFriedrich-Alexander University Erlangen-Nürnberg, Michael PhilippsenFriedrich-Alexander University Erlangen-Nürnberg (FAU) Link to publication DOI | ||
11:30 30mTalk | MiMIs: Simple, Efficient, and Fast Bounded-Exhaustive Test Case Generators Research Papers Kyle DeweyCalifornia State University, Northridge, Shant HairapetianUniversity of Oregon, Miroslav GavrilovUniversity of California, Santa Barbara Link to publication DOI | ||
12:00 30mTalk | Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection Research Papers Link to publication DOI |
14:00 - 15:00 | RT4 - Misc 1Research Papers at Infante +11h Chair(s): Andrea StoccoUniversità della Svizzera italiana (USI) | ||
14:00 30mTalk | Determining Method-Call Sequences for Object Creation in C++ Research Papers Link to publication DOI | ||
14:30 30mTalk | NodeRacer: Event Race Detection for Node.js Applications Research Papers André T. EndoFederal University of Technology - Paraná (UTFPR), Anders MøllerAarhus University, Denmark Link to publication DOI |
16:15 - 17:15 | Keynote 1Research Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC | ||
16:15 60mKeynote | One-click formal methods Research Papers Liana HadareanAmazon Link to publication |
20:00 - 20:15 | Intro & WelcomeResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | ||
20:00 15mDay opening | Intro & Welcome Research Papers |
20:15 - 21:45 | RT1 - Fault Localization and Debugging InfanteResearch Papers at Infante Chair(s): Jose CamposUniversity of Lisbon, Portugal | ||
20:15 30mTalk | Can We Predict the Quality of Spectrum-based Fault Localization? Research Papers Mojdeh GolaghaTechnical University of Munich, Alexander PretschnerTechnical University of Munich, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
20:45 30mTalk | Substate Profiling for Enhanced Fault Detection and Localization: An Empirical Study Research Papers Rawad Abou AssiAmerican University of Beirut, Wes MasriAmerican University of Beirut, Chadi TradAmerican University of Beirut Link to publication DOI | ||
21:15 30mTalk | More Accurate Dynamic Slicing for Better Supporting Software Debugging Research Papers Link to publication DOI |
22:00 - 23:30 | RT3 - Testing Deep Learning and Robotic SystemsResearch Papers at Farfetch (D. Maria) Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC | ||
22:00 30mTalk | An Empirical Evaluation of Mutation Operators for Deep Learning Systems Research Papers Link to publication DOI | ||
22:30 30mTalk | Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study Research Papers Fitash Ul HaqUniversity of Luxembourg, Donghwan ShinUniversity of Luxembourg, Shiva NejatiUniversity of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
23:00 30mTalk | A Study on Challenges of Testing Robotic Systems Research Papers Afsoon AfzalCarnegie Mellon University, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Christopher Steven TimperleyCarnegie Mellon University Link to publication DOI |
22:00 - 23:30 | RT2 - Test Generation InfanteResearch Papers at Infante Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
22:00 30mTalk | Language-Agnostic Generation of Compilable Test Programs Research Papers Patrick KreutzerFriedrich-Alexander University Erlangen-Nürnberg, Stefan KrausFriedrich-Alexander University Erlangen-Nürnberg, Michael PhilippsenFriedrich-Alexander University Erlangen-Nürnberg (FAU) Link to publication DOI | ||
22:30 30mTalk | MiMIs: Simple, Efficient, and Fast Bounded-Exhaustive Test Case Generators Research Papers Kyle DeweyCalifornia State University, Northridge, Shant HairapetianUniversity of Oregon, Miroslav GavrilovUniversity of California, Santa Barbara Link to publication DOI | ||
23:00 30mTalk | Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection Research Papers Link to publication DOI |
Mon 26 Oct Times are displayed in time zone: Lisbon change
01:00 - 02:00 | RT4 - Misc 1Research Papers at Infante Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
01:00 30mTalk | Determining Method-Call Sequences for Object Creation in C++ Research Papers Link to publication DOI | ||
01:30 30mTalk | NodeRacer: Event Race Detection for Node.js Applications Research Papers André T. EndoFederal University of Technology - Paraná (UTFPR), Anders MøllerAarhus University, Denmark Link to publication DOI |
09:00 - 09:30 | Most Influential Paper AwardsResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room +11h Chair(s): Luís CruzDeflt University of Technology | ||
09:00 15mAwards | Using Mutation to Automatically Suggest Fixes for Faulty Programs Research Papers Media Attached | ||
09:15 15mAwards | Searching for a needle in a haystack predicting security vulnerabilities for Windows Vista Research Papers Thomas ZimmermannMicrosoft Research, Nachiappan NagappanMicrosoft Research, USA, Laurie WilliamsNorth Carolina State University Media Attached |
09:30 - 10:30 | |||
10:45 - 12:15 | RT6 - Web TestingResearch Papers at Infante +11h Chair(s): Jose CamposUniversity of Lisbon, Portugal | ||
10:45 30mTalk | Fostering the Diversity of Exploratory Testing in Web Applications Research Papers Julien LeveauLaBRI, Xavier BlancLaBRI, Laurent RéveillèreLaBRI, Jean-Rémy FalleriBordeaux INP, Romain RouvoyUniv. Lille / Inria / IUF Link to publication DOI | ||
11:15 30mTalk | Dependency-Aware Web Test Generation Research Papers Matteo BiagiolaFondazione Bruno Kessler (FBK), Andrea StoccoUniversità della Svizzera italiana (USI), Filippo RiccaUniversità di Genova, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
11:45 30mTalk | Metamorphic Security Testing for Web Systems Research Papers Phu X. MaiUniversity of Luxembourg, Fabrizio PastoreUniversity of Luxembourg, Arda GoknilSnT, University of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI |
13:45 - 15:15 | RT7 - Optimizing TestingResearch Papers at Infante +11h Chair(s): Fabiano FerrariFederal University of São Carlos | ||
13:45 30mTalk | Optimizing Mutation Testing by Discovering Dynamic Mutant Subsumption Relations Research Papers Marcio Augusto GuimarãesFederal University of Alagoas (UFAL), Leo FernandesFederal Institute of Alagoas (IFAL), Márcio RibeiroFederal University of Alagoas, Brazil, Marcelo d'AmorimFederal University of Pernambuco, Rohit GheyiFederal University of Campina Grande Link to publication DOI | ||
14:15 30mTalk | Massively Parallel, Highly Efficient, but What About the Test Suite Quality? Applying Mutation Testing to GPU Programs Research Papers Link to publication DOI Pre-print | ||
14:45 30mTalk | STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases Research Papers Abdullah AlsharifThe University of Sheffield, Gregory KapfhammerAllegheny College, USA, Phil McMinnUniversity of Sheffield Link to publication DOI |
15:30 - 17:00 | RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) +11h Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
15:30 30mTalk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
16:00 30mTalk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio LeottaUniversità di Genova, Matteo BiagiolaFondazione Bruno Kessler (FBK), Filippo RiccaUniversità di Genova, Mariano CeccatoUniversity of Verona, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
16:30 30mTalk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel BöhmeMonash University, Australia, Charaka GeethalMonash University, Van-Thuan PhamMonash University Link to publication DOI |
15:30 - 17:00 | RT8 - Misc 2Journal-First Papers / Research Papers at Infante +11h Chair(s): Alin StefanescuUniversity of Bucharest | ||
15:30 30mTalk | Implementation-induced Inconsistency and Nondeterminism in Deterministic Clustering Algorithms Research Papers Xin YinNew Jersey Institute of Technology, Iulian NeamtiuNew Jersey Institute of Technology, USA, Saketan PatilNew Jersey Institute of Technology, Sean AndrewsNew Jersey Institute of Technology Link to publication DOI | ||
16:00 30mTalk | CBR: Controlled Burst Recording Research Papers Oscar CornejoUniversity of Milano Bicocca, Italy, Daniela BriolaUniversity of Milano Bicocca, Daniela MicucciUniversity of Milano Bicocca, Leonardo MarianiUniversity of Milano Bicocca Link to publication DOI | ||
16:30 30mTalk | Mahtab: Phase-wise acceleration of regression testing for C Journal-First Papers Link to publication DOI Media Attached |
20:00 - 20:30 | Most Influential Paper AwardsResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | ||
20:00 15mAwards | Using Mutation to Automatically Suggest Fixes for Faulty Programs Research Papers Media Attached | ||
20:15 15mAwards | Searching for a needle in a haystack predicting security vulnerabilities for Windows Vista Research Papers Thomas ZimmermannMicrosoft Research, Nachiappan NagappanMicrosoft Research, USA, Laurie WilliamsNorth Carolina State University Media Attached |
21:45 - 23:15 | RT6 - Web TestingResearch Papers at Infante Chair(s): Fabiano FerrariFederal University of São Carlos | ||
21:45 30mTalk | Fostering the Diversity of Exploratory Testing in Web Applications Research Papers Julien LeveauLaBRI, Xavier BlancLaBRI, Laurent RéveillèreLaBRI, Jean-Rémy FalleriBordeaux INP, Romain RouvoyUniv. Lille / Inria / IUF Link to publication DOI | ||
22:15 30mTalk | Dependency-Aware Web Test Generation Research Papers Matteo BiagiolaFondazione Bruno Kessler (FBK), Andrea StoccoUniversità della Svizzera italiana (USI), Filippo RiccaUniversità di Genova, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
22:45 30mTalk | Metamorphic Security Testing for Web Systems Research Papers Phu X. MaiUniversity of Luxembourg, Fabrizio PastoreUniversity of Luxembourg, Arda GoknilSnT, University of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI |
Tue 27 Oct Times are displayed in time zone: Lisbon change
02:30 - 04:00 | RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | ||
02:30 30mTalk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
03:00 30mTalk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio LeottaUniversità di Genova, Matteo BiagiolaFondazione Bruno Kessler (FBK), Filippo RiccaUniversità di Genova, Mariano CeccatoUniversity of Verona, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
03:30 30mTalk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel BöhmeMonash University, Australia, Charaka GeethalMonash University, Van-Thuan PhamMonash University Link to publication DOI |
02:30 - 04:00 | |||
02:30 30mTalk | Implementation-induced Inconsistency and Nondeterminism in Deterministic Clustering Algorithms Research Papers Xin YinNew Jersey Institute of Technology, Iulian NeamtiuNew Jersey Institute of Technology, USA, Saketan PatilNew Jersey Institute of Technology, Sean AndrewsNew Jersey Institute of Technology Link to publication DOI | ||
03:00 30mTalk | CBR: Controlled Burst Recording Research Papers Oscar CornejoUniversity of Milano Bicocca, Italy, Daniela BriolaUniversity of Milano Bicocca, Daniela MicucciUniversity of Milano Bicocca, Leonardo MarianiUniversity of Milano Bicocca Link to publication DOI | ||
03:30 30mTalk | Mahtab: Phase-wise acceleration of regression testing for C Journal-First Papers Link to publication DOI Media Attached |
09:00 - 10:00 | RT10 - Misc 3Research Papers at Infante +11h Chair(s): Thomas DurieuxKTH Royal Institute of Technology, Sweden | ||
09:00 30mTalk | A Framework for In-Vivo Testing of Mobile Applications Research Papers Mariano CeccatoUniversity of Verona, Davide CorradiniFondazione Bruno Kessler, Luca Gazzola Università degli Studi di Milano-Bicocca, Fitsum KifetewFondazione Bruno Kessler, Leonardo MarianiUniversity of Milano Bicocca, Matteo OrruUniversity of Milano Bicocca, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
09:30 30mTalk | Prioritizing Runtime Verification Violations Research Papers Breno MirandaFederal University of Pernambuco, Igor LimaFederal University of Pernambuco, Owolabi LegunsenCornell University, Marcelo d'AmorimFederal University of Pernambuco Link to publication DOI |
10:00 - 11:00 | Testing toolsTesting Tools Track at Infante +11h Chair(s): Andy ZaidmanDelft University of Technology, René JustUniversity of Washington, USA | ||
10:00 15mTalk | Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems Testing Tools Track Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design Link to publication DOI | ||
10:15 15mTalk | Run Java Applications and Test Them In-Vivo Meantime Testing Tools Track Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
10:30 15mTalk | AFLNet: A Greybox Fuzzer for Network Protocols Testing Tools Track Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore Link to publication DOI | ||
10:45 15mTalk | ct-fuzz: Fuzzing for Timing Leaks Testing Tools Track Link to publication DOI |
11:15 - 12:15 | Open Steering Commitee & ICST 2021 PresentationResearch Papers at Plenary Room Chair(s): Shin YooKorea Advanced Institute of Science and Technology | ||
11:15 60mMeeting | Open Steering Commitee & ICST 2021 Presentation Research Papers |
13:45 - 14:45 | Keynote 3Research Papers at Plenary Room Chair(s): Rui AbreuFaculty of Engineering, University of Porto, Portugal | ||
13:45 60mKeynote | SunDew: Systematic Automated Security Testing Research Papers Franjo IvancicGoogle |
20:00 - 21:00 | |||
20:00 30mTalk | A Framework for In-Vivo Testing of Mobile Applications Research Papers Mariano CeccatoUniversity of Verona, Davide CorradiniFondazione Bruno Kessler, Luca Gazzola Università degli Studi di Milano-Bicocca, Fitsum KifetewFondazione Bruno Kessler, Leonardo MarianiUniversity of Milano Bicocca, Matteo OrruUniversity of Milano Bicocca, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
20:30 30mTalk | Prioritizing Runtime Verification Violations Research Papers Breno MirandaFederal University of Pernambuco, Igor LimaFederal University of Pernambuco, Owolabi LegunsenCornell University, Marcelo d'AmorimFederal University of Pernambuco Link to publication DOI |
21:00 - 22:00 | Testing toolsTesting Tools Track at Infante Chair(s): René JustUniversity of Washington, USA, Andy ZaidmanDelft University of Technology | ||
21:00 15mTalk | Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems Testing Tools Track Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design Link to publication DOI | ||
21:15 15mTalk | Run Java Applications and Test Them In-Vivo Meantime Testing Tools Track Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
21:30 15mTalk | AFLNet: A Greybox Fuzzer for Network Protocols Testing Tools Track Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore Link to publication DOI | ||
21:45 15mTalk | ct-fuzz: Fuzzing for Timing Leaks Testing Tools Track Link to publication DOI |
Call for Papers
The Tools Track seeks to bridge the gap between research and practice, focusing on software testing, verification, and validation. Its main goal is to advance both the state of the art and the state of the practice.
The Tools Track invites submissions that describe tools at several stages of maturity, including promising research prototypes, widely used research tools, and commercial tools.
A Tools Track submission should clearly describe the complexity of the addressed problem, solved technical challenges, and possibilities for technology transfer into practice. Researchers are welcome to submit papers that describe novel approaches and how these approaches support tool developers in solving challenging problems. Moreover, this track seeks non-academic submissions to foster discussions between researchers and practitioners.
Areas of interest:
The Tools Track invites submissions related, but not limited, to the following areas:
- Design and development of novel tools for software testing, verification, and validation;
- Working prototypes for novel research approaches, reporting on the technical challenges and early feedback obtained from practitioners;
- Strategies and challenges for tool adoption as well as lessons learned from technology transfer of academic approaches into open source or commercial tools;
- Case studies and/or experiments involving tools in particular contexts, reporting on lessons learned and negative results (if any).
Evaluation:
Each submission will be evaluated based on:
- The relevance and significance of the addressed problem.
- The innovation element of the approach.
- The availability, maturity, and adoption of the tool.
- The presence of lessons learned from developing or using the tool.
- The quality of the presentation.
Accepted papers will appear in the conference proceedings and the IEEE digital library.
Submission:
Submissions will be handled via easychair (Testing Tools Track): https://easychair.org/my/conference?conf=icst2020
The Testing Tools Track of ICST 2020 uses single-blind reviewing, which means that papers and tools need not be anonymized. All submitted papers must conform to the two column IEEE conference publication format. Templates for Latex and Word are available at: http://www.ieee.org/conferences_events/conferences/publishing/templates.html;
- Use the letter format templates and the “conference” option.
- The paper must be submitted as PDF and m ust not exceed 6 pages.
Presentation format:
Author(s) of an accepted paper will present their work at the ICST conference on two occasions:
- Lightning talk to advertise the tool and talk (~1 minute).
- Paper presentation (~15 minutes).
Accepted Papers
Title | |
---|---|
AFLNet: A Greybox Fuzzer for Network Protocols Testing Tools Track Link to publication DOI | |
CDST: A Toolkit for Testing Cockpit Display Systems Testing Tools Track Link to publication DOI | |
Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems Testing Tools Track Link to publication DOI | |
Open-source Defect Injection Benchmark Testbed for the Evaluation of Testing Testing Tools Track Link to publication DOI | |
Run Java Applications and Test Them In-Vivo Meantime Testing Tools Track Link to publication DOI | |
ct-fuzz: Fuzzing for Timing Leaks Testing Tools Track Link to publication DOI |