ICST 2020 invites high quality submissions in all areas of software testing, verification, and validation. Papers for the research track should present novel and original work that pushes the state-of-the-art. Case studies and empirical research papers are also welcome.
Sun 25 Oct Times are displayed in time zone: Greenwich Mean Time : Lisbon change
09:00 - 09:15: Intro & WelcomeResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room +11h Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
09:00 - 09:15 Day opening | Intro & Welcome Research Papers |
09:15 - 10:45: RT1 - Fault Localization and Debugging InfanteResearch Papers at Infante +11h Chair(s): Andreas ZellerCISPA, Germany | |||
09:15 - 09:45 Talk | Can We Predict the Quality of Spectrum-based Fault Localization? Research Papers Mojdeh GolaghaTechnical University of Munich, Alexander PretschnerTechnical University of Munich, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
09:45 - 10:15 Talk | Substate Profiling for Enhanced Fault Detection and Localization: An Empirical Study Research Papers Rawad Abou AssiAmerican University of Beirut, Wes MasriAmerican University of Beirut, Chadi TradAmerican University of Beirut Link to publication DOI | ||
10:15 - 10:45 Talk | More Accurate Dynamic Slicing for Better Supporting Software Debugging Research Papers Link to publication DOI |
11:00 - 12:30: RT3 - Testing Deep Learning and Robotic SystemsResearch Papers at Farfetch (D. Maria) +11h Chair(s): Antonio FilieriImperial College London | |||
11:00 - 11:30 Talk | An Empirical Evaluation of Mutation Operators for Deep Learning Systems Research Papers Link to publication DOI | ||
11:30 - 12:00 Talk | Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study Research Papers Fitash Ul HaqUniversity of Luxembourg, Donghwan ShinUniversity of Luxembourg, Shiva NejatiUniversity of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
12:00 - 12:30 Talk | A Study on Challenges of Testing Robotic Systems Research Papers Afsoon AfzalCarnegie Mellon University, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Christopher Steven TimperleyCarnegie Mellon University Link to publication DOI |
11:00 - 12:30: RT2 - Test Generation InfanteResearch Papers at Infante +11h Chair(s): Marcelo d'AmorimFederal University of Pernambuco | |||
11:00 - 11:30 Talk | Language-Agnostic Generation of Compilable Test Programs Research Papers Patrick KreutzerFriedrich-Alexander University Erlangen-Nürnberg, Stefan KrausFriedrich-Alexander University Erlangen-Nürnberg, Michael PhilippsenFriedrich-Alexander University Erlangen-Nürnberg (FAU) Link to publication DOI | ||
11:30 - 12:00 Talk | MiMIs: Simple, Efficient, and Fast Bounded-Exhaustive Test Case Generators Research Papers Kyle DeweyCalifornia State University, Northridge, Shant HairapetianUniversity of Oregon, Miroslav GavrilovUniversity of California, Santa Barbara Link to publication DOI | ||
12:00 - 12:30 Talk | Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection Research Papers Link to publication DOI |
14:00 - 15:00: RT4 - Misc 1Research Papers at Infante +11h Chair(s): Andrea StoccoUniversità della Svizzera italiana (USI) | |||
14:00 - 14:30 Talk | Determining Method-Call Sequences for Object Creation in C++ Research Papers Link to publication DOI | ||
14:30 - 15:00 Talk | NodeRacer: Event Race Detection for Node.js Applications Research Papers André T. EndoFederal University of Technology - Paraná (UTFPR), Anders MøllerAarhus University, Denmark Link to publication DOI |
16:15 - 17:15: Keynote 1Research Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC | |||
16:15 - 17:15 Keynote | One-click formal methods Research Papers Liana HadareanAmazon Link to publication |
20:00 - 20:15: Intro & WelcomeResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
20:00 - 20:15 Day opening | Intro & Welcome Research Papers |
20:15 - 21:45: IT1 - Test Generation & RefactoringIndustry Track at Farfetch (D. Maria) Chair(s): John MiccoVMware | |||
20:15 - 20:45 Talk | Parallelization of Classical Numerical Optimization in Quantum Variational Algorithms Industry Track Marco PistoiaJPMorgan Chase & Co, Peng LiuGoogle, Chun-Fu Richard ChenIBM, Shaohan HuIBM, Stephen WoodIBM Link to publication DOI | ||
20:45 - 21:15 Talk | SPECMATE: Automated Creation of Test Cases from Acceptance Criteria Industry Track Jannik FischbachQualicen GmbH, Andreas VogelsangUniversity of Cologne, Dominik SpiesQualicen GmbH, Andreas WehrleQualicen GmbH, Maximilian JunkerQualicen GmbH, Dietmar FreudensteinAllianz Deutschland AG Link to publication DOI | ||
21:15 - 21:45 Talk | Tool Support for Refactoring Manual Tests Industry Track Elodie BernardFEMTO-ST Institute, Univ. Bourgogne Franche-Comté, Sogeti, Julien BotellaSmartesting, Fabrice AmbertFEMTO-ST Institute, Bruno LegeardFEMTO-ST / DISC, Mark UttingUniversity of the Sunshine Coast Link to publication DOI |
20:15 - 21:45: RT1 - Fault Localization and Debugging InfanteResearch Papers at Infante Chair(s): Jose CamposUniversity of Lisbon, Portugal | |||
20:15 - 20:45 Talk | Can We Predict the Quality of Spectrum-based Fault Localization? Research Papers Mojdeh GolaghaTechnical University of Munich, Alexander PretschnerTechnical University of Munich, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
20:45 - 21:15 Talk | Substate Profiling for Enhanced Fault Detection and Localization: An Empirical Study Research Papers Rawad Abou AssiAmerican University of Beirut, Wes MasriAmerican University of Beirut, Chadi TradAmerican University of Beirut Link to publication DOI | ||
21:15 - 21:45 Talk | More Accurate Dynamic Slicing for Better Supporting Software Debugging Research Papers Link to publication DOI |
22:00 - 23:30: RT3 - Testing Deep Learning and Robotic SystemsResearch Papers at Farfetch (D. Maria) Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC | |||
22:00 - 22:30 Talk | An Empirical Evaluation of Mutation Operators for Deep Learning Systems Research Papers Link to publication DOI | ||
22:30 - 23:00 Talk | Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study Research Papers Fitash Ul HaqUniversity of Luxembourg, Donghwan ShinUniversity of Luxembourg, Shiva NejatiUniversity of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI | ||
23:00 - 23:30 Talk | A Study on Challenges of Testing Robotic Systems Research Papers Afsoon AfzalCarnegie Mellon University, Claire Le GouesCarnegie Mellon University, Michael HiltonCarnegie Mellon University, USA, Christopher Steven TimperleyCarnegie Mellon University Link to publication DOI |
22:00 - 23:30: RT2 - Test Generation InfanteResearch Papers at Infante Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | |||
22:00 - 22:30 Talk | Language-Agnostic Generation of Compilable Test Programs Research Papers Patrick KreutzerFriedrich-Alexander University Erlangen-Nürnberg, Stefan KrausFriedrich-Alexander University Erlangen-Nürnberg, Michael PhilippsenFriedrich-Alexander University Erlangen-Nürnberg (FAU) Link to publication DOI | ||
22:30 - 23:00 Talk | MiMIs: Simple, Efficient, and Fast Bounded-Exhaustive Test Case Generators Research Papers Kyle DeweyCalifornia State University, Northridge, Shant HairapetianUniversity of Oregon, Miroslav GavrilovUniversity of California, Santa Barbara Link to publication DOI | ||
23:00 - 23:30 Talk | Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection Research Papers Link to publication DOI |
Mon 26 Oct Times are displayed in time zone: Greenwich Mean Time : Lisbon change
01:00 - 02:00: RT4 - Misc 1Research Papers at Infante Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | |||
01:00 - 01:30 Talk | Determining Method-Call Sequences for Object Creation in C++ Research Papers Link to publication DOI | ||
01:30 - 02:00 Talk | NodeRacer: Event Race Detection for Node.js Applications Research Papers André T. EndoFederal University of Technology - Paraná (UTFPR), Anders MøllerAarhus University, Denmark Link to publication DOI |
09:00 - 09:30: Most Influential Paper AwardsResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room +11h Chair(s): Luis CruzDeflt University of Technology | |||
09:00 - 09:15 Awards | Using Mutation to Automatically Suggest Fixes for Faulty Programs Research Papers Media Attached | ||
09:15 - 09:30 Awards | Searching for a needle in a haystack predicting security vulnerabilities for Windows Vista Research Papers Thomas ZimmermannMicrosoft Research, Nachiappan NagappanMicrosoft Research, USA, Laurie WilliamsNorth Carolina State University Media Attached |
09:30 - 10:30: Keynote 2Research Papers at Plenary Room Chair(s): Luis CruzDeflt University of Technology | |||
10:45 - 12:15: IT2 - MiscIndustry Track at Farfetch (D. Maria) +11h Chair(s): Luis CruzDeflt University of Technology | |||
10:45 - 11:15 Talk | EC.LANG - A Language for Specifying Response Time Analyses of Event Chains Industry Track Max FrieseChristian-Albrechts-Universität zu Kiel, Johannes TraubDaimler AG, Dirk NowotkaChristian-Albrechts-Universität zu Kiel Link to publication DOI | ||
11:15 - 11:45 Talk | Detecting Higher-Order Merge Conflicts in Large Software Projects Industry Track Link to publication DOI | ||
11:45 - 12:15 Talk | Scaling Test Case Generation For Expressive Decision Tables Industry Track Supriya AgrawalTata Consultancy Services Ltd. (TCS), R VenkateshTata Consultancy Services Ltd. (TCS), Ulka ShrotriTata Consultancy Services Ltd. (TCS), Amey ZareTata Consultancy Services Ltd. (TCS), Sagar VermaTata Consultancy Services Ltd. (TCS) Link to publication DOI |
10:45 - 12:15: RT6 - Web TestingResearch Papers at Infante +11h Chair(s): Jose CamposUniversity of Lisbon, Portugal | |||
10:45 - 11:15 Talk | Fostering the Diversity of Exploratory Testing in Web Applications Research Papers Julien LeveauLaBRI, Xavier BlancLaBRI, Laurent RéveillèreLaBRI, Jean-Rémy FalleriBordeaux INP, Romain RouvoyUniv. Lille / Inria / IUF Link to publication DOI | ||
11:15 - 11:45 Talk | Dependency-Aware Web Test Generation Research Papers Matteo BiagiolaFondazione Bruno Kessler (FBK), Andrea StoccoUniversità della Svizzera italiana (USI), Filippo RiccaUniversità di Genova, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
11:45 - 12:15 Talk | Metamorphic Security Testing for Web Systems Research Papers Phu X. MaiUniversity of Luxembourg, Fabrizio PastoreUniversity of Luxembourg, Arda GoknilSnT, University of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI |
13:45 - 15:15: Journal First 1Journal-First Papers at Farfetch (D. Maria) +11h Chair(s): Maurício AnicheDelft University of Technology, Netherlands | |||
13:45 - 14:15 Talk | Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems Journal-First Papers Tao MaSimula Research Laboratory, Shaukat AliSimula Research Laboratory, Tao YueNanjing University of Aeronautics and Astronautics Link to publication DOI | ||
14:15 - 14:45 Talk | Empirical evaluation of mutation-based test case prioritization techniques Journal-First Papers Donghwan ShinUniversity of Luxembourg, Shin YooKorea Advanced Institute of Science and Technology, Mike PapadakisUniversity of Luxembourg, Luxembourg, Doo-Hwan BaeKorea Advanced Institute of Science and Technology Link to publication DOI | ||
14:45 - 15:15 Talk | Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects Journal-First Papers Fabian TrautschUniversity of Göttingen, Steffen HerboldUniversity of Göttingen, Jens GrabowskiUniversity of Göttingen Link to publication DOI |
13:45 - 15:15: RT7 - Optimizing TestingResearch Papers at Infante +11h Chair(s): Fabiano FerrariFederal University of São Carlos | |||
13:45 - 14:15 Talk | Optimizing Mutation Testing by Discovering Dynamic Mutant Subsumption Relations Research Papers Marcio Augusto GuimarãesFederal University of Alagoas (UFAL), Leo FernandesFederal Institute of Alagoas (IFAL), Márcio RibeiroFederal University of Alagoas, Brazil, Marcelo d'AmorimFederal University of Pernambuco, Rohit GheyiFederal University of Campina Grande Link to publication DOI | ||
14:15 - 14:45 Talk | Massively Parallel, Highly Efficient, but What About the Test Suite Quality? Applying Mutation Testing to GPU Programs Research Papers Link to publication DOI Pre-print | ||
14:45 - 15:15 Talk | STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases Research Papers Abdullah AlsharifThe University of Sheffield, Gregory KapfhammerAllegheny College, USA, Phil McMinnUniversity of Sheffield Link to publication DOI |
15:30 - 17:00: RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) +11h Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | |||
15:30 - 16:00 Talk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
16:00 - 16:30 Talk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio LeottaUniversità di Genova, Matteo BiagiolaFondazione Bruno Kessler (FBK), Filippo RiccaUniversità di Genova, Mariano CeccatoUniversity of Verona, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
16:30 - 17:00 Talk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel BöhmeMonash University, Australia, Charaka GeethalMonash University, Van-Thuan PhamMonash University Link to publication DOI |
15:30 - 17:00: RT8 - Misc 2Research Papers / Journal-First Papers at Infante +11h Chair(s): Alin StefanescuUniversity of Bucharest | |||
15:30 - 16:00 Talk | Implementation-induced Inconsistency and Nondeterminism in Deterministic Clustering Algorithms Research Papers Xin YinNew Jersey Institute of Technology, Iulian NeamtiuNew Jersey Institute of Technology, USA, Saketan PatilNew Jersey Institute of Technology, Sean AndrewsNew Jersey Institute of Technology Link to publication DOI | ||
16:00 - 16:30 Talk | CBR: Controlled Burst Recording Research Papers Oscar CornejoUniversity of Milano Bicocca, Italy, Daniela BriolaUniversity of Milano Bicocca, Daniela MicucciUniversity of Milano Bicocca, Leonardo MarianiUniversity of Milano Bicocca Link to publication DOI | ||
16:30 - 17:00 Talk | Mahtab: Phase-wise acceleration of regression testing for C Journal-First Papers Link to publication DOI Media Attached |
20:00 - 20:30: Most Influential Paper AwardsResearch Papers / Tool Demos Track / Journal-First Papers / Industry Track / Testing Tools Track at Plenary Room Chair(s): João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC, Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
20:00 - 20:15 Awards | Using Mutation to Automatically Suggest Fixes for Faulty Programs Research Papers Media Attached | ||
20:15 - 20:30 Awards | Searching for a needle in a haystack predicting security vulnerabilities for Windows Vista Research Papers Thomas ZimmermannMicrosoft Research, Nachiappan NagappanMicrosoft Research, USA, Laurie WilliamsNorth Carolina State University Media Attached |
21:45 - 23:15: IT2 - MiscIndustry Track at Farfetch (D. Maria) Chair(s): Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
21:45 - 22:15 Talk | EC.LANG - A Language for Specifying Response Time Analyses of Event Chains Industry Track Max FrieseChristian-Albrechts-Universität zu Kiel, Johannes TraubDaimler AG, Dirk NowotkaChristian-Albrechts-Universität zu Kiel Link to publication DOI | ||
22:15 - 22:45 Talk | Detecting Higher-Order Merge Conflicts in Large Software Projects Industry Track Link to publication DOI | ||
22:45 - 23:15 Talk | Scaling Test Case Generation For Expressive Decision Tables Industry Track Supriya AgrawalTata Consultancy Services Ltd. (TCS), R VenkateshTata Consultancy Services Ltd. (TCS), Ulka ShrotriTata Consultancy Services Ltd. (TCS), Amey ZareTata Consultancy Services Ltd. (TCS), Sagar VermaTata Consultancy Services Ltd. (TCS) Link to publication DOI |
21:45 - 23:15: RT6 - Web TestingResearch Papers at Infante Chair(s): Fabiano FerrariFederal University of São Carlos | |||
21:45 - 22:15 Talk | Fostering the Diversity of Exploratory Testing in Web Applications Research Papers Julien LeveauLaBRI, Xavier BlancLaBRI, Laurent RéveillèreLaBRI, Jean-Rémy FalleriBordeaux INP, Romain RouvoyUniv. Lille / Inria / IUF Link to publication DOI | ||
22:15 - 22:45 Talk | Dependency-Aware Web Test Generation Research Papers Matteo BiagiolaFondazione Bruno Kessler (FBK), Andrea StoccoUniversità della Svizzera italiana (USI), Filippo RiccaUniversità di Genova, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
22:45 - 23:15 Talk | Metamorphic Security Testing for Web Systems Research Papers Phu X. MaiUniversity of Luxembourg, Fabrizio PastoreUniversity of Luxembourg, Arda GoknilSnT, University of Luxembourg, Lionel BriandUniversity of Luxembourg, University of Ottawa Link to publication DOI |
Tue 27 Oct Times are displayed in time zone: Greenwich Mean Time : Lisbon change
00:45 - 02:15: Journal First 1Journal-First Papers at Farfetch (D. Maria) Chair(s): Fabiano FerrariFederal University of São Carlos | |||
00:45 - 01:15 Talk | Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems Journal-First Papers Tao MaSimula Research Laboratory, Shaukat AliSimula Research Laboratory, Tao YueNanjing University of Aeronautics and Astronautics Link to publication DOI | ||
01:15 - 01:45 Talk | Empirical evaluation of mutation-based test case prioritization techniques Journal-First Papers Donghwan ShinUniversity of Luxembourg, Shin YooKorea Advanced Institute of Science and Technology, Mike PapadakisUniversity of Luxembourg, Luxembourg, Doo-Hwan BaeKorea Advanced Institute of Science and Technology Link to publication DOI | ||
01:45 - 02:15 Talk | Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects Journal-First Papers Fabian TrautschUniversity of Göttingen, Steffen HerboldUniversity of Göttingen, Jens GrabowskiUniversity of Göttingen Link to publication DOI |
02:30 - 04:00: RT9 - Empirical Analysis and Program RepairResearch Papers at Farfetch (D. Maria) Chair(s): Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | |||
02:30 - 03:00 Talk | An Empirical Analysis of Blind Tests Research Papers Link to publication DOI | ||
03:00 - 03:30 Talk | A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development Research Papers Maurizio LeottaUniversità di Genova, Matteo BiagiolaFondazione Bruno Kessler (FBK), Filippo RiccaUniversità di Genova, Mariano CeccatoUniversity of Verona, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
03:30 - 04:00 Talk | Human-In-The-Loop Automatic Program Repair Research Papers Marcel BöhmeMonash University, Australia, Charaka GeethalMonash University, Van-Thuan PhamMonash University Link to publication DOI |
02:30 - 03:00 Talk | Implementation-induced Inconsistency and Nondeterminism in Deterministic Clustering Algorithms Research Papers Xin YinNew Jersey Institute of Technology, Iulian NeamtiuNew Jersey Institute of Technology, USA, Saketan PatilNew Jersey Institute of Technology, Sean AndrewsNew Jersey Institute of Technology Link to publication DOI | ||
03:00 - 03:30 Talk | CBR: Controlled Burst Recording Research Papers Oscar CornejoUniversity of Milano Bicocca, Italy, Daniela BriolaUniversity of Milano Bicocca, Daniela MicucciUniversity of Milano Bicocca, Leonardo MarianiUniversity of Milano Bicocca Link to publication DOI | ||
03:30 - 04:00 Talk | Mahtab: Phase-wise acceleration of regression testing for C Journal-First Papers Link to publication DOI Media Attached |
09:00 - 10:00: RT10 - Misc 3Research Papers at Infante +11h Chair(s): Thomas DurieuxKTH Royal Institute of Technology, Sweden | |||
09:00 - 09:30 Talk | A Framework for In-Vivo Testing of Mobile Applications Research Papers Mariano CeccatoUniversity of Verona, Davide CorradiniFondazione Bruno Kessler, Luca Gazzola Università degli Studi di Milano-Bicocca, Fitsum KifetewFondazione Bruno Kessler, Leonardo MarianiUniversity of Milano Bicocca, Matteo OrruUniversity of Milano Bicocca, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
09:30 - 10:00 Talk | Prioritizing Runtime Verification Violations Research Papers Breno MirandaFederal University of Pernambuco, Igor LimaFederal University of Pernambuco, Owolabi LegunsenCornell University, Marcelo d'AmorimFederal University of Pernambuco Link to publication DOI |
10:00 - 11:00: IT3 - Safety & SecurityIndustry Track at Farfetch (D. Maria) +11h Chair(s): Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
10:00 - 10:30 Talk | Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems Industry Track Alessandro CalòTechnical University of Munich, Paolo ArcainiNational Institute of Informatics
, Shaukat AliSimula Research Laboratory, Florian HauerTechnical University of Munich, Fuyuki IshikawaNational Institute of Informatics Link to publication DOI | ||
10:30 - 11:00 Talk | Checking Security Properties of Cloud Service REST APIs Industry Track Vaggelis AtlidakisColumbia University, Patrice GodefroidMicrosoft Research, USA, Marina PolishchukMicrosoft Research, USA Link to publication DOI |
10:00 - 11:00: Testing toolsTesting Tools Track at Infante +11h Chair(s): Andy ZaidmanDelft University of Technology, René JustUniversity of Washington, USA | |||
10:00 - 10:15 Talk | Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems Testing Tools Track Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design Link to publication DOI | ||
10:15 - 10:30 Talk | Run Java Applications and Test Them In-Vivo Meantime Testing Tools Track Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
10:30 - 10:45 Talk | AFLNet: A Greybox Fuzzer for Network Protocols Testing Tools Track Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore Link to publication DOI | ||
10:45 - 11:00 Talk | ct-fuzz: Fuzzing for Timing Leaks Testing Tools Track Link to publication DOI |
11:15 - 12:15: Open Steering Commitee & ICST 2021 PresentationResearch Papers at Plenary Room Chair(s): Shin YooKorea Advanced Institute of Science and Technology | |||
11:15 - 12:15 Meeting | Open Steering Commitee & ICST 2021 Presentation Research Papers |
13:45 - 14:45: Keynote 3Research Papers at Plenary Room Chair(s): Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
13:45 - 14:45 Keynote | SunDew: Systematic Automated Security Testing Research Papers Franjo IvancicGoogle |
20:00 - 21:00: RT10 - Misc 3Research Papers at Infante Chair(s): Luis CruzDeflt University of Technology | |||
20:00 - 20:30 Talk | A Framework for In-Vivo Testing of Mobile Applications Research Papers Mariano CeccatoUniversity of Verona, Davide CorradiniFondazione Bruno Kessler, Luca Gazzola Università degli Studi di Milano-Bicocca, Fitsum KifetewFondazione Bruno Kessler, Leonardo MarianiUniversity of Milano Bicocca, Matteo OrruUniversity of Milano Bicocca, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
20:30 - 21:00 Talk | Prioritizing Runtime Verification Violations Research Papers Breno MirandaFederal University of Pernambuco, Igor LimaFederal University of Pernambuco, Owolabi LegunsenCornell University, Marcelo d'AmorimFederal University of Pernambuco Link to publication DOI |
21:00 - 22:00: IT3 - Safety & SecurityIndustry Track at Farfetch (D. Maria) Chair(s): Rui AbreuFaculty of Engineering, University of Porto, Portugal | |||
21:00 - 21:30 Talk | Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems Industry Track Alessandro CalòTechnical University of Munich, Paolo ArcainiNational Institute of Informatics
, Shaukat AliSimula Research Laboratory, Florian HauerTechnical University of Munich, Fuyuki IshikawaNational Institute of Informatics Link to publication DOI | ||
21:30 - 22:00 Talk | Checking Security Properties of Cloud Service REST APIs Industry Track Vaggelis AtlidakisColumbia University, Patrice GodefroidMicrosoft Research, USA, Marina PolishchukMicrosoft Research, USA Link to publication DOI |
21:00 - 22:00: Testing toolsTesting Tools Track at Infante Chair(s): René JustUniversity of Washington, USA, Andy ZaidmanDelft University of Technology | |||
21:00 - 21:15 Talk | Callisto: Entropy-based Test Generation and Data Quality Assessment for Machine Learning Systems Testing Tools Track Sakshi Udeshi, Xingbin JiangSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design Link to publication DOI | ||
21:15 - 21:30 Talk | Run Java Applications and Test Them In-Vivo Meantime Testing Tools Track Antonia BertolinoCNR-ISTI, Guglielmo De AngelisCNR-IASI, CNR-ISTI, Breno MirandaFederal University of Pernambuco, Paolo TonellaUniversità della Svizzera Italiana (USI) Link to publication DOI | ||
21:30 - 21:45 Talk | AFLNet: A Greybox Fuzzer for Network Protocols Testing Tools Track Van-Thuan PhamMonash University, Marcel BöhmeMonash University, Australia, Abhik RoychoudhuryNational University of Singapore, Singapore Link to publication DOI | ||
21:45 - 22:00 Talk | ct-fuzz: Fuzzing for Timing Leaks Testing Tools Track Link to publication DOI |
Not scheduled yet
Not scheduled yet Talk | Smart - and also reliable and gas-efficient - Contracts Research Papers Elvira AlbertComplutense University of Madrid, Spain |
Call for Papers
ICST 2020 invites high quality submissions in all areas of software testing, verification, and validation. Papers for the research track should present novel and original work that advances the state-of-the-art. Case studies and empirical research papers are also welcome.
Topics of Interest
Topics of interest include, but are not limited to the following:
- Fuzz testing
- Manual testing practices and techniques
- Search based software testing
- Security testing
- Model based testing
- Test automation
- Static analysis and symbolic execution
- Formal verification and model checking
- Software reliability
- Testability and design
- Testing and development processes
- Testing education
- Testing in specific domains, such as mobile, web, embedded, concurrent, distributed, cloud, GUI and real-time systems
- Testing for learning-enabled software, including deep learning
- Testing/debugging tools
- Theory of software testing
- Empirical studies
- Experience reports
Each submission will be reviewed by at least three members of the ICST Program Committee.
Papers that have a strong industrial/practical component and focus more on impact rather than (technical) novelty are encouraged to consider the industry track instead.
Submission Format
Full Research papers as well as Industry papers must conform to the two-column IEEE conference publication format, not exceed 10 pages, including all text, figures, tables, and appendices; two additional pages containing only references are permitted. It must conform to the the IEEE Conference Proceedings Formatting Guidelines (please use the letter format template and conference option). The ICST 2020 research track only accepts full research papers. Short papers are not accepted to the research track.
The submission must also comply with the ACM plagiarism policy and procedures. In particular, it must not have been published elsewhere and must not be under review elsewhere while under review for ICST. The submission must also comply with the IEEE Policy on Authorship.
Lastly, the ICST 2020 Research papers track will employ a double-blind review process. Thus, no submission may reveal its authors’ identities. The authors must make every effort to honor the double-blind review process. In particular, the authors’ names must be omitted from the submission and references to their prior work should be in the third person. Further advice, guidance and explanation about the double-blind review process can be found in the Q&A page.
Submissions to the Research Papers Track that meet the above requirements can be made via the Research Papers Track submission site by the submission deadline. by the submission deadline.
Any submission that does not comply with the above requirements may be rejected by the PC Chairs without further review.
If a submission is accepted, at least one author of the paper is required to attend the conference and present the paper in person for the paper to be published in the ICST 2020 conference proceedings.
New in 2020. Submissions must supply all information that is needed to replicate the results, and therefore are expected to include or point to a replication package with the necessary software, data, and instructions. Reviewers may consult these packages to resolve open issues. There can be good reasons for the absence of a replication package, such as confidential code and/or data, the research being mostly qualitative, or the paper being fully self-contained. If a paper does not come with a replication package, authors should comment on its absence in the submission data; reviewers will take such comments into account.
Submission Site: https://easychair.org/my/conference?conf=icst2020
Submitting to ICST2020: Q&A
Please note that the Double Blind Review (DBR) process is not used by all tracks, e.g., Industry Track. Check in the call for papers whether DBR is used or not.
Q: Is the deadline firm? Can I ask for an extension?
Yes. No.
Q: How can I submit an anonymous replication package?
Set up an anonymous account at dropbox or a similar public file sharing service and place your (anonymized) package there. Include the URL in your submission (typically at the end).
Q: Do I have to provide a replication package?
There are good reasons not to provide a package. Typical reasons include (1) your approach is simple enough such that it can be fully described in the paper; (2) the concept of a replication package is not applicable because of your specific research methodology; or (3) parts of your research are confidential. Just state the reason in the paper, such that reviewers can assess it.
Q: How does one prepare an ICST 2020 submission for double-blind reviewing?
In order to comply, you do not have to make your identity undiscoverable; the double-blind aspect of the review process is not an adversarial identity discovery process. Essentially, the guiding principle should be to maximize the number of people who could plausibly be authors, subject to the constraint that no change is made to any technical details of the work. Therefore, you should ensure that the reviewers are able to read and review your paper without having to know who any of the authors are. Specifically, this involves at least the following four points:
- Omit all authors’ names, affiliations, emails and related information from the title page as well as in the paper itself.
- Refer to your own work in the third person. You should not change the names of your own tools, approaches or systems, since this would clearly compromise the review process. It breaks the constraint that “no change is made to any technical details of the work”. Instead, refer to the authorship or provenance of tools, approaches or systems in the third person, so that it is credible that another author could have written your paper.
- Anonymize project and grant names and numbers or those of funding agencies or countries as well as any acknowledgements of support to the work you report on.
- If necessary, set up an anonymous account at dropbox or a similar public file sharing service and place your (anonymized) package there. Include the URL in your submission (typically at the end).
We further expect you to follow the excellent advice on anonymization from ACM.
When anonymizing your email, affiliations, name, etc., try to refrain from being overly creative or “funny” by coming up with your own, anonymized versions. For emails preferably use [email protected], [email protected], etc., since initial DBR screening will be done by an automated tool.
Q: I previously published an earlier version of this work in a venue that does not have double-blind. What should I do about acknowledging that previous work?
Double-blind does not and cannot mean that it is impossible for the referees to discover the identity of the author. However, we require authors to help make it easy for author identity to not play a role in the reviewing process. Therefore, we ask that in the materials you submit to us to be reviewed author identity is not revealed.
If the work you are submitting for review has previously been published in a non-peer-reviewed venue (e.g., arXiv departmental tech report), there is no need to cite it, because unrefereed work is not truly part of the scientific literature. If the previous work is published in a peer-reviewed venue, then it should be cited, but in the third person so that it is not clear whether or not this work was done by the author of the submitted paper or some other set of authors unknown. However, if citing in the third person would still risk that it is easy to identify the authors please err on the side of caution by also anonymising the papers being extended (both when cited and in the reference list).
Q: Our submission makes use of work from a PhD/master’s thesis dissertation/report which has been published. Citing the dissertation might compromise anonymity. What should we do?
It is perfectly OK to publish work from a PhD/master’s thesis, and there is no need to cite it in the version submitted for review because prior dissertation publication does not compromise novelty. In the final (post-review, camera ready) version of the paper, please do cite the dissertation to acknowledge its contribution, but in the refereed version of the paper that you submit, please refrain from citing the dissertation.
However, you need not worry whether or not the dissertation has appeared, since your job is only to help the committee review your work without awareness of author identity, but not to make it impossible for them to discover the identity of authors. The referees will be trying hard not to discover the authors’ identity, so they will likely not be searching the web to check whether there is a dissertation related to this work.
Q: I am submitting to the industry track. Should I double-blind my submission?
No you should not. Since industry papers typically relies heavily on the industrial or practical context in which the work was carried out it would be too much to ask to require this context to be anonymized.
Q: I want to include a link to an online appendix in my submission. How should I do this?
Ideally the information in the appendix should be anonymous and it should be uploaded to an anonymous service such as for example figshare or create a new github (or other) sharing account that is not associated with your real name. These sites will give you a link that is anonymous. Later, if the paper is accepted you can turn that link into a non-anonymized link or just put the appendix on your own site and change the link in the camera ready version of the paper. An alternative solution is to not include the link in the submission; normally papers should be possible to review based on only the material of the paper itself.
To upload material on figshare please create an account there, then add a new item, use the keywords “Supplemental Materials” and add the other item-specific data and then select “Make file(s) confidential” and select “Generate private link”. Copy the url generated there and then “Save changes”. Your file(s) can now be accessed anonymously at the given url so you can put it in your ICST submission.
Q: What if we want to cite some unpublished work of our own (as motivation for example)
If the unpublished paper is an earlier version of the paper you want to submit to ICST and is currently under review, then you have to wait until your earlier version is through its review process before you can build on it with further submissions (this would be considered double-submission and violates ACM plagiarism policy and procedures). Otherwise, if the unpublished work is not an earlier version of the proposed ICST submission, then you should simply make it available on a website, for example, and cite it in the third person to preserve anonymity, as you are doing with others of your works.
Q: Can I disseminate non-blinded version of my submitted work by discussing it with colleagues, giving talks, publishing it at ArXiV, etc.?
You can discuss and present your work that is under submission at small meetings (e.g., job talks, visits to research labs, a Dagstuhl or Shonan meeting), but you should avoid broadly advertising it in a way that reaches the reviewers even if they are not searching for it. For example, you are allowed to put your submission on your home page and present your work at small professional meetings. However, you should not discuss your work with members of the program committee, publicize your work on mailing lists or media that are widely shared and can reach the program committee, or post your work on ArXiV or a similar site just before or after submitting to the conference.