Identifying Flakiness in Quantum Programs

Thumbnail Image

Files

2302.03256.pdf (202.68 KB)

Links to Files

https://arxiv.org/abs/2302.03256

Permanent Link

https://doi.org/10.48550/arXiv.2302.03256
 http://hdl.handle.net/11603/26942

Collections

UMBC Information Systems Department
UMBC Faculty Collection

Author/Creator

Author/Creator ORCID

https://orcid.org/0000-0002-4424-4942

Date

2023-02-07

Type of Work

journal articles
preprints
Text

Department

Program

Citation of Original Publication

Rights

This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

Subjects

Abstract

In recent years, software engineers have explored ways to assist quantum software programmers. Our goal in thi s paper is to continue this exploration and see if quantum soft - ware programmers deal with some problems plaguing classica l programs. Specifically, we examine whether intermittently failing tests, i.e., flaky tests, affect quantum software development. To explore flakiness, we conduct a preliminary analysis of 14 quantum software repositories. Then, we identify flaky tests and categorize their causes and methods of fixing them. We found flaky tests in 12 out of 14 quantum software repositories. In these 12 repositories, the lower boundary of the percentage of flaky tests ranges between 0.26% and 1.85% per repository. We identified 38 distinct flaky tests with 10 groups of causes and 7 common solutions. Further, we notice that quantum programmers are not using some of the recent flaky test countermeasures developed by software engineers. This work may interest practitioners, as it provides useful insight into the resolution of flaky tests in quantum programs. Researchers may also find the paper helpful as it offers quantitative data on flaky tests in quantum software and points to new research opportunities.