The IRMA Community
Newsletters
Research IRM
Click a keyword to search titles using our InfoSci-OnDemand powered search:
|
Benchmarking Software Test Cases Produced by Generative AI Against Traditional Methods: A Design Science Approach
|
|
Author(s): Mark L. Gillenson (University of Memphis, USA), Pavankumar Mulgund (University of Memphis, USA)and Ankur Arora (University of Memphis, USA)
Copyright: 2026
Volume: 37
Issue: 1
Pages: 32
Source title:
Journal of Database Management (JDM)
Editor(s)-in-Chief: Keng Siau (Singapore Management University, Singapore)
DOI: 10.4018/JDM.409970
Purchase
|
Abstract
As artificial intelligence (AI) continues to shape commercial and governmental innovation, understanding the potential of large language models (LLMs) to improve software testing has become increasingly critical. This paper explores the application of LLMs for generating test cases—an essential yet resource-intensive activity in software quality assurance. Using theoretical lenses of the Design Science Research Evaluation (DSRE) framework, Measurement Theory, and Cognitive Load Theory, the authors systematically design and evaluate procedures for test case generation and execution to benchmark AI-generated test cases against those produced by human testers and traditional pairwise analysis. They develop two custom-built applications seeded with intentional defects and execute test cases generated through each method. This research contributes to the evolving discourse on AI-driven software engineering, offering insights into when LLMs can autonomously generate test cases, when human expertise is indispensable, and when hybrid approaches yield the greatest value.
Related Content
|
Yang Liu, May Almousa, Mohd Anwar.
© 2026.
23 pages.
|
|
Nitasha Hasteer, Mahak Singh, Rahul Sindhwani, Justin Z. Zhang, David Yulong Liu.
© 2026.
32 pages.
|
|
Lavlin Agrawal, Pavankumar Mulgund, Richelle Oakley DaSouza, Srikanth Venkatesan, Pankaj Chaudhary.
© 2026.
36 pages.
|
|
Mark L. Gillenson, Pavankumar Mulgund, Ankur Arora.
© 2026.
32 pages.
|
|
Kyle Nash.
© 2026.
22 pages.
|
|
Keshav Kaushik, Akashdeep Bhardwaj, Xiaochun Cheng, Susheela Dahiya, Achyut Shankar, Manoj Kumar, Tushar Mehrotra.
© 2025.
21 pages.
|
|
Jianyu Li, Peizhong Yang, Kun Yue, Liang Duan, Zehao Huang.
© 2025.
24 pages.
|
|
|