arXiv Science⌕ Search

arXiv subjects

Publio Silva

Publications and source records attributed to Publio Silva.

2 recordsLinked to original sources

Exploring Test Smells Across Programming Languages: A Systematic Mapping Study

Tests are essential for ensuring code quality in software development. However, poor implementation practices can compromise the maintainability and evolution of test code, leading to additional costs and effort. These practices, known as test smells, have been the focus of numerous studies that expanded the catalog of known test smells. Despite this progress, little attention has been given to examining test smells from the perspective of programming languages, particularly their relationship with specific languages or test frameworks. As a result, researchers and practitioners have proposed various approaches and tools to detect and refactor the same test smells across different languages. In this work, we address this gap by analyzing test smells in the literature from a programming language perspective. We conducted a systematic mapping study, gathering data from papers published up to August 2025, examining 117 papers. We identified the most prevalent languages and frameworks associated with test smells, the most common test smells in each language, refactorings for addressing test smells, datasets of test smells, the criticality of specific test smells, and a collection of tools to detect or refactor test smells for various languages and frameworks. Our results show that while many test smells have been addressed, research remains limited to a few languages and frameworks. We also found datasets and detection/refactoring tools for some languages and analyzed the criticality of frequently mentioned test smells. This study highlights the need to expand research to more languages and develop more universal solutions.

cs.SE↗

Quality Assessment of Python Tests Generated by Large Language Models

The manual generation of test scripts is a time-intensive, costly, and error-prone process, indicating the value of automated solutions. Large Language Models (LLMs) have shown great promise in this domain, leveraging their extensive knowledge to produce test code more efficiently. This study investigates the quality of Python test code generated by three LLMs: GPT-4o, Amazon Q, and LLama 3.3. We evaluate the structural reliability of test suites generated under two distinct prompt contexts: Text2Code (T2C) and Code2Code (C2C). Our analysis includes the identification of errors and test smells, with a focus on correlating these issues to inadequate design patterns. Our findings reveal that most test suites generated by the LLMs contained at least one error or test smell. Assertion errors were the most common, comprising 64% of all identified errors, while the test smell Lack of Cohesion of Test Cases was the most frequently detected (41%). Prompt context significantly influenced test quality; textual prompts with detailed instructions often yielded tests with fewer errors but a higher incidence of test smells. Among the evaluated LLMs, GPT-4o produced the fewest errors in both contexts (10% in C2C and 6% in T2C), whereas Amazon Q had the highest error rates (19% in C2C and 28% in T2C). For test smells, Amazon Q had fewer detections in the C2C context (9%), while LLama 3.3 performed best in the T2C context (10%). Additionally, we observed a strong relationship between specific errors, such as assertion or indentation issues, and test case cohesion smells. These findings demonstrate opportunities for improving the quality of test generation by LLMs and highlight the need for future research to explore optimized generation scenarios and better prompt engineering strategies.

cs.SE↗