arXiv subjects

Markus Borg

Publications and source records attributed to Markus Borg.

At least 19 recordsLinked to original sources

arXiv2026

Code Health in LLM-Based Test Generation: Effectiveness and Token Efficiency

Coding agents powered by Large Language Models (LLMs) are now prominent in software engineering. Previous work has shown that AI tools perform better on high-quality source code that is easy to maintain. In this study, we investigate how the effectiveness of LLM-generated unit tests varies across maintainability levels measured by CodeScene's CodeHealth (CH). We assess test effectiveness using traditional coverage metrics and mutation score across Python, Java, and C++. Moreover, we study how code with different levels of CH translates into input tokens using common industrial tokenizers. Our results suggest that CH provides a weak but consistent signal of LLM-generated test effectiveness and is negatively correlated with input-token count. These findings provide further evidence for a relationship between maintainability and LLM-based software development.

cs.SE↗

arXiv2026

Technical Debt Friction for Maintenance Prioritization: An Industrial Multi-Case Study

Software-intensive organizations need effective ways to identify where maintenance and refactoring efforts will yield the greatest practical benefit. Although software analytics such as code health, hotspots, and coupling provide valuable signals, they do not always capture the experienced burden of change that slows software evolution in practice. This paper presents a multi-case industrial study of technical debt friction as a prioritization-oriented concept for identifying where technical debt most strongly affects maintenance and evolution. We investigate how practitioners interpret the concept, whether friction-related analysis aligns with perceived maintenance pain points and refactoring needs, and what broader maintenance and evolution insights friction can provide beyond individual refactoring candidates. To this end, we conducted structured walkthrough sessions with practitioners across multiple industrial cases using analysis artifacts including code health, hotspots, coupling, refactoring targets, and socio-technical views. Our findings show that practitioners generally considered technical debt friction useful for reasoning about maintenance burden, especially when interpreted together with complementary technical and socio-technical views. At the file level, friction often aligned with known problematic areas and, in several cases, with files that later received maintenance attention, although its practical relevance depended strongly on context. In addition, our exploratory project-level analysis suggests that friction distributions may reveal broader maintenance and evolution patterns. These results indicate that technical debt friction is promising as a decision-support concept, but most effective when used with contextual knowledge and supporting evidence.

cs.SE↗

arXiv2026

Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics

We are entering a hybrid era in which human developers and AI coding agents work in the same codebases. While industry practice has long optimized code for human comprehension, it is increasingly important to ensure that LLMs with different capabilities can edit code reliably. In this study, we investigate the concept of ``AI-friendly code'' via LLM-based refactoring on a dataset of 5,000 Python files from competitive programming. We find a meaningful association between CodeHealth, a quality metric calibrated for human comprehension, and semantic preservation after AI refactoring. Our findings confirm that human-friendly code is also more compatible with AI tooling. These results suggest that organizations can use CodeHealth to guide where AI interventions are lower risk and where additional human oversight is warranted. Investing in maintainability not only helps humans; it also prepares for large-scale AI adoption.

cs.SE↗

arXiv2025

QUPER-MAn: Benchmark-Guided Target Setting for Maintainability Requirements

Maintainable source code is essential for sustainable development in any software organization. Unfortunately, many studies show that maintainability often receives less attention than its importance warrants. We argue that requirements engineering can address this gap the problem by fostering discussions and setting appropriate targets in a responsible manner. In this preliminary work, we conducted an exploratory study of industry practices related to requirements engineering for maintainability. Our findings confirm previous studies: maintainability remains a second-class quality concern. Explicit requirements often make sweeping references to coding conventions. Tools providing maintainability proxies are common but typically only used in implicit requirements related to engineering practices. To address this, we propose QUPER-MAn, a maintainability adaption of the QUPER model, which was originally developed to help organizations set targets for performance requirements. Developed using a design science approach, QUPER-MAn, integrates maintainability benchmarks and supports target setting. We posit that it can shift maintainability from an overlooked development consequence to an actively managed goal driven by informed and responsible engineering decisions.

cs.SE↗

arXiv2025

ACE: Automated Technical Debt Remediation with Validated Large Language Model Refactorings

The remarkable advances in AI and Large Language Models (LLMs) have enabled machines to write code, accelerating the growth of software systems. However, the bottleneck in software development is not writing code but understanding it; program understanding is the dominant activity, consuming approximately 70% of developers' time. This implies that improving existing code to make it easier to understand has a high payoff and - in the age of AI-assisted coding - is an essential activity to ensure that a limited pool of developers can keep up with ever-growing codebases. This paper introduces Augmented Code Engineering (ACE), a tool that automates code improvements using validated LLM output. Developed through a data-driven approach, ACE provides reliable refactoring suggestions by considering both objective code quality improvements and program correctness. Early feedback from users suggests that AI-enabled refactoring helps mitigate code-level technical debt that otherwise rarely gets acted upon.

cs.SE↗

arXiv2025

Echoes of AI: Investigating the Downstream Effects of AI Assistants on Software Maintainability

[Context] AI assistants, like GitHub Copilot and Cursor, are transforming software engineering. While several studies highlight productivity improvements, their impact on maintainability requires further investigation. [Objective] This study investigates whether co-development with AI assistants affects software maintainability, specifically how easily other developers can evolve the resulting source code. [Method] We conducted a two-phase controlled experiment involving 151 participants, 95% of whom were professional developers. In Phase 1, participants added a new feature to a Java web application, with or without AI assistance. In Phase 2, a randomized controlled trial, new participants evolved these solutions without AI assistance. [Results] Phase 2 revealed no significant differences in subsequent evolution with respect to completion time or code quality. Bayesian analysis suggests that any speed or quality improvements from AI use were at most small and highly uncertain. Observational results from Phase 1 corroborate prior research: using an AI assistant yielded a 30.7% median reduction in completion time, and habitual AI users showed an estimated 55.9% speedup. [Conclusions] Overall, we did not detect systematic maintainability advantages or disadvantages when other developers evolved code co-developed with AI assistants. Within the scope of our tasks and measures, we observed no consistent warning signs of degraded code-level maintainability. Future work should examine risks such as code bloat from excessive code generation and cognitive debt as developers offload more mental effort to assistants.

cs.SE↗

arXiv2024

Trust Calibration in IDEs: Paving the Way for Widespread Adoption of AI Refactoring

In the software industry, the drive to add new features often overshadows the need to improve existing code. Large Language Models (LLMs) offer a new approach to improving codebases at an unprecedented scale through AI-assisted refactoring. However, LLMs come with inherent risks such as braking changes and the introduction of security vulnerabilities. We advocate for encapsulating the interaction with the models in IDEs and validating refactoring attempts using trustworthy safeguards. However, equally important for the uptake of AI refactoring is research on trust development. In this position paper, we position our future work based on established models from research on human factors in automation. We outline action research within CodeScene on development of 1) novel LLM safeguards and 2) user interaction that conveys an appropriate level of trust. The industry collaboration enables large-scale repository analysis and A/B testing to continuously guide the design of our research interventions.

cs.SE↗

arXiv2024

Industrial Code Quality Benchmarks: Toward Gamification of Software Maintainability

Software maintainability is essential for long-term success in the software industry. Despite widespread evidence of the high costs associated with poor maintainability, market pressure drives many organizations to prioritize short-term releases. This focus leads to accumulating technical debt worldwide. In this preliminary work, we propose maintainability gamification through anonymous leaderboards to encourage organizations to maintain a sustained focus on code quality. Our approach envisions benchmarking to foster motivation and urgency across companies by highlighting thresholds for leaders and laggards. To initiate this concept, we analyze a sample of over 1,000 proprietary projects using CodeHealth scores. By examining the distribution of these scores across various dimensions, we assess the feasibility of creating effective leaderboards. Findings from this study offer valuable insights for future design activities in maintainability gamification.

cs.SE↗

arXiv2024

Ghost Echoes Revealed: Benchmarking Maintainability Metrics and Machine Learning Predictions Against Human Assessments

As generative AI is expected to increase global code volumes, the importance of maintainability from a human perspective will become even greater. Various methods have been developed to identify the most important maintainability issues, including aggregated metrics and advanced Machine Learning (ML) models. This study benchmarks several maintainability prediction approaches, including State-of-the-Art (SotA) ML, SonarQube's Maintainability Rating, CodeScene's Code Health, and Microsoft's Maintainability Index. Our results indicate that CodeScene matches the accuracy of SotA ML and outperforms the average human expert. Importantly, unlike SotA ML, CodeScene also provides end users with actionable code smell details to remedy identified issues. Finally, caution is advised with SonarQube due to its tendency to generate many false positives. Unfortunately, our findings call into question the validity of previous studies that solely relied on SonarQube output for establishing ground truth labels. To improve reliability in future maintainability and technical debt studies, we recommend employing more accurate metrics. Moreover, reevaluating previous findings with Code Health would mitigate this revealed validity threat.

cs.SE↗

arXiv2024

Does Co-Development with AI Assistants Lead to More Maintainable Code? A Registered Report

[Background/Context] AI assistants like GitHub Copilot are transforming software engineering; several studies have highlighted productivity improvements. However, their impact on code quality, particularly in terms of maintainability, requires further investigation. [Objective/Aim] This study aims to examine the influence of AI assistants on software maintainability, specifically assessing how these tools affect the ability of developers to evolve code. [Method] We will conduct a two-phased controlled experiment involving professional developers. In Phase 1, developers will add a new feature to a Java project, with or without the aid of an AI assistant. Phase 2, a randomized controlled trial, will involve a different set of developers evolving random Phase 1 projects - working without AI assistants. We will employ Bayesian analysis to evaluate differences in completion time, perceived productivity, code quality, and test coverage.

cs.SE↗

arXiv2024

Requirements for Organizational Resilience: Engineering Developer Happiness

Can the right requirements boost developer satisfaction and happiness? We believe they can. In keeping with this issue's theme, "Well-Being for Resilience: Developers Thrive," we discuss the connection between the three keywords, well-being, resilience, and thriving. How could requirements engineering foster these qualities? While there hasn't been much research on this topic, we see opportunities for future work. Let's initiate the discussion!

cs.SE↗

arXiv2024

Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasets

Safety measures need to be systemically investigated to what extent they evaluate the intended performance of Deep Neural Networks (DNNs) for critical applications. Due to a lack of verification methods for high-dimensional DNNs, a trade-off is needed between accepted performance and handling of out-of-distribution (OOD) samples. This work evaluates rejecting outputs from semantic segmentation DNNs by applying a Mahalanobis distance (MD) based on the most probable class-conditional Gaussian distribution for the predicted class as an OOD score. The evaluation follows three DNNs trained on the Cityscapes dataset and tested on four automotive datasets and finds that classification risk can drastically be reduced at the cost of pixel coverage, even when applied on unseen datasets. The applicability of our findings will support legitimizing safety measures and motivate their usage when arguing for safe usage of DNNs in automotive perception.

cs.LG↗

arXiv2024

Increasing, not Diminishing: Investigating the Returns of Highly Maintainable Code

Understanding and effectively managing Technical Debt (TD) remains a vital challenge in software engineering. While many studies on code-level TD have been published, few illustrate the business impact of low-quality source code. In this study, we combine two publicly available datasets to study the association between code quality on the one hand, and defect count and implementation time on the other hand. We introduce a value-creation model, derived from regression analyses, to explore relative changes from a baseline. Our results show that the associations vary across different intervals of code quality. Furthermore, the value model suggests strong non-linearities at the extremes of the code quality spectrum. Most importantly, the model suggests amplified returns on investment in the upper end. We discuss the findings within the context of the "broken windows" theory and recommend organizations to diligently prevent the introduction of code smells in files with high churn. Finally, we argue that the value-creation model can be used to initiate discussions regarding the return on investment in refactoring efforts.

cs.SE↗

arXiv2024

Quality Requirements for Code: On the Untapped Potential in Maintainability Specifications

Quality requirements are critical for successful software engineering, with maintainability being a key internal quality. Despite significant attention in software metrics research, maintainability has attracted surprisingly little focus in the Requirements Engineering (RE) community. This position paper proposes a synergistic approach, combining code-oriented research with RE expertise, to create meaningful industrial impact. We introduce six illustrative use cases and propose three future research directions. Preliminary findings indicate that the established QUPER model, designed for setting quality targets, does not adequately address the unique aspects of maintainability.

cs.SE↗

arXiv2023

Summary of the 4th International Workshop on Requirements Engineering and Testing (RET 2017)

The RET (Requirements Engineering and Testing) workshop series provides a meeting point for researchers and practitioners from the two separate fields of Requirements Engineering (RE) and Testing. The long term aim is to build a community and a body of knowledge within the intersection of RE and Testing, i.e., RET. The 4th workshop was co-located with the 25th International Requirements Engineering Conference (RE'17) in Lisbon, Portugal and attracted about 20 participants. In line with the previous workshop instances, RET 2017 o ered an interactive setting with a keynote, an invited talk, paper presentations, and a concluding hands-on exercise.

cs.SE↗

arXiv2023

A multi-case study of agile requirements engineering and the use of test cases as requirements

Context: It is an enigma that agile projects can succeed 'without requirements' when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. Objective: We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. Method: We performed an iterative case study at three companies and collected data through 14 interviews and two focus groups. Results: The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. Conclusions: The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.

cs.SE↗

arXiv2023

Summary of the 3rd International Workshop on Requirements Engineering and Testing

The RET (Requirements Engineering and Testing) workshop series provides a meeting point for researchers and practitioners from the two separate fields of Requirements Engineering (RE) and Testing. The goal is to improve the connection and alignment of these two areas through an exchange of ideas, challenges, practices, experiences and results. The long term aim is to build a community and a body of knowledge within the intersection of RE and Testing, i.e. RET. The 3rd workshop was held in co-location with REFSQ 2016 in Gothenburg, Sweden. The workshop continued in the same interactive vein as the predecessors and included a keynote, paper presentations with ample time for discussions, and panels. In order to create an RET knowledge base, this crosscutting area elicits contributions from both RE and Testing, and from both researchers and practitioners. A range of papers were presented from short positions papers to full research papers that cover connections between the two fields.

cs.SE↗

arXiv2023

An Industrial Case Study on Test Cases as Requirements

It is a conundrum that agile projects can succeed 'without requirements' when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understand how test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study. The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.

cs.SE↗