Roshan N. Rajapakse | arXiv Science

arXiv2026

A Mixed-Method Empirical Study of LLM Assistance in Software Engineering Workflows

Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their effects are often discussed without distinguishing between task types, developer seniority, and verification demands. This paper presents a mixed-method empirical study of LLM-assisted software engineering with first-year and fourth-year undergraduates. Phase 1 is a preliminary survey (n=157) that characterizes LLM exposure, reliance, and trust calibration among the two groups. Phase 2 is a task-based quasi-experiment with a purposive sample from both cohorts (n=20). Here, we compare AI-assisted and non-AI conditions on a structured set of software engineering tasks spanning implementation, constraint-driven algorithm selection, and architectural reasoning. We then analyze performance outcomes alongside behavioral traces captured via screen recording and a qualitative coding process. Survey results indicate widespread LLM adoption and substantial verification effort, alongside cohort differences in perceived LLM capability for constraint-heavy scenarios. The quasi-experiment further shows that AI assistance changes workflow structure. For example, participants frequently adopt AI-first task entry, copy-transfer integration, and AI-mediated debugging, whereas non-AI workflows rely more on documentation, prior templates, and iterative trial-error refinement. Overall, our findings suggest that the benefits of LLM assistance are task-dependent and mediated by expertise and verification practices, rather than by generation speed alone.

cs.SE↗

arXiv2021

Challenges and solutions when adopting DevSecOps: A systematic review

Context: DevOps has become one of the fastest-growing software development paradigms in the industry. However, this trend has presented the challenge of ensuring secure software delivery while maintaining the agility of DevOps. The efforts to integrate security in DevOps have resulted in the DevSecOps paradigm, which is gaining significant interest from both industry and academia. However, the adoption of DevSecOps in practice is proving to be a challenge. Objective: This study aims to systemize the knowledge about the challenges faced by practitioners when adopting DevSecOps and the proposed solutions reported in the literature. We also aim to identify the areas that need further research in the future. Method: We conducted a Systematic Literature Review of 54 peer-reviewed studies. The thematic analysis method was applied to analyze the extracted data. Results: We identified 21 challenges related to adopting DevSecOps, 31 specific solutions, and the mapping between these findings. We also determined key gap areas in this domain by holistically evaluating the available solutions against the challenges. The results of the study were classified into four themes: People, Practices, Tools, and Infrastructure. Our findings demonstrate that tool-related challenges and solutions were the most frequently reported, driven by the need for automation in this paradigm. Shift-left security and continuous security assessment were two key practices recommended for DevSecOps. Conclusions: We highlight the need for developer-centered application security testing tools that target the continuous practices in DevSecOps. More research is needed on how the traditionally manual security practices can be automated to suit rapid software deployment cycles. Finally, achieving a suitable balance between the speed of delivery and security is a significant issue practitioners face in the DevSecOps paradigm.

cs.SE↗