arXiv Science⌕ Search

arXiv subjects

Benedetta Donato

Publications and source records attributed to Benedetta Donato.

6 recordsLinked to original sources

STORY2DIFF: A Dataset Linking Real-World User Stories to Code Changes

Research on requirements-to-code tasks needs examples in which a genuine requirement is paired with the change that implemented it. Existing user-story collections, however, largely contain isolated or synthetic text. To address this limitation, we present STORY2DIFF, a dataset of 7,785 English user stories linked to merged pull requests from January 2017 to July 2026. The dataset is mined from a set of 982 public Java and Python repositories. Each record in the dataset combines the original issue, extracted Connextra elements, the associated pull request and commit metadata, a file-level diff capturing the code changes introduced by the associated pull request, and the affected files before and after the change. The tool we built to mine STORY2DIFF can be used to repeat the automatic extraction of the records, using different parameters (e.g., a different timeframe). The dataset can support a variety of tasks, including requirement discovery, traceability, change- impact estimation, and requirements-aware code generation. Dataset: https://github.com/riganelli/user-story-mining-pipeline/ releases/tag/V4.0.1. Video: https://youtu.be/qOhXUeyeiTQ.

cs.SE↗

PR4Code: A Pull Requests Dataset for AI Code Generation

AI-driven code generation is evolving from handling small, localized code-completion tasks toward supporting the implementation of full software features. To support this transition, this paper introduces PR4Code, a dataset of 4,508 Java and 8,831 Python curated Pull Requests (PRs) collected from GitHub, each enriched with metadata, commit histories, and detailed code changes. Unlike resources focused on isolated code fragments, PR4Code captures the feature implementation process, delivering a large and diverse collection of real-world development tasks that can be used to assess AI-driven solutions. Our analysis reveals substantial variability in PR structure, commit granularity, and textual content. In addition, we release scripts to regenerate and update the dataset, ensuring reproducibility and maintainability.

cs.SE↗

Software Engineering for AI-Driven Science

AI has made significant contributions to scientific discovery. Autonomous agents can now independently analyse complex problems, design experiments, write and execute code, analyse findings, and derive scientific conclusions. Nevertheless, errors in AI-generated code, experimental procedures, or evidence interpretation can lead to invalid research conclusions, undermining the reliability and reproducibility of automated scientific discovery. In this vision paper, we propose Software Engineering for AI-Driven Science (SE4AI4Sci) as an emerging research area for addressing this challenge. As an organising framework, we introduce the Scientific Assurance Chain, connecting intent, experimental design, implementation, execution, evidence, and claim. Through six documented failures in recent AI-for-science systems, we show where this chain can break, where existing assurance mechanisms fall short, and what new research challenges follow. We also outline a research agenda for adapting and extending software engineering techniques across the chain.

cs.SE↗

MultiMind: A Plug-in for the Implementation of Development Tasks Aided by AI Assistants

The integration of AI assistants into software development workflows is rapidly evolving, shifting from automation-assisted tasks to collaborative interactions between developers and AI. Large Language Models (LLMs) have demonstrated their effectiveness in several development activities, including code completion, test case generation, and documentation production. However, embedding AI-assisted tasks within Integrated Development Environments (IDEs) presents significant challenges. It requires designing mechanisms to invoke AI assistants at the appropriate time, coordinate interactions with multiple assistants, process the generated outputs, and present feedback in a way that seamlessly integrates with the development workflow. To address these issues, we introduce MultiMind, a Visual Studio Code plug-in that streamlines the creation of AI-assisted development tasks. MultiMind provides a modular and extensible framework, enabling developers to cost-effectively implement and experiment with new AI-powered interactions without the need for complex IDE customizations. MultiMind has been tested in two use cases: one for the automatic generation of code comments and the other about the definition of AI-powered chat.

cs.SE↗

Studying How Configurations Impact Code Generation in LLMs: the Case of ChatGPT

Leveraging LLMs for code generation is becoming increasingly common, as tools like ChatGPT can suggest method implementations with minimal input, such as a method signature and brief description. Empirical studies further highlight the effectiveness of LLMs in handling such tasks, demonstrating notable performance in code generation scenarios. However, LLMs are inherently non-deterministic, with their output influenced by parameters such as temperature, which regulates the model's level of creativity, and top-p, which controls the choice of the tokens that shall appear in the output. Despite their significance, the role of these parameters is often overlooked. This paper systematically studies the impact of these parameters, as well as the number of prompt repetitions required to account for non-determinism, in the context of 548 Java methods. We observe significantly different performances across different configurations of ChatGPT, with temperature having a marginal impact compared to the more prominent influence of the top-p parameter. Additionally, we show how creativity can enhance code generation tasks. Finally, we provide concrete recommendations for addressing the non-determinism of the model.

cs.SE↗