From Prompting to Engineering: A Research Agenda for Prompt Engineering in Software Engineering

Vincenzo De Martino; Giovanna Broccia; Fabiano Pecorelli; Jennifer Horkoff; Riccardo Coppola; Antonino Ferraro; Quim Motger; Emma McKenzie; Shahbaz Siddeeq

arXiv · 2609.02248

From Prompting to Engineering: A Research Agenda for Prompt Engineering in Software Engineering

Vincenzo De Martino·Giovanna Broccia·Fabiano Pecorelli·Jennifer Horkoff·Riccardo Coppola·Antonino Ferraro·Quim Motger·Emma McKenzie·Shahbaz Siddeeq

View original source ↗Read source document ↗Export metadata ↓

Abstract

Prompt engineering is increasingly used across Software Engineering (SE) activities, including requirements analysis, coding, testing, documentation, repository analysis, and planning. Yet prompts and related instruction artifacts are often created and evolved through task-specific and informal practices, with limited support for their systematic evaluation, management, traceability, and governance. To examine how SE can contribute to the maturation of these practices, we organized a structured community discussion at the First International Workshop on Empirical Prompt Engineering for Software Engineering (PROMPT-SE), co-located with EASE 2026. Participants discussed current prompting practices, challenges to their adoption and evaluation, and future directions for integrating prompt engineering into software development. We synthesized these discussions into five areas: prompt artifacts and standardization; evaluation and benchmarking; lifecycle integration; human-AI collaboration and skills; and governance, privacy, and technical debt. Based on these areas, we outline a research agenda to move prompt engineering from predominantly ad hoc interactions toward more systematic, maintainable, evaluable, traceable, and governable SE practices.

Explore related subjects

cs.SE

Keep this discovery

BibTeX RIS

Vincenzo De Martino, Giovanna Broccia, Fabiano Pecorelli, Jennifer Horkoff, Riccardo Coppola, Antonino Ferraro, Quim Motger, Emma McKenzie, Shahbaz Siddeeq. 2026-09-02. From Prompting to Engineering: A Research Agenda for Prompt Engineering in Software Engineering. https://arxiv.org/abs/2609.02248

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.SE subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researchers

AI-assisted qualitative data analysis (QDA) offers unprecedented opportunities to streamline software engineering (SE) research, yet uncritical use risks compromising analytical rigor and flooding the field with accelerated production of low-quality research. While tactical best practices will naturally evolve over time, SE researchers currently lack strategic guidance to identify and mitigate methodological risks when attempting AI-assisted QDA. Based on our decades of qualitative SE research expertise and experience combined with an understanding of the emerging landscape of AI-assisted QDA, this paper presents a catalog of antipatterns in AI-assisted QDA - a set of assumptions and practices that initially appear advantageous but ultimately undermine analytical rigor and validity. The antipatterns are grouped into three categories reflecting escalating impact: Dangerous Drivers, Operational Missteps, and Analytical Failures. As more SE researchers attempt AI-assisted QDA, these antipatterns will help them identify and avoid common temptations and pitfalls, while reviewers can be equipped with the vocabulary and criteria to call out problematic and failed practice. Ultimately, this catalog of antipatterns can serve as a stepping stone in our responsible methodological evolution toward principled and meaningful human-AI collaboration in qualitative research.

cs.SE↗

arXiv2026

What Does an Evaluation License? A Commit-Bound Census of Claim Replay in Inspect Evals

Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historical claims. Most units stop because the evidence required for replay is not bound. Where replay is possible, different claims remain stable at different resolutions. We make this otherwise implicit inference step explicit and executable.

cs.SE↗

arXiv2026

Essence Coach: A Bot for Software Practice Adoption

Although Essence has been proposed as a unifying framework for understanding and evaluating software engineering practices, its adoption remains challenging, calling for tools that can act as coaches for learners and practitioners. We present Essence Coach, a chatbot that integrates LLMs with retrieval-augmented generation (RAG) from a curated Essence knowledge base. Experiments with multiple LLMs show that RAG consistently improves response quality for Essence-related queries. Some preliminary tests suggest that such systems can help both technical and non-technical students understand and apply Essence concepts. While validation through broader user studies is needed, this work-in-progress highlights how LLMs can bridge abstract SE frameworks and practice adoption.

cs.SE↗