Keith Webster | arXiv Science

arXiv2026

SE Research is a Complex Ecosystem: Isolated Fixes Keep Failing -- and Systems Thinking Shows Why

The software engineering research community is productive, yet it faces a constellation of challenges: swamped review processes, metric-driven incentives, distorted publication practices, and increasing pressures from AI, scale, and outright scams. These issues are often treated in isolation, yet they arise from deep structural dynamics within the research ecosystem itself and distract us from the larger role of research in society. Meaningful progress requires a holistic system-level view. We sketch such a framework drawing on ideas from complex systems, ecosystems, and theory of change. Reframing SE's challenges through this lens reveals non-linear feedback loops that sustain current dysfunctions, and it helps to identify leverage points for reform. These are less a matter of isolated fixes and more a matter of exploring coordinated sets of fixes that operate across the SE ecosystem

cs.SE↗

arXiv2015

Standing Together for Reproducibility in Large-Scale Computing: Report on reproducibility@XSEDE

This is the final report on reproducibility@xsede, a one-day workshop held in conjunction with XSEDE14, the annual conference of the Extreme Science and Engineering Discovery Environment (XSEDE). The workshop's discussion-oriented agenda focused on reproducibility in large-scale computational research. Two important themes capture the spirit of the workshop submissions and discussions: (1) organizational stakeholders, especially supercomputer centers, are in a unique position to promote, enable, and support reproducible research; and (2) individual researchers should conduct each experiment as though someone will replicate that experiment. Participants documented numerous issues, questions, technologies, practices, and potentially promising initiatives emerging from the discussion, but also highlighted four areas of particular interest to XSEDE: (1) documentation and training that promotes reproducible research; (2) system-level tools that provide build- and run-time information at the level of the individual job; (3) the need to model best practices in research collaborations involving XSEDE staff; and (4) continued work on gateways and related technologies. In addition, an intriguing question emerged from the day's interactions: would there be value in establishing an annual award for excellence in reproducible research?

cs.DC↗