arXiv Science⌕ Search

arXiv · 2307.12477

A Taxonomy for Requirements Engineering and Software Test Alignment

Abstract

Requirements Engineering and Software Testing are mature areas and have seen a lot of research. Nevertheless, their interactions have been sparsely explored beyond the concept of traceability. To fill this gap, we propose a definition of requirements engineering and software test (REST) alignment, a taxonomy that characterizes the methods linking the respective areas, and a process to assess alignment. The taxonomy can support researchers to identify new opportunities for investigation, as well as practitioners to compare alignment methods and evaluate alignment, or lack thereof. We constructed the REST taxonomy by analyzing alignment methods published in literature, iteratively validating the emerging dimensions. The resulting concept of an information dyad characterizes the exchange of information required for any alignment to take place. We demonstrate use of the taxonomy by applying it on five in-depth cases and illustrate angles of analysis on a set of thirteen alignment methods. In addition, we developed an assessment framework (REST-bench), applied it in an industrial assessment, and showed that it, with a low effort, can identify opportunities to improve REST alignment. Although we expect that the taxonomy can be further refined, we believe that the information dyad is a valid and useful construct to understand alignment.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Michael Unterkalmsteiner, Robert Feldt, Tony Gorschek. 2023-07-24. A Taxonomy for Requirements Engineering and Software Test Alignment. https://doi.org/10.1145/2523088

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code

Large language models (LLMs) frequently generate defective code, ranging from logical bugs to security vulnerabilities. Training-data quality is one contributor to these outcomes alongside model architecture, optimization and alignment, prompting and inference context, and decoding. However, the literature connecting specific training data quality issues to observable generated code quality issues has not been synthesized systematically. We review 114 included papers and establish taxonomies of eight generated-code quality dimensions and two broad groups of training-data issues. We then synthesize 12 recurring relationship families between training-data issues and generated-code outcomes, tracing each to its supporting papers, evidence basis, and limitations. We also synthesize detection and mitigation techniques across the data, model, and generation lifecycle. The findings position data curation as one layer of code-generation quality assurance coordinated with model- and inference-level controls, and identify provenance and controlled attribution as open research needs. Our repository is available at https://github.com/SYSUSELab/From-Data-to-Code.

cs.SE↗

Directed Symbolic Execution for Vulnerability Discovery: An LLM-Guided Approach in KLEE

Symbolic execution effectively discovers security violations but suffers from path explosion. Engines like KLEE therefore use path prioritization heuristics to order state exploration, typically optimizing code coverage. However, path prioritization can become trapped in cyclic control-flow regions, where repeated branching consumes the exploration budget before exploration reaches vulnerable code beyond these cyclic regions. We propose KLEECopilot, a Large Language Model (LLM)-guided directed symbolic execution approach built on KLEE. KLEECopilot uses LLMs to mark potentially vulnerable code and guide path prioritization. It also integrates loop-exit prioritization to escape potentially non-vulnerable cycles and progress toward deeper vulnerabilities. Compared with baselines such as Empc, KLEECopilot improves basic block coverage by 42.24% and line coverage by 125.82%. It discovers 1,335 total violations and 87 unique violations, outperforming the second-best baseline by 32.2% in total violations and Empc by 24.3% in unique violations. Although KLEECopilot is sensitive to model family, it exhibits only marginal sensitivity to model scale, supporting the efficacy of integrating security semantics and loop-exit prioritization. Ablation studies further show that individual components contribute to effectiveness: alternative configurations involving searchers, internal components, marking sources, and prompt variants yield only 54--61 unique violations, while KLEECopilot maintains competitive code coverage.

cs.SE↗

Safety Analysis of CPSs Over-the-air Updates: Test Case Prioritization and Selection

Over-the-air (OTA) updates for Cyber-Physical Systems (CPS) allow engineers to modify their software wirelessly. In the automotive domain, they enable manufacturers to update software without physical access to the vehicle. However, OTA updates can generate a large number of possible software configurations. For example, end users may decide whether or not to install any given OTA updates. The combinatorial explosion of these configurations hampers SW testing, a standard practice for detecting errors in CPS. To overcome this limitation, we propose CRISP (CRIticality-based Selection and Prioritization), a novel framework supporting test case prioritization and selection in the context of OTA updates. Our framework leverages test case robustness, i.e., how far they are from violating the system requirements, to drive test case prioritization and selection. It enables engineers to rank test case executions from most to least critical, and to select only the critical test cases relevant to a given OTA update. We evaluated our approach using a representative SW product line consisting of seven different vehicles, 22 driving scenarios, and two software components. We considered 224 tests, requiring 59 hours of execution time. We assessed the effectiveness of CRISP across five OTA updates. We compared our solution with (a) a random test case prioritization and selection algorithm, and (b) AP-DISC, identified as the state-of-the-art solution in a recent empirical study on the subject. Our results show that CRISP outperforms our baselines in finding failure-revealing test cases, and that our selection procedure can be configured to find reasonable and practical tradeoffs among accuracy (percentage of failure-revealing test cases identified) and computational time (time required to find them).

cs.SE↗