arXiv Science⌕ Search

arXiv subjects

Farnaz Delirie

Publications and source records attributed to Farnaz Delirie.

1 recordsLinked to original sources

Ask the Peers: Ranking NFR Coverage Gaps for Requirements Review

Missing or underrepresented non-functional requirements (NFR) categories leave little direct evidence in a requirements set, making them difficult to identify during requirements review. In this study, we frame NFR coverage-gap review as a project-level ranking task: given a target requirements specification, the goal is to rank the 11 PROMISE NFR categories for analyst inspection. We compare seven statistical, embedding-based, and large language model (LLM)-based configurations that differ in both ranking mechanism and evidence source, including target requirement statements, peer requirements, their semantic representation, and peer category statistics. We evaluate the configurations using controlled gap injection with leave-one-project-out evaluation on PROMISE, PROMISE+, PURE-Transport, and PURE-Business. Four configurations that use peer category statistics are effectively tied under complete removal (0.875-0.889 mean AUROC), despite different ranking mechanisms, and supplying those statistics to a direct LLM ranker produces the largest evidence gain. All configurations weaken when a category is present but underrepresented, and none is best for every NFR category. For count-changing category-level gaps, structured peer evidence matters more than ranker sophistication. The findings suggest that the rankings should guide analyst inspection rather than serve as automatic judgements of requirements completeness.

cs.SE↗