arXiv ScienceSearch

arXiv subjects

Sabry E. Farrag

Publications and source records attributed to Sabry E. Farrag.

3 recordsLinked to original sources

Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development

Coding agents author commits, open pull requests and push code in production repositories. Responsibility is settled in two layers that do not refer to each other: platform controls gating what an agent may do, and provider terms allocating responsibility for its output. Objective: Where the two layers disagree at a workflow event, and what each states there about authority, execution, verification, consequence and record. Method: A qualitative document study of 121 items archived byte-exact: documentation for four agentic coding tools, the platform controls on their output, and eighteen policy documents from seven providers, read by deductive content analysis against an a-priori system of five dimensions and nine workflow events. An independent second coder blind-recoded the verification-mechanism classification, a complete enumeration and not a sample (Cohen's kappa 0.81, n = 13, on whether a mechanism compels; 0.75, n = 12, on who performs it). Results: Cursor's terms make the user responsible for evaluating the use of any suggestion; its documentation describes a product that performs that evaluation and records it as an approval. No collected artifact records that the verification the terms make the user's duty took place. At merge two tools compel a person, one documents an agent approving below a configured risk threshold, and one only advises. Six of the fifteen mechanisms that compel a check have a default the vendor states; nine have one this study inferred. Readers who had not made them refuted five of the eight absence claims; three survived, one materially qualified. Conclusions: The terms attach duty and consequence to output as a class, the platform records events, nothing records the duty discharged. One vendor documents a product that forms the approval judgement, and the terms allocate consequence against the artifact regardless.

cs.SE

Last Translation Benchmark

For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, standard benchmarks for machine translation are approaching saturation. Further, automatic translation metrics are unreliable, vulnerable to reward-hacking, and provide unactionable assessments. Even gold human evaluation is not problem-free, because it often lacks reproducibility, objectivity, and scalability. Overall, this prevents us from tracking objective progress in the field and identifying pathways for improvement. We introduce the Last Translation Benchmark, a collection of human-authored and peer-reviewed examples (texts, images, audio, videos) that break leading machine translation models. We also present a new evaluation approach: each example comes with handcrafted verification rules describing concrete failure cases on that example, therefore allowing reliable and actionable future evaluation. The Last Translation Benchmark is a live dataset that accepts ongoing contributions. The latest version is LTBv1, containing accepted contributions prior to September 1st 2026, with future releases planned as new data is continuously collected.

cs.CL

The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development

Since 2022, AI-powered coding assistants have produced contradictory evidence: controlled studies report 20-56% productivity gains on well-scoped tasks, while the most rigorous RCT documents a 19% slowdown for experienced developers, and telemetry across 10,000+ developers shows 98% more pull requests but 91% longer review times with flat delivery metrics. This paper argues these findings constitute the Productivity-Reliability Paradox (PRP): a systematic phenomenon emerging from non-deterministic code generators and insufficient specification discipline. Through a multivocal literature review of 67 sources (2022-2026), this paper: (1) formally defines the PRP with three moderating variables (task abstraction, codebase maturity, developer experience) and two amplifying mechanisms (code review bottleneck, context window constraint); (2) proposes the AI-Augmented Methodology Taxonomy (AAMT), classifying six methodologies under three AI integration tiers; (3) introduces the Specification Governance Model (SGM), grounded in Transaction Cost Economics, with a practical governance decision guide; and (4) evaluates Spec Kit and TDAD as SGM instantiations via a four-month pilot study. Specification discipline, not model capability, is the binding constraint on AI-assisted software dependability.

cs.SE