arXiv subjects

Willem-Jan van den Heuvel

Publications and source records attributed to Willem-Jan van den Heuvel.

At least 19 recordsLinked to original sources

arXiv2026

Actionable Meta-Requirements for AI Robustness Auditing

AI-enabled systems must remain dependable when the conditions under which they operate change (e.g., due to input perturbations, distribution shifts, or changes in external dependencies). This makes robustness auditing of such systems important for judging whether the available evidence is sufficient to support a defined robustness conclusion. However, designing an AI robustness auditing framework is challenging. A key unresolved issue is determining the requirements such a framework should support; these requirements have not yet been consolidated. We address this gap through meta-requirements: higher-order requirements specifying what a class of auditing frameworks should support. In design science research, literature and domain experts are common sources of meta-requirements; accordingly, we derive the initial set from literature and validate and refine it with practitioners. Our systematic literature review retained 74 of 2,890 records and derived 43 meta-requirements across 16 categories. We evaluated them with 128 practitioners, obtaining 5,504 ratings and 586 substantive comments. Overall, 81.20% of ratings indicated agreement or strong agreement. In particular, 41 of the 43 literature-derived meta-requirements were classified as strongly supported. Practitioner feedback refined the requirements and identified an additional requirement concerning external components and supply chain dependencies. The final set comprises 44 meta-requirements across 16 categories, of which 43 received quantitative validation. These meta-requirements can help organizations implement AI robustness auditing frameworks, assess existing frameworks for completeness, and refine them.

cs.AI↗

arXiv2026

Chaos Engineering in the Wild: Findings from GitHub

Chaos Engineering (CE) enhances system resilience by deliberately injecting faults to uncover weaknesses before they cause outages. Despite the availability of many CE tools, little is known about how they are adopted and maintained in open-source software (OSS) projects. This study empirically characterizes the adoption, evolution, and practical use of CE tools in OSS projects, examining who adopts them, how they are maintained, and which fault types they target. We conducted a large-scale mining study of GitHub repositories associated with ten widely used CE tools. Starting from 5,410 candidate repositories, we systematically filtered and manually validated 1,275 records and analyzed repository metadata, commit histories, and documentation. We found that adoption is concentrated around a few tools, with Toxiproxy, Chaos Mesh, and Chaos Monkey accounting for 68.86% of the validated repositories. In terms of adopter context, development is the predominant repository purpose (56.55%), while industry represents the largest ownership category (34.82%), closely followed by personal repositories (33.49%). At the project level, activity varies substantially, with 51.42% of repositories having at most 50 commits. In terms of fault coverage, network faults (44.85%) and instance termination (29.96%) together account for 74.81% of the 2,410 observed fault instances, whereas application-level faults account for only 2.57%. Taken together, these findings suggest that practitioners should consider repository activity and fault coverage when selecting CE tools, while researchers should investigate the factors behind concentrated adoption and limited application-level experimentation.

cs.SE↗

arXiv2026

A Taxonomy of Runtime Faults in Model Context Protocol Servers

MCP (Model Context Protocol) enables LLMs (Large Language Models) to interact with external tools and data sources via a standardized protocol. Its rapid adoption in tool-augmented Artificial Intelligence (AI) workflows has introduced new reliability challenges, such as configuration parameters that are accepted but not enforced at runtime, leading to unintended default behavior, whose runtime fault characteristics remain empirically unexamined. We present the first empirical taxonomy of runtime faults in MCP servers. We manually analyzed 833 confirmed runtime fault threads identified through a repository-mining pipeline covering 473 actively maintained MCP server GitHub repositories and derived a taxonomy using a bottom-up open coding procedure. The taxonomy comprises 11 top-level categories and 27 subcategories (73 leaf fault types), covering recurrent failures across protocol interactions, tool invocations, schema enforcement, state management, model-provider integration, security validation, and timeouts or explicit cancellations of in-progress operations. To assess the taxonomy's external validity, we surveyed 55 MCP server developers. Respondents reported experiencing an average of 20 of the 27 fault subcategories, and no category remained unobserved. These results indicate that the taxonomy reflects widely observed runtime failures in MCP-based systems and shall assist AI software maintenance and evolution in the future.

cs.SE↗

arXiv2026

When Stakeholder-centric Requirements Engineering is Not Enough: An Action Research Study on Legacy System Modernisation

Legacy system modernisation is a major challenge in digital transformation, especially when organisations depend on long-lived, business-critical systems that are only partly understood. In such context, organisations must define future needs while determining what current systems actually do and which functions to retain, adapt, or replace. Modernisation is therefore not only a technical challenge but also a requirements engineering (RE) problem, shaped by stakeholder perspectives. This study examines how far stakeholder-centric RE can support gap analysis between the system-as-is and the system-to-be in a legacy modernisation context. We conducted an action research study in a multinational energy company engaged in system modernisation. In the study, we applied stakeholder-centric RE practices, including stakeholder identification, semi-structured elicitation interviews, agreement-building through the Delphi method, and prioritisation with the MoSCoW method. The results show that this process was effective in producing requirements stakeholders generally viewed as understandable and correct, but less effective in achieving agreement on how elicited requirements mapped to legacy system functionality. The findings suggest that stakeholder-centric RE is necessary but not sufficient, pointing to the need for uncertainty-aware, iterative, and evidence-based modernisation practices that combine stakeholder perspectives with manual and tool-assisted analysis of legacy systems.

cs.SE↗

arXiv2026

Sustainability of Open-Source Machine Learning Robustness Assessment Tools: A Repository Mining Study

Robustness evaluation is essential for deploying machine-learning (ML) systems in real-world settings, where models may face adversarial perturbations, distribution shifts, and other operational stressors. Many open-source tools, including Adversarial Robustness Toolbox, Foolbox, and Robustness Gym, support robustness testing and evaluation. However, little is known about how these tools are maintained, publicly engaged with, and sustained over time, even though practitioners may rely on them to select evaluation dependencies, reproduce robustness assessments, and provide evidence for AI assurance. We present an empirical study of the open-source robustness tooling ecosystem. Starting from a curated seed set derived from prior work, we systematically searched GitHub and identified 28 robustness-tool repositories. We analyzed repository artifacts to characterize observable community engagement, maintenance activity, and project longevity using established software-engineering metrics. Our results show that engagement and maintenance are unevenly distributed, with sustained activity concentrated in a small subset of repositories. At the data collection date of January 21, 2026, five repositories were classified as active, 22 as inactive, and one as archived. These findings highlight the need to treat robustness tools as evolving software systems.

cs.SE↗

arXiv2026

Internet-of-Things Architectures for Secure Cyber-Physical Spaces: the VISOR Experience Report

Internet of things (IoT) technologies are becoming a more and more widespread part of civilian life in common urban spaces, which are rapidly turning into cyber-physical spaces. Simultaneously, the fear of terrorism and crime in such public spaces is ever-increasing. Due to the resulting increased demand for security, video-based IoT surveillance systems have become an important area for research. Considering the large number of devices involved in the illicit recognition task, we conducted a field study in a Dutch Easter music festival in a national interest project called VISOR to select the most appropriate device configuration in terms of performance and results. We iteratively architected solutions for the security of cyber-physical spaces using IoT devices. We tested the performance of multiple federated devices encompassing drones, closed-circuit television, smart phone cameras, and smart glasses to detect real-case scenarios of potentially malicious activities such as mosh-pits and pick-pocketing. Our results pave the way to select optimal IoT architecture configurations -- i.e., a mix of CCTV, drones, smart glasses, and camera phones in our case -- to make safer cyber-physical spaces' a reality.

cs.CR↗

arXiv2026

"Show Me You Comply... Without Showing Me Anything": Zero-Knowledge Software Auditing for AI-Enabled Systems

Classical software verification and validation techniques, such as procedural audits, formal methods, or model documentation, are the traditional mechanisms used to achieve the verifiable accountability now required by regulations like the EU AI Act. These methods are either expensive or heavily manual, and ill-suited for the opaque, "black box" nature of most Artificial Intelligence (AI) models. A conflict arises: high auditability and verifiability are required by law, but such transparency conflicts with the need to protect the assets being audited (e.g., confidential data and proprietary models). This paper introduces ZKMLOps, an \ac{MLOps} verification framework that operationalizes Zero-Knowledge Proofs (ZKPs) within Machine-Learning Operations lifecycles; a ZKP allows a prover to convince a verifier that a statement is true without revealing any information about the statement itself. By integrating ZKP with established software engineering patterns, ZKMLOps provides a modular and repeatable process for generating verifiable cryptographic evidence-proofs of well-defined computational statements about the audited model and its inputs-that auditors can use as input to a regulatory compliance determination. We evaluate the framework along two dimensions. First, framework viability: orchestration overhead is bounded and stable across architecturally heterogeneous ZKP backends and models of increasing size. Second, cost-versus-assurance trade-offs: the audit-on-demand setting is the regime in which full zero-knowledge auditing is the appropriate tool, where it provides confidentiality and integrity guarantees that lighter-weight alternatives cannot match.

cs.SE↗

arXiv2026

Privacy Engineering: A Systematic Literature Review

Privacy obligations under GDPR increasingly shape software engineering. We synthesize 90 studies from 2018 to 2025 using a systematic review with thematic synthesis to chart privacy engineering. Thirteen dimensions form two recurrent cores: Privacy Enhancing Technologies (PETs) with Privacy Metrics (PM) and Verification and Testing (VT) and Governance and Accountability (GA) with Transparency and Communication (TC) and Organizational Measures (OM). Modeling and Specification (MS) mediates between the cores. Lifecycle mapping shows concentrations at requirements and design (MS, GA), at implementation and verification (PETs, VT, PM, TC) and at operation and decommissioning (GA, OM, Data Subject Rights Management (DSRM), Incident Response and Management (IRM), Lifelong Management (LM)). Handoffs link models to rules and tests, mechanisms to metrics and deployments such as enclaves and ledgers to governance records. Domains reweight but do not alter structure: healthcare weights GA with VT and PETs, IoT and edge weight PETs with VT and PM at device and edge, web measurement weights TC with VT, AI and ML weight PETs with PM. IRM, LM and Data Minimization and Purpose Limitation (DMPL) are less often primary foci, signaling priorities for future work. The results provide a practical map and a replication-ready scaffold for assessment and updates.

cs.SE↗

arXiv2025

IaC Generation with LLMs: An Error Taxonomy and A Study on Configuration Knowledge Injection

Large Language Models (LLMs) currently exhibit low success rates in generating correct and intent-aligned Infrastructure as Code (IaC). This research investigated methods to improve LLM-based IaC generation, specifically for Terraform, by systematically injecting structured configuration knowledge. To facilitate this, an existing IaC-Eval benchmark was significantly enhanced with cloud emulation and automated error analysis. Additionally, a novel error taxonomy for LLM-assisted IaC code generation was developed. A series of knowledge injection techniques was implemented and evaluated, progressing from Naive Retrieval-Augmented Generation (RAG) to more sophisticated Graph RAG approaches. These included semantic enrichment of graph components and modeling inter-resource dependencies. Experimental results demonstrated that while baseline LLM performance was poor (27.1% overall success), injecting structured configuration knowledge increased technical validation success to 75.3% and overall success to 62.6%. Despite these gains in technical correctness, intent alignment plateaued, revealing a "Correctness-Congruence Gap" where LLMs can become proficient "coders" but remain limited "architects" in fulfilling nuanced user intent.

cs.AI↗

arXiv2025

Reusability in MLOps: Leveraging Ports and Adapters to Build a Microservices Architecture for the Maritime Domain

ML-Enabled Systems (MLES) are inherently complex since they require multiple components to achieve their business goal. This experience report showcases the software architecture reusability techniques applied while building Ocean Guard, an MLES for anomaly detection in the maritime domain. In particular, it highlights the challenges and lessons learned to reuse the Ports and Adapters pattern to support building multiple microservices from a single codebase. This experience report hopes to inspire software engineers, machine learning engineers, and data scientists to apply the Hexagonal Architecture pattern to build their MLES.

cs.SE↗

arXiv2025

Sustainability of Machine Learning-Enabled Systems: The Machine Learning Practitioner's Perspective

Software sustainability is a key multifaceted non-functional requirement that encompasses environmental, social, and economic concerns, yet its integration into the development of Machine Learning (ML)-enabled systems remains an open challenge. While previous research has explored high-level sustainability principles and policy recommendations, limited empirical evidence exists on how sustainability is practically managed in ML workflows. Existing studies predominantly focus on environmental sustainability, e.g., carbon footprint reduction, while missing the broader spectrum of sustainability dimensions and the challenges practitioners face in real-world settings. To address this gap, we conduct an empirical study to characterize sustainability in ML-enabled systems from a practitioner's perspective. We investigate (1) how ML engineers perceive and describe sustainability, (2) the software engineering practices they adopt to support it, and (3) the key challenges hindering its adoption. We first perform a qualitative analysis based on interviews with eight experienced ML engineers, followed by a large-scale quantitative survey with 203 ML practitioners. Our key findings reveal a significant disconnection between sustainability awareness and its systematic implementation, highlighting the need for more structured guidelines, measurement frameworks, and regulatory support.

cs.SE↗

arXiv2025

MLOps with Microservices: A Case Study on the Maritime Domain

This case study describes challenges and lessons learned on building Ocean Guard: a Machine Learning-Enabled System (MLES) for anomaly detection in the maritime domain. First, the paper presents the system's specification, and architecture. Ocean Guard was designed with a microservices' architecture to enable multiple teams to work on the project in parallel. Then, the paper discusses how the developers adapted contract-based design to MLOps for achieving that goal. As a MLES, Ocean Guard employs code, model, and data contracts to establish guidelines between its services. This case study hopes to inspire software engineers, machine learning engineers, and data scientists to leverage similar approaches for their systems.

cs.SE↗

arXiv2025

Accountability of Robust and Reliable AI-Enabled Systems: A Preliminary Study and Roadmap

This vision paper presents initial research on assessing the robustness and reliability of AI-enabled systems, and key factors in ensuring their safety and effectiveness in practical applications, including a focus on accountability. By exploring evolving definitions of these concepts and reviewing current literature, the study highlights major challenges and approaches in the field. A case study is used to illustrate real-world applications, emphasizing the need for innovative testing solutions. The incorporation of accountability is crucial for building trust and ensuring responsible AI development. The paper outlines potential future research directions and identifies existing gaps, positioning robustness, reliability, and accountability as vital areas for the development of trustworthy AI systems of the future.

cs.SE↗

arXiv2025

Recursive Self-Similarity in Deep Weight Spaces of Neural Architectures: A Fractal and Coarse Geometry Perspective

This paper conceptualizes the Deep Weight Spaces (DWS) of neural architectures as hierarchical, fractal-like, coarse geometric structures observable at discrete integer scales through recursive dilation. We introduce a coarse group action termed the fractal transformation, $T_{r_k} $, acting under the symmetry group $G = (\mathbb{Z}, +) $, to analyze neural parameter matrices or tensors, by segmenting the underlying discrete grid $Ω$ into $N(r_k)$ fractals across varying observation scales $ r_k $. This perspective adopts a box count technique, commonly used to assess the hierarchical and scale-related geometry of physical structures, which has been extensively formalized under the topic of fractal geometry. We assess the structural complexity of neural layers by estimating the Hausdorff-Besicovitch dimension of their layers and evaluating a degree of self-similarity. The fractal transformation features key algebraic properties such as linearity, identity, and asymptotic invertibility, which is a signature of coarse structures. We show that the coarse group action exhibits a set of symmetries such as Discrete Scale Invariance (DSI) under recursive dilation, strong invariance followed by weak equivariance to permutations, alongside respecting the scaling equivariance of activation functions, defined by the intertwiner group relations. Our framework targets large-scale structural properties of DWS, deliberately overlooking minor inconsistencies to focus on significant geometric characteristics of neural networks. Experiments on CIFAR-10 using ResNet-18, VGG-16, and a custom CNN validate our approach, demonstrating effective fractal segmentation and structural analysis.

cs.NE↗

arXiv2024

Architectural Design Decisions for Self-Serve Data Platforms in Data Meshes

Data mesh is an emerging decentralized approach to managing and generating value from analytical enterprise data at scale. It shifts the ownership of the data to the business domains closest to the data, promotes sharing and managing data as autonomous products, and uses a federated and automated data governance model. The data mesh relies on a managed data platform that offers services to domain and governance teams to build, share, and manage data products efficiently. However, designing and implementing a self-serve data platform is challenging, and the platform engineers and architects must understand and choose the appropriate design options to ensure the platform will enhance the experience of domain and governance teams. For these reasons, this paper proposes a catalog of architectural design decisions and their corresponding decision options by systematically reviewing 43 industrial gray literature articles on self-serve data platforms in data mesh. Moreover, we used semi-structured interviews with six data engineering experts with data mesh experience to validate, refine, and extend the findings from the literature. Such a catalog of design decisions and options drawn from the state of practice shall aid practitioners in building data meshes while providing a baseline for further research on data mesh architectures.

cs.SE↗

arXiv2022

Automated Test-Case Generation for Solidity Smart Contracts: the AGSolT Approach and its Evaluation

Blockchain and smart contract technology are novel approaches to data and code management that facilitate trusted computing by allowing for development in a distributed and decentralized manner. Testing smart contracts comes with its own set of challenges which have not yet been fully identified and explored. Although existing tools can identify and discover known vulnerabilities and their interactions on the Ethereum blockchain through random search or symbolic execution, these tools generally do not produce test suites suitable for human oracles. In this paper, we present AGSOLT (Automated Generator of Solidity Test Suites). We demonstrate its efficiency by implementing two search algorithms to automatically generate test suites for stand-alone Solidity smart contracts, taking into account some of the blockchain-specific challenges. To test AGSOLT, we compared a random search algorithm and a genetic algorithm on a set of 36 real-world smart contracts. We found that AGSOLT is capable of achieving high branch coverage with both approaches and even discovered some errors in some of the most popular Solidity smart contracts on Github.

cs.SE↗

arXiv2020

DeepIaC: Deep Learning-Based Linguistic Anti-pattern Detection in IaC

Linguistic anti-patterns are recurring poor practices concerning inconsistencies among the naming, documentation, and implementation of an entity. They impede readability, understandability, and maintainability of source code. This paper attempts to detect linguistic anti-patterns in infrastructure as code (IaC) scripts used to provision and manage computing environments. In particular, we consider inconsistencies between the logic/body of IaC code units and their names. To this end, we propose a novel automated approach that employs word embeddings and deep learning techniques. We build and use the abstract syntax tree of IaC code units to create their code embedments. Our experiments with a dataset systematically extracted from open source repositories show that our approach yields an accuracy between0.785and0.915in detecting inconsistencies

cs.SE↗

arXiv2020

SDSN@RT: a middleware environment for single-instance multi-tenant cloud applications

With the Single-Instance Multi-Tenancy (SIMT) model for composite Software-as-a-Service (SaaS) applications, a single composite application instance can host multiple tenants, yielding the benefits of better service and resource utilization, and reduced operational cost for the SaaS provider. An SIMT application needs to share services and their aggregation (the application) among its tenants while supporting variations in the functional and performance requirements of the tenants. The SaaS provider requires a middleware environment that can deploy, enact and manage a designed SIMT application, to achieve the varied requirements of the different tenants in a controlled manner. This paper presents the SDSN@RT (Software-Defined Service Networks @ RunTime) middleware environment that can meet the aforementioned requirements. SDSN@RT represents an SIMT composite cloud application as a multi-tenant service network, where the same service network simultaneously hosts a set of virtual service networks (VSNs), one for each tenant. A service network connects a set of services, and coordinates the interactions between them. A VSN realizes the requirements for a specific tenant and can be deployed, configured, and logically isolated in the service network at runtime. SDSN@RT also supports the monitoring and runtime changes of the deployed multi-tenant service networks. We show the feasibility of SDSN@RT with a prototype implementation, and demonstrate its capabilities to host SIMT applications and support their changes with a case study. The performance study of the prototype implementation shows that the runtime capabilities of our middleware incur little overhead.

cs.DC↗