arXiv subjects

Marcus Krellner

Publications and source records attributed to Marcus Krellner.

8 recordsLinked to original sources

arXiv2026

Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems

As the capabilities and adoption of Artificial Intelligence (AI) systems grow, trust in these AI systems is an increasingly urgent concern. Much research has focused on models of AI governance and has primarily examined incentives for safe development and effective regulation. Hence they typically represented users trust as a one-shot adoption choice rather than as a dynamic, evolving process shaped by repeated interactions. We instead model trust as the dynamic choice of reduced monitoring in a repeated, asymmetric interaction between users and AI developers, where checking developers' behaviour is costly. Using evolutionary game theory, we study how users' strategies of trust and developers' strategies of providing safe (compliant) or unsafe (non-compliant) AI co-evolve under different levels of monitoring cost and institutional regimes. We conduct the analysis on both imitation-based and learning-based perspectives, with the stochastic finite-population dynamics, the infinite-population replicator analysis and the reinforcement learning analysis. We find three robust long-run regimes: no adoption by users while developers provide unsafe AI, unsafe but widely adopted systems, and safe systems that are widely adopted. Only the last is desirable, and it arises when penalties for unsafe behaviour exceed the extra cost of safety and users can still afford to monitor at least occasionally. Our results formally support governance proposals that emphasise transparency, low-cost monitoring, and meaningful sanctions, and they show that neither regulation alone nor blind user trust is sufficient to prevent the drift towards unsafe or low-adoption outcomes.

cs.AI↗

arXiv2025

Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis

When developers of artificial intelligence (AI) products need to decide between profit and safety for the users, they likely choose profit. Untrustworthy AI technology must come packaged with tangible negative consequences. Here, we envisage those consequences as the loss of reputation caused by media coverage of their misdeeds, disseminated to the public. We explore whether media coverage has the potential to push AI creators into the production of safe products, enabling widespread adoption of AI technology. We created artificial populations of self-interested creators and users and studied them through the lens of evolutionary game theory. Our results reveal that media is indeed able to foster cooperation between creators and users, but not always. Cooperation does not evolve if the quality of the information provided by the media is not reliable enough, or if the costs of either accessing media or ensuring safety are too high. By shaping public perception and holding developers accountable, media emerges as a powerful soft regulator -- guiding AI safety even in the absence of formal government oversight.

cs.AI↗

arXiv2025

Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents

There is general agreement that fostering trust and cooperation within the AI development ecosystem is essential to promote the adoption of trustworthy AI systems. By embedding Large Language Model (LLM) agents within an evolutionary game-theoretic framework, this paper investigates the complex interplay between AI developers, regulators and users, modelling their strategic choices under different regulatory scenarios. Evolutionary game theory (EGT) is used to quantitatively model the dilemmas faced by each actor, and LLMs provide additional degrees of complexity and nuances and enable repeated games and incorporation of personality traits. Our research identifies emerging behaviours of strategic AI agents, which tend to adopt more "pessimistic" (not trusting and defective) stances than pure game-theoretic agents. We observe that, in case of full trust by users, incentives are effective to promote effective regulation; however, conditional trust may deteriorate the "social pact". Establishing a virtuous feedback between users' trust and regulators' reputation thus appears to be key to nudge developers towards creating safe AI. However, the level at which this trust emerges may depend on the specific LLM used for testing. Our results thus provide guidance for AI regulation systems, and help predict the outcome of strategic LLM agents, should they be used to aid regulation itself.

cs.AI↗

arXiv2025

Media and responsible AI governance: a game-theoretic and LLM analysis

This paper investigates the complex interplay between AI developers, regulators, users, and the media in fostering trustworthy AI systems. Using evolutionary game theory and large language models (LLMs), we model the strategic interactions among these actors under different regulatory regimes. The research explores two key mechanisms for achieving responsible governance, safe AI development and adoption of safe AI: incentivising effective regulation through media reporting, and conditioning user trust on commentariats' recommendation. The findings highlight the crucial role of the media in providing information to users, potentially acting as a form of "soft" regulation by investigating developers or regulators, as a substitute to institutional AI regulation (which is still absent in many regions). Both game-theoretic analysis and LLM-based simulations reveal conditions under which effective regulation and trustworthy AI development emerge, emphasising the importance of considering the influence of different regulatory regimes from an evolutionary game-theoretic perspective. The study concludes that effective governance requires managing incentives and costs for high quality commentaries.

cs.AI↗

arXiv2024

Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation

There is general agreement that some form of regulation is necessary both for AI creators to be incentivised to develop trustworthy systems, and for users to actually trust those systems. But there is much debate about what form these regulations should take and how they should be implemented. Most work in this area has been qualitative, and has not been able to make formal predictions. Here, we propose that evolutionary game theory can be used to quantitatively model the dilemmas faced by users, AI creators, and regulators, and provide insights into the possible effects of different regulatory regimes. We show that creating trustworthy AI and user trust requires regulators to be incentivised to regulate effectively. We demonstrate the effectiveness of two mechanisms that can achieve this. The first is where governments can recognise and reward regulators that do a good job. In that case, if the AI system is not too risky for users then some level of trustworthy development and user trust evolves. We then consider an alternative solution, where users can condition their trust decision on the effectiveness of the regulators. This leads to effective regulation, and consequently the development of trustworthy AI and user trust, provided that the cost of implementing regulations is not too high. Our findings highlight the importance of considering the effect of different regulatory regimes from an evolutionary game theoretic perspective.

cs.AI↗

arXiv2023

Words are not Wind -- How Public Joint Commitment and Reputation Solve the Prisoner's Dilemma

To achieve common goals, we often use joint commitments. Our commitment helps us to coordinate with our partners and assures them that their cooperative efforts will benefit themselves. However, if one of us can exploit the other's cooperation (as in the Prisoner's Dilemma), our commitment appears less useful. It cannot remove the temptation for our partners to exploit us. Using methods from evolutionary game theory, we study the function of joint commitments in the Prisoner's Dilemma. We propose a reputation system akin to indirect reciprocity, wherein agents observe interactions even when not directly involved. They judge cooperation as good and defection as bad, but, crucially, only if the parties involved had committed to cooperate. This results in stable cooperation even though judgments are made privately, which had been a weakness in previous models of indirect reciprocity. Our work shows that joint commitments have utility beyond coordination problems, which could explain their prevalence. The proposed link between joint commitments and reputation could also explain why some joint commitments are pointedly public, like wedding vows. A reputation-based mechanism might have been particularly relevant in our distant past, in which no institutions existed to enforce commitments.

cs.GT↗

arXiv2023

Ethics in rotten apples: A network epidemiology approach for active cyber defense

As Internet of Things (IoT) technology grows, so does the threat of malware infections. A proposed countermeasure, the use of benevolent "white worms" to combat malicious "black worms", presents unique ethical and practical challenges. This study examines these issues via network epidemiology models and simulations, considering the propagation dynamics of both types of worms in various network topologies. Our findings highlight the critical role of the rate at which white worms activate themselves, relative to the user's system update rate, as well as the impact of the network structure on worm propagation. The results point to the potential of white worms as an effective countermeasure, while underscoring the ethical and practical complexities inherent in their deployment.

physics.soc-ph↗

arXiv2023

We both think you did wrong -- How agreement shapes and is shaped by indirect reciprocity

Humans judge each other's actions, which at least partly functions to detect and deter cheating and to enable helpfulness in an indirect reciprocity fashion. However, most forms of judging do not only concern the action itself, but also the moral status of the receiving individual (to deter cheating it must be morally acceptable to withhold help from cheaters). This is a problem, when not everybody agrees who is good and who is bad. Although it has been widely acknowledged that disagreement may exist and that it can be detrimental for indirect reciprocity, the details of this crucial feature of moral judgments have never been studied in depth. We show, that even when everybody assesses individually (aka privately), some moral judgement systems (aka norms) can lead to high levels of agreement. We give a detailed account of the mechanisms which cause it and we show how to predict agreement analytically without requiring agent-based simulations, and for any observation rate. Finally, we show that agreement may increase or decrease reputations and therefore how much helpfulness (aka cooperation) occurs.

physics.soc-ph↗