arXiv subjects

Simon WS Fischer

Publications and source records attributed to Simon WS Fischer.

3 recordsLinked to original sources

arXiv2026

Friction in AI-Assisted Clinical Decision-Making: A Case Study on The Role of Questions and 'What-if' Scenarios

Clinical decision-making is augmented by decision-support systems (DSSs). To counter overreliance on DSSs, several methods have been proposed that create friction in order to promote cognitive engagement and reflection. In this paper, we investigate how two such forms of friction, namely data-driven questions and `what-if' analysis, are perceived by medical experts. For a real-world decision task, we replicated a DSS used in clinical practice and gathered clinicians' feedback on a prototype through in-situ interviews (n=7). Our findings suggest that while the questions were perceived as unhelpful for reflective thinking, they could serve as reminders to consider relevant information. Furthermore, inspecting `what-if' hypotheticals was found useful for potentially improving patient care. Clinicians saw our prototype as a promising training tool for novice clinicians. From the clinicians' feedback, we make recommendations for designing friction in work practices. Our work contributes to human-AI interaction research, which aims to encourage reflection to mitigate AI overreliance.

cs.HC↗

arXiv2026

Supporting Reflection and Forward-Looking Reasoning With Data-Driven Questions

Many generative AI systems as well as decision-support systems (DSSs) provide operators with predictions or recommendations. Various studies show, however, that people can mistakenly adopt the erroneous results presented by those systems. Hence, it is crucial to promote critical thinking and reflection during interaction. One approach we are focusing on involves encouraging reflection during machine-assisted decision-making by presenting decision-makers with data-driven questions. In this short paper, we provide a brief overview of our work in that regard, namely: 1) the development of a question taxonomy, 2) the development of a prototype in the medical domain and the feedback received from clinicians, 3) a method for generating questions using a large language model, and 4) a proposed scale for measuring cognitive engagement in human-AI decision-making. In doing so, we contribute to the discussion about the design, development, and evaluation of tools for thought, i.e., AI systems that provoke critical thinking and enable novel ways of sense-making.

cs.HC↗

arXiv2024

Questioning AI: Promoting Decision-Making Autonomy Through Reflection

Decision-making is increasingly supported by machine recommendations. In healthcare, for example, a clinical decision support system is used by the physician to find a treatment option for a patient. In doing so, people can rely too much on these systems, which impairs their own reasoning process. The European AI Act addresses the risk of over-reliance and postulates in Article 14 on human oversight that people should be able "to remain aware of the possible tendency of automatically relying or over-relying on the output". Similarly, the EU High-Level Expert Group identifies human agency and oversight as the first of seven key requirements for trustworthy AI. The following position paper proposes a conceptual approach to generate machine questions about the decision at hand, in order to promote decision-making autonomy. This engagement in turn allows for oversight of recommender systems. The systematic and interdisciplinary investigation (e.g., machine learning, user experience design, psychology, philosophy of technology) of human-machine interaction in relation to decision-making provides insights to questions like: how to increase human oversight and calibrate over- and under-reliance on machine recommendations; how to increase decision-making autonomy and remain aware of other possibilities beyond automated suggestions that repeat the status-quo?

cs.HC↗