arXiv subjects

Benjamin Olsen

Publications and source records attributed to Benjamin Olsen.

3 recordsLinked to original sources

arXiv2026

FaithfulBench: Does AI Counsel Uphold or Undermine the User's Professed Faith?

Do AI assistants help believers reason about moral dilemmas consistently with their faith? We present FaithfulBench, the first benchmark to score AI counsel across traditions by how well it adheres to the user's professed faith. Scenarios are drawn from each tradition's most respected texts, with the faithful answer known and applied by the judges as the standard. We test five frontier models under three conditions: the AI does not know the user's tradition; it receives a one-line prompt identifying the user as a practicing adherent; or it receives a companion-counselor guide rooted in the tradition's sources. Two judges score the initial response and whether the model caves or holds when pressured toward the answer the user wants. When the tradition is unstated, models counsel from a secular therapeutic default and every model fails some believers. Naming the faith wins a faithful first answer but not steadfastness; the guide improves both.

cs.HC↗

arXiv2026

JaleesBench: Are AI Assistants Good Spiritual Company?

Large language models are already advisors to millions of people of faith who bring them real decisions. The pressing question for a person of faith is not what a model knows or professes but what its counsel does to the person who receives it. We introduce JaleesBench, which measures whether an AI agent is a righteous companion, judged by the residue an exchange leaves on the user, in the manner of the perfume-seller and the blacksmith. It comprises 140 two-turn scenarios drawn from a classical compilation organized by virtue (Riyad al-Salihin), under six adversarial pressures and three framings, scored by two frontier judges against each scenario's own supporting texts. Across eight systems: (1) generic frontier models are only middling companions out of the box but a one-page guide makes them genuinely good ones, on par with the domain-tuned assistant: the frontier APIs climb from +0.28/+0.23 to a Guided +0.84-0.87, so most of the expert's edge is companionship instruction that fits in a prompt; (2) every system caves under relational pressure, insistence and personal appeal; (3) the domain-tuned assistant's advantage is overwhelmingly its retrieval-and-prompting layer, not its base model (+0.74 over the identical underlying model); and (4) it can be used to improve existing systems: guided by its diagnosis, a single steadfastness instruction lifts a deployed Islamic assistant from +0.48 to +0.84 (Faith unstated, after pressure), matching the best guided frontier systems while preserving first-response quality. The construct is faith-general; we instantiate it for Islam as the first of a planned cross-tradition family. Code, scenario bank, and rubric are open source (github.com/iaser-ai/jaleesbench), with an interactive results browser at s.iaser.ai/jb.

cs.HC↗

arXiv2025

(Working Paper) Good Faith Design: Religion as a Resource for Technologists

Previous work has found a lack of research in HCI on religion, partly driven by misunderstandings of values and practices between religious and technical communities. To bridge this divide in an empirically rigorous way, we conducted an interview study with 48 religious people and/or experts from 11 faiths, and we document how religious people experience, understand, and imagine technologies. We show that religious stakeholders find non-neutral secular embeddings in technologies and the firms and people that design them, and how these manifest in unintended harms for religious and nonreligious users. Our findings reveal how users navigate technoreligious practices with religiously informed mental models and what they desire from technologies. Informed by this, we distill six design values -- wonder, humility, space, embodiedness, community, and eternity -- to guide technologists in considering and leveraging religion as an additional, valid sociocultural resource when designing for a holistic user. We further spell out directions for future research.

cs.CY↗