arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2410.21195

Belief in the Machine: Investigating Epistemological Blind Spots of Language Models

Mirac Suzgun·Tayfun Gur·Federico Bianchi·Daniel E. Ho·Thomas Icard·Dan Jurafsky·James Zou
View original source ↗Read source document ↗Export metadata ↓

Abstract

As language models (LMs) become integral to fields like healthcare, law, and journalism, their ability to differentiate between fact, belief, and knowledge is essential for reliable decision-making. Failure to grasp these distinctions can lead to significant consequences in areas such as medical diagnosis, legal judgments, and dissemination of fake news. Despite this, current literature has largely focused on more complex issues such as theory of mind, overlooking more fundamental epistemic challenges. This study systematically evaluates the epistemic reasoning capabilities of modern LMs, including GPT-4, Claude-3, and Llama-3, using a new dataset, KaBLE, consisting of 13,000 questions across 13 tasks. Our results reveal key limitations. First, while LMs achieve 86% accuracy on factual scenarios, their performance drops significantly with false scenarios, particularly in belief-related tasks. Second, LMs struggle with recognizing and affirming personal beliefs, especially when those beliefs contradict factual data, which raises concerns for applications in healthcare and counseling, where engaging with a person's beliefs is critical. Third, we identify a salient bias in how LMs process first-person versus third-person beliefs, performing better on third-person tasks (80.7%) compared to first-person tasks (54.4%). Fourth, LMs lack a robust understanding of the factive nature of knowledge, namely, that knowledge inherently requires truth. Fifth, LMs rely on linguistic cues for fact-checking and sometimes bypass the deeper reasoning. These findings highlight significant concerns about current LMs' ability to reason about truth, belief, and knowledge while emphasizing the need for advancements in these areas before broad deployment in critical sectors.

Explore related subjects

cs.CLcs.AIcs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Mirac Suzgun, Tayfun Gur, Federico Bianchi, Daniel E. Ho, Thomas Icard, Dan Jurafsky, James Zou. 2024-10-28. Belief in the Machine: Investigating Epistemological Blind Spots of Language Models. https://arxiv.org/abs/2410.21195

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Controlled Reevaluation of Coreference Resolution Models

All state-of-the-art coreference resolution (CR) models involve finetuning a pretrained language model. Whether the superior performance of one CR model over another is due to the choice of language model or other factors, such as the task-specific architecture, is difficult or impossible to determine due to lack of a standardized experimental setup. To resolve this ambiguity, we systematically evaluate five CR models and control for certain design decisions including the pretrained language model used by each. When controlling for language model size, encoder-based CR models outperform more recent decoder-based models in terms of both accuracy and inference speed. Surprisingly, among encoder-based CR models, more recent models are not always more accurate, and the oldest CR model that we test generalizes the best to out-of-domain textual genres. We conclude that controlling for the choice of language model reduces most, but not all, of the increase in F1 score reported in the past five years.

cs.CL↗
arXiv2026

Balancing Global Quality and Pronoun-Specific Feedback for Context-Aware Machine Translation

Context-aware machine translation can expose the evidence needed for pronoun choice, but standard fine-tuning does not explicitly prioritize these sparse discourse-sensitive decisions. We study ProNMT, a reward-guided iterative self-training method that combines sentence-level quality estimation with a signed confidence signal at generated pronoun positions. For each current sentence and its preceding source context, ProNMT samples candidate translations, scores them using reference-free quality estimation together with a reference-derived pronoun label, and fine-tunes on the highest-scoring candidate. On filtered English--German Europarl and English--French News Commentary data, ProNMT improves over context-aware supervised fine-tuning on BLEU and COMET. Ablations show that pronoun-only feedback can severely degrade sentence-level translation quality on these pronoun-focused data, while hard binary feedback underperforms confidence-weighted feedback. These results indicate that targeted linguistic feedback is most useful when combined with both a global quality signal and the context relevant to the targeted decision. We make the code publicly available at https://github.com/Harshit2807161/ProNMT.

cs.CL↗
arXiv2026

All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue

Task-oriented dialogue (ToD) systems aim to help users accomplish goals through natural language interaction. Beyond task success, effective ToD systems must also maintain positive emotional interaction and accurately convey information in inherently noisy and ambiguous conversational environments. Recent advances in large language models (LLMs) have substantially improved conversational fluency and contextual understanding. However, although emotion has been incorporated into existing ToD systems, its integration is typically confined to limited roles, restricting its influence on overall dialogue behaviour and user experience. To address this, we propose a novel end-to-end framework that combines fully lexicalised representations with an LLM backbone for improved semantic accuracy and employs online reinforcement learning with short-term emotional and long-term task-success rewards for joint optimisation of emotional interaction and task success. We further develop a challenging and realistic simulation environment that enables systematic evaluation across diverse ToD set-ups. Experiments show that incorporating coarse affective signals improves task success and sentiment rating in aggregate. Furthermore, integrating emotional interaction throughout the dialogue system using our proposed framework results in more pronounced improvements in both task success and user emotional experience.

cs.CL↗