arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.24990

The World as Viewed by 136 Countries: An Open, Curated, Annotated, and Multilingual Dataset of 118 Million State Media Articles

Bryan Li·Samar Haider·Grace Kim·Fiona Luo·Jinqi Luo·Yuan Yuan·Duncan Watts·René Vidal·Chris Callison-Burch
View original source ↗Read source document ↗Export metadata ↓

Abstract

State media outlets often project their governments' narratives to target both domestic and foreign audiences. Because this content is plentiful and free to access, it appears widely in the web corpora that large language models (LLMs) are trained on and that retrieval systems draw from. However, the prevalence of state media in these corpora and its potential influence on LLM outputs remain largely unexplored. In this paper, we introduce WorldStateMedia, an open dataset of 118 million state media articles from 136 countries in 114 languages. We extract these articles from existing pretraining corpora and annotate each outlet with level of state control from State Media Monitor. Using WorldStateMedia to audit these corpora, we find that, in English web crawls, the share of state media pages from editorially controlled (vs. independent) outlets rose from 39% in 2013 to 93% in 2025. In several low-resource languages, state media make up a substantial share of all web data, such as 17.8% for Uyghur and 7.8% for Belarusian. To characterize the differing narratives of news outlets, we compare Russian and Ukrainian state media in English and in their native languages, revealing asymmetric topical focus and opposing framings of contested issues such as Crimea. Furthermore, in a retrieval-augmented generation (RAG) experiment on territorial disputes, retrieving a country's state media shifts model answers toward that country's claims, most strongly for English-language (foreign-targeted) media. We release WorldStateMedia to support multilingual, cross-national insights on state media at https://huggingface.co/collections/manestay/worldstatemedia

Explore related subjects

cs.CLcs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Bryan Li, Samar Haider, Grace Kim, Fiona Luo, Jinqi Luo, Yuan Yuan, Duncan Watts, René Vidal, Chris Callison-Burch. 2026-10-18. The World as Viewed by 136 Countries: An Open, Curated, Annotated, and Multilingual Dataset of 118 Million State Media Articles. https://arxiv.org/abs/2610.24990

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Post-hoc Disentanglement of Textual and Acoustic Features in Pre-trained Speech Encoders

Pre-trained speech models build entangled internal representations, which capture a variety of features (e.g., pitch, loudness, syntax, or semantics of an utterance) in a distributed encoding. This entanglement makes it difficult to track how such representations rely on textual versus acoustic information when used in downstream applications, limiting their interpretability and transparency. In this paper, we explore an approach to post-hoc disentanglement that builds on the Information Bottleneck principle. We take representations from pre-trained speech encoders as input, and separate them into two distinct components in a cascaded fashion: first we learn a vector encoding content (i.e., what can be transcribed as text), and nothing else; second, we learn a vector encoding all complementary acoustic features relevant to a target task. We apply and evaluate our proposed setup to emotion recognition and speaker identification target tasks, quantifying the relative contribution of textual and acoustic features at each model layer. Finally, we use the approach for disentangled feature attribution as it allows us to identify the most salient speech frames from both the textual and acoustic perspectives.

cs.CL↗
arXiv2026

The Super Emotion Dataset

Despite the wide-scale usage and development of emotion classification datasets in NLP, the field lacks a standardized, large-scale resource that follows a psychologically grounded taxonomy. Existing datasets either use inconsistent emotion categories, suffer from limited sample size, or focus on specific domains. The SuperEmotion dataset addresses this gap by harmonizing diverse text sources into a unified framework based on Shaver's empirically validated emotion taxonomy, enabling more consistent cross-domain emotion recognition research. This report describes version 2 of the dataset. We also report baseline results for a fine-tuned ModernBERT encoder.

cs.CL↗
arXiv2026

AutoMind: Adaptive Knowledgeable Agent for Automated Data Science

Large Language Model (LLM) agents have shown great potential in addressing real-world data science problems. LLM-driven data science agents promise to automate the entire machine learning pipeline, yet their real-world effectiveness remains limited. Existing frameworks depend on rigid, pre-defined workflows and inflexible coding strategies; consequently, they excel only on relatively simple, classical problems and fail to capture the empirical expertise that human practitioners bring to complex, innovative tasks. In this work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework that overcomes these deficiencies through three key advances: (1) a curated expert knowledge base that grounds the agent in domain expert knowledge, (2) an agentic knowledgeable tree search algorithm that strategically explores possible solutions, and (3) a self-adaptive coding strategy that dynamically tailors code generation to task complexity. Evaluations on two automated data science benchmarks demonstrate that AutoMind delivers superior performance versus state-of-the-art baselines. Additional analyses confirm favorable effectiveness, efficiency, and qualitative solution quality, highlighting AutoMind as an efficient and robust step toward fully automated data science. Code is at https://github.com/zjunlp/AutoMind.

cs.CL↗