arXiv Science⌕ Search

arXiv subjects

Jia Wang Peh

Publications and source records attributed to Jia Wang Peh.

2 recordsLinked to original sources

Whose Values, Which Contexts, and What Signals? Grounding Cultural Value Alignment with Re3Culture

Cultural value alignment requires not only model optimization but also a well-specified target grounded in real-human evidence. Yet how such targets should be grounded in human data remains largely unexplored, raising three fundamental questions: (1) whose values should be represented? (2) where should cultural values be observed? and (3) what signals reveal underlying values? We investigate these questions through Re3Culture, a large-scale dataset linking (i) Representative communities: 700+ human participants across four cultures recruited through cultural-affiliation screening and demographic-aware sampling; (ii) Realistic scenario: culturally sensitive scenarios selected by community members, e.g., family and kinship; and (iii) Revealed value signals: 1,400+ value surveys, 20,000+ topic-grounded behavioral principles and explicit value priorities with responses for 6,000+ situated questions. Our analysis shows why these choices matter for defining the target, with three implications. First, estimating cultural values requires both community affiliation and broad internal coverage. Substituting other cultures or restricting subgroup coverage shifts value estimates. LLM simulations also disproportionately reflect particular subgroups. Second, values must be observed in concrete, community-guided situations. Abstract questionnaire profiles weakly recover situated priorities, and communities differ in which scenarios they identify as culturally value-revealing. Third, value priorities should be elicited explicitly, not learned from response preferences or bottom-line answers alone. Preferences conflate value agreement with other response-level qualities, and similar answers can be derived from different value priorities. By addressing these requirements, Re3Culture provides the first human-grounded resource that explicitly informs target definition for cultural value alignment.

cs.AI↗

CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia

Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains. Each simulated and evaluated episode produces a scored interaction where the assistant assists the user and infers cultural constraints from partial information. The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues. In our benchmark evaluation of 18 models, GPT-5 mini achieves the highest assistance quality. Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment. Performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-domain assistance and transfer out-of-domain to cultural MCQ and safety classification benchmarks. We release the harness, both splits, and judge prompts to support interactive evaluation of cultural competency.

cs.CL↗