arXiv Science⌕ Search

arXiv · 2611.14526

Partial pooling beats synthetic respondents as estimators, not as covariates: a preregistered small-sample tournament

Abstract

Survey estimates for population segments, such as media use by sex and age, rest on few respondents per cell, and a growing literature offers synthetic respondents generated by large language models to fill the gap. That literature has not tested them against the small-area estimators a methodologist would otherwise use. We ran ten estimators, preregistered, on identical household-clustered splits of a Korean media panel across five calibration budgets and three holdout regimes. A multilevel partial-pooling model of the MRP family led every calibration-only estimator at every budget, matching at one hundred respondents what direct estimation reaches with roughly four times as many. On unobserved segments, information fixed as a per-segment prior was inert for synthetic panel and previous wave alike; attached as a bias correction it transferred, at the price of estimation noise. No registered synthetic advantage over MRP survived the paired decision rule. Entered instead as a covariate inside the multilevel model, an exploratory hybrid, the same panel improved on MRP on unobserved segments: the information survives where the registered constructions did not. Interval coverage stayed below nominal throughout and worsened with budget for partial pooling, contrary to prediction.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Howard Kim. 2026-09-30. Partial pooling beats synthetic respondents as estimators, not as covariates: a preregistered small-sample tournament. https://arxiv.science/abs/2611.14526

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Bayesian Framework for Latent Compliance Modeling in Cluster Randomized Trials with One-Sided Noncompliance

In pragmatic cluster randomized controlled trials (PCRCTs), healthcare providers are randomized, but intervention delivery and patient receipt may vary after assignment. In many PCRCTs, cluster-level implementation is measured using continuous implementation metrics, while individual compliance is recorded as a binary indicator. Standard complier average causal effect (CACE) estimands focus on individual-level compliance and do not account for heterogeneity in implementation across clusters. When intervention uptake is shaped by both cluster- and individual-level processes, it is of scientific interest to characterize how effects vary across these sources of compliance. We propose a Bayesian framework for PCRCTs with one-sided binary noncompliance at the individual level and one-sided partial compliance at the cluster level. The method uses a latent mixture model to summarize heterogeneity in cluster-level implementation based on baseline characteristics and treatment-arm implementation measures, and links these latent implementation strata to individual compliance and outcomes through a joint model. Because individual compliance and treatment-defined implementation measures are observed only under treatment assignment, the model imputes these quantities for control clusters and individuals assigned to control. The framework enables estimation of finite- and super-population intent-to-treat (ITT) and CACE estimands, both marginally and within latent implementation strata. We apply the method to the METRIcAL trial, a pragmatic cluster randomized study evaluating a personalized music intervention for nursing home residents with dementia. The analysis illustrates how accounting for implementation heterogeneity and individual compliance can provide insights beyond standard ITT analyses.

stat.AP↗

A Physically Informed Changepoint Score for Fault Prioritisation in District Heating

Poorly cooling district-heating installations raise return temperatures and pumped flow, increasing heat losses and pumping energy and hindering low-temperature operation. Faults often go unreported for weeks, so their onset is unknown and energy is wasted in the meantime. Utilities flag poorly cooling installations by a low supply-return temperature difference, but not when the problem began. Changepoint detectors such as the Bayesian Estimator of Abrupt Change, Seasonality, and Trend (BEAST) can date a change but report only its probability, so a large, fault-like change and a marginal one can look alike. We therefore develop a decision score that weighs each BEAST change in temperature difference and flow by its probability, size, deviation from recent operation and physical direction. Joint falls in temperature difference and rises in flow, the signature of insufficient cooling, rank first. Across 1,522 cases, 648 with a reported fault, 24 of the 25 highest-ranked cases were reported faults, and Average Precision was 0.772 against a prevalence of 0.426, likely conservative, as unreported cases may contain faults. Without using the report date, the strongest flow rise in reported-fault cases came a median of 11 days before the report (128 days in comparison cases). Combined with temperature-difference screening, the score raised Average Precision in season-matched cases from 0.486 to 0.646. After the detected change, flagged reported-fault installations returned water 16 K warmer and about half their circulated water delivered no heat. The score gives utilities a short list for inspection and a candidate onset date for each change.

stat.AP↗

Copula-Based Synthetic Data Generation: A Use-Inspired Perspective on Trade-offs

The generation of synthetic data offers a promising way to address challenges related to data scarcity and privacy concerns. While many approaches have been developed, tabular synthetic data generation methods often require complex models and extensive training data, while lacking interpretability. In this work, we investigate versatile probabilistic approaches using copulas, and leverage in particular the Gaussian Mixture Copula Model, to model the joint distribution of multivariate data, with a main focus on moderate-dimensional data sets. Taking a use-inspired perspective, we examine key trade-offs inherent to synthetic data evaluation, spanning fidelity, utility, and privacy, and demonstrate that copula-based methods achieve a sound balance across all three dimensions while remaining interpretable and providing an explicit probabilistic model. We illustrate and highlight the potential benefits of the method on a test case focusing on systematic missing values in a cardiovascular risk calculator and develop specific evaluation metrics for this application.

stat.AP↗