arXiv Science⌕ Search

arXiv · 2611.12344

The Fragility Of Synthetic Populations Over Time: An Evaluation Of Forecasting Strategies For Agent-Based Models

Abstract

Agent-based simulation models are widely used to assess the future mobility projects. However, their effectiveness hinges on the accuracy of synthetic populations, which are often generated using data from a single reference year. This study explores the temporal validity of these populations by evaluating their performance over multi-year datasets in a simulation-like environment. We focus on multi-dimensional agents, ensuring that all attributes critical to the simulation are comprehensively represented. To assess the quality of synthetic populations, we compare three generation methods: direct use of original training data, Bayesian Network sampling, and diffusion model. We then evaluate four agent-forecasting approaches: static projection (both idealistic and realistic), resampling, and dynamic projection. While dynamic projection holds potential, it requires complex longitudinal data to model the evolution of agent attributes. To address this, we introduce a heuristic model based on aggregated statistics, though its limitation lies in ignoring attribute correlations, which reduces its accuracy. Our results demonstrate that synthetic populations lose relevance over time, emphasizing the necessity of effective agent-forecasting. Among the methods tested, idealistic static projection yields the best performance but requires data that is difficult to obtain. The other methods show limited improvements, highlighting the ongoing challenges in accurately forecasting agent attributes.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Vianey Darsel, Etienne C{ô}me, Jeppe Rich, Francisco Camara Pereira, Latifa Oukhellou. 2026-11-09. The Fragility Of Synthetic Populations Over Time: An Evaluation Of Forecasting Strategies For Agent-Based Models. https://arxiv.science/abs/2611.12344

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Bayesian Framework for Latent Compliance Modeling in Cluster Randomized Trials with One-Sided Noncompliance

In pragmatic cluster randomized controlled trials (PCRCTs), healthcare providers are randomized, but intervention delivery and patient receipt may vary after assignment. In many PCRCTs, cluster-level implementation is measured using continuous implementation metrics, while individual compliance is recorded as a binary indicator. Standard complier average causal effect (CACE) estimands focus on individual-level compliance and do not account for heterogeneity in implementation across clusters. When intervention uptake is shaped by both cluster- and individual-level processes, it is of scientific interest to characterize how effects vary across these sources of compliance. We propose a Bayesian framework for PCRCTs with one-sided binary noncompliance at the individual level and one-sided partial compliance at the cluster level. The method uses a latent mixture model to summarize heterogeneity in cluster-level implementation based on baseline characteristics and treatment-arm implementation measures, and links these latent implementation strata to individual compliance and outcomes through a joint model. Because individual compliance and treatment-defined implementation measures are observed only under treatment assignment, the model imputes these quantities for control clusters and individuals assigned to control. The framework enables estimation of finite- and super-population intent-to-treat (ITT) and CACE estimands, both marginally and within latent implementation strata. We apply the method to the METRIcAL trial, a pragmatic cluster randomized study evaluating a personalized music intervention for nursing home residents with dementia. The analysis illustrates how accounting for implementation heterogeneity and individual compliance can provide insights beyond standard ITT analyses.

stat.AP↗

A Physically Informed Changepoint Score for Fault Prioritisation in District Heating

Poorly cooling district-heating installations raise return temperatures and pumped flow, increasing heat losses and pumping energy and hindering low-temperature operation. Faults often go unreported for weeks, so their onset is unknown and energy is wasted in the meantime. Utilities flag poorly cooling installations by a low supply-return temperature difference, but not when the problem began. Changepoint detectors such as the Bayesian Estimator of Abrupt Change, Seasonality, and Trend (BEAST) can date a change but report only its probability, so a large, fault-like change and a marginal one can look alike. We therefore develop a decision score that weighs each BEAST change in temperature difference and flow by its probability, size, deviation from recent operation and physical direction. Joint falls in temperature difference and rises in flow, the signature of insufficient cooling, rank first. Across 1,522 cases, 648 with a reported fault, 24 of the 25 highest-ranked cases were reported faults, and Average Precision was 0.772 against a prevalence of 0.426, likely conservative, as unreported cases may contain faults. Without using the report date, the strongest flow rise in reported-fault cases came a median of 11 days before the report (128 days in comparison cases). Combined with temperature-difference screening, the score raised Average Precision in season-matched cases from 0.486 to 0.646. After the detected change, flagged reported-fault installations returned water 16 K warmer and about half their circulated water delivered no heat. The score gives utilities a short list for inspection and a candidate onset date for each change.

stat.AP↗

Copula-Based Synthetic Data Generation: A Use-Inspired Perspective on Trade-offs

The generation of synthetic data offers a promising way to address challenges related to data scarcity and privacy concerns. While many approaches have been developed, tabular synthetic data generation methods often require complex models and extensive training data, while lacking interpretability. In this work, we investigate versatile probabilistic approaches using copulas, and leverage in particular the Gaussian Mixture Copula Model, to model the joint distribution of multivariate data, with a main focus on moderate-dimensional data sets. Taking a use-inspired perspective, we examine key trade-offs inherent to synthetic data evaluation, spanning fidelity, utility, and privacy, and demonstrate that copula-based methods achieve a sound balance across all three dimensions while remaining interpretable and providing an explicit probabilistic model. We illustrate and highlight the potential benefits of the method on a test case focusing on systematic missing values in a cardiovascular risk calculator and develop specific evaluation metrics for this application.

stat.AP↗