arXiv ScienceSearch

arXiv · 2109.10141

Accommodating heterogeneous missing data patterns for prostate cancer risk prediction

Abstract

Objective: We compared six commonly used logistic regression methods for accommodating missing risk factor data from multiple heterogeneous cohorts, in which some cohorts do not collect some risk factors at all, and developed an online risk prediction tool that accommodates missing risk factors from the end-user. Study Design and Setting: Ten North American and European cohorts from the Prostate Biopsy Collaborative Group (PBCG) were used for fitting a risk prediction tool for clinically significant prostate cancer, defined as Gleason grade group greater or equal 2 on standard TRUS prostate biopsy. One large European PBCG cohort was withheld for external validation, where calibration-in-the-large (CIL), calibration curves, and area-underneath-the-receiver-operating characteristic curve (AUC) were evaluated. Ten-fold leave-one-cohort-internal validation further validated the optimal missing data approach. Results: Among 12,703 biopsies from 10 training cohorts, 3,597 (28%) had clinically significant prostate cancer, compared to 1,757 of 5,540 (32%) in the external validation cohort. In external validation, the available cases method that pooled individual patient data containing all risk factors input by an end-user had best CIL, under-predicting risks as percentages by 2.9% on average, and obtained an AUC of 75.7%. Imputation had the worst CIL (-13.3%). The available cases method was further validated as optimal in internal cross-validation and thus used for development of an online risk tool. For end-users of the risk tool, two risk factors were mandatory: serum prostate-specific antigen (PSA) and age, and ten were optional: digital rectal exam, prostate volume, prior negative biopsy, 5-alpha-reductase-inhibitor use, prior PSA screen, African ancestry, Hispanic ethnicity, first-degree prostate-, breast-, and second-degree prostate-cancer family history.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Matthias Neumair, Michael W. Kattan, Stephen J. Freedland, Alexander Haese, Lourdes Guerrios-Rivera, Amanda M. De Hoedt, Michael A. Liss, Robin J. Leach, Stephen A. Boorjian, Matthew R. Cooperberg, Cedric Poyet, Karim Saba, Kathleen Herkommer, Valentin H. Meissner, Andrew J. Vickers, Donna P. Ankerst. 2021-09-21. Accommodating heterogeneous missing data patterns for prostate cancer risk prediction. https://doi.org/10.1186/s12874-022-01674-x

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Uncertainty-Aware Missing-Data Multimodal Latent for Fetal-Growth Analysis

Objective: Routine third-trimester examination yields fetal biometry, maternal, Doppler and fetal-cardiac measurements, acquired at clinical discretion and therefore often incomplete. We show that these four measurement blocks are close to mutually uninformative, and that this single property determines what a representation of them can impute, what it can audit, and what fetal size alone cannot indicate. Methods: A linear-Gaussian factor model (K = 8 by parallel analysis, VARIMAX-rotated) was fitted to 25 measurements in four blocks from 977 fetuses (169 SGA, 61 severe; 77 LGA). Posterior precision sums contributions from observed measurements only, so missing values are marginalized rather than imputed. Data quality was screened using the standardized residual between each measurement and its reconstruction. Results: Predicting any one block from the other three gives an out-of-fold R2 of 0.023. The representation is a continuous growth spectrum with no cluster structure (Hartigan dip p = 0.99, gap statistic k = 1, three-cluster silhouette 0.07) ordering fetuses by birthweight centile (Spearman rho = 0.55). Among 169 SGA fetuses the haemodynamic redistribution axis separated the 25 adverse outcomes (AUC 0.70, 0.585-0.808) where measured size did not (0.60, 0.451-0.738). With Doppler censored, the marginalized interval covered held-out measurements in 97% and 93% of cases against nominal 95% and 90%. The reconstruction residual flagged 38 of 977 records, 36 confirmed transcription errors in the registry. Conclusion: Marginalizing missing measurements yields a representation whose uncertainty reflects the available data, and whose reconstruction residual doubles as a data-quality screen. Because the blocks are nearly independent, confirmed flags are within-block errors, and a synthetic benchmark gives the coupling needed before cross-block detection becomes available.

stat.AP

Auditing the Global Carbon Budget: Exploring the 2024--2025 Vintage Shift

The Global Carbon Budget (GCB), the community reference dataset for the carbon cycle, is reissued annually. The 2025 release introduces several adjustments to the published series that we compare with prior releases starting in 2017. On a common 1959-2016 sample, the mean of the GCB budget imbalance jumps from within +/-0.17 GtC/yr of zero for every vintage 2017-2024 to 0.61 GtC/yr in 2025, the only vintage whose 95% confidence interval for the imbalance mean excludes zero. The size of the imbalance changes much less: its mean absolute value rises from 0.61 to 0.76 GtC/yr. It is the mean, the quantity the budget identity constrains, that moves. We document and explore this shift in two ways. First, we conduct a model-free analysis, where we attribute the shift to a new adjustment that places the published land sink 0.40 GtC/yr below its ensemble mean (the average of the underlying models), a smaller adjustment in the ocean sink in the opposite direction, and a change in the composition of the bookkeeping ensemble. Second, we consider a dynamic statistical GCB model augmented with climate covariates. Its parameters are estimated for every GCB vintage 2017-2025. The coefficients of atmospheric concentrations in the sink equations shift in opposite directions on the 2025 issue, mirroring the model-free findings. A constant in the budget equation, statistically unnecessary in every vintage from 2017-2024, is required in 2025 and is estimated at -0.59 (0.09) GtC/yr. There is a persistent drifting imbalance across the entire sample in the budget equation. Each of the three adjustments is documented in the 2025 release and rests on evidence about the component it corrects. Their joint effect is a budget that closes over the last ten years and carries a mean imbalance of 0.61 GtC/yr over the full record. We argue that this cost to the full sample outweighs the gain on the last ten years.

stat.AP

Bridging Network Psychometrics and Artificial Intelligence: An Ising-Potts Model with LLM-Derived Weights

The Potts model extends the Ising model to multinomial data. We introduce a Rater Ising-Potts model that uses agreement indicators between pairs of ratings and category labels, with weights derived from LLM embeddings. The model does not presuppose ordered category thresholds or equidistant scoring; instead, it focuses on pairwise agreement among ratings and assigns category-specific positive weights, making it suited for multi-category scoring reliability. We evaluate the model on three constructed-response datasets spanning a corpus of K=14,466 short answers on a three-level rubric and two AERA essay prompts of roughly 1,200-1,400 responses on four-point rubrics. We compare three strategies for sharpening the similarity signal: top-K pruning, min-max normalization with a power transformation, and ColBERT late-interaction similarities. Top-K pruning, which replaces the dense similarity graph with a sparse local network of strongest semantic neighbors, consistently yields the highest accuracy and Cohen's kappa, and the selected neighborhoods are always a small fraction of the corpus. Power tuning consistently ranks second, while ColBERT is competitive on longer essay prompts and adds little on short answers. Across all settings, most misclassifications occur between adjacent score levels, confirming that the model preserves the ordinal structure of scoring rubrics without imposing rigid assumptions. These findings suggest that LLM-derived similarities, combined with a parsimonious Potts formulation and a sparse local graph, offer a robust and interpretable framework for reliability auditing in educational assessment. We discuss extensions to multiple raters and hierarchical rating designs.

stat.AP