arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.13625

Kernel-based External Validity of Experimental Studies

Federico Matteucci·Gerhard Neumann
View original source ↗Read source document ↗Export metadata ↓

Abstract

The external validity of a study is the extent to which its findings generalize to unseen experimental conditions, such as datasets. We formalize the results of a study together with its research question, and define external validity as the probability that two independent and same-sized replications of the study produce results whose maximum mean discrepancy (MMD) is below a given tolerance. This approach, centered on the choice of a space of results and of an appropriate reproducing kernel for the MMD, is flexible enough to be deployed on multiple different types of results and research questions. Additionally, we establish an asymptotic power-law relation between the size of a study and the quantiles of the MMD, which we use to estimate the desired size of an externally valid study from a small-scale preliminary study. We deploy our framework to rankings with ties and discuss kernels suited to different research questions. Experiments on existing case studies and on a simulated newly designed study illustrate the applicability of the framework. We release our code as part of the genexpy Python module.

Explore related subjects

stat.MEcs.LGstat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Federico Matteucci, Gerhard Neumann. 2026-10-09. Kernel-based External Validity of Experimental Studies. https://arxiv.org/abs/2610.13625

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Moment-assisted Approach for Improving Subsampling-based MLE with Large-scale data

The maximum likelihood estimation is computationally demanding for large datasets, particularly when the likelihood function includes integrals. Subsampling can reduce the computational burden, but it often results in efficiency loss.This paper proposes a moment-assisted subsampling (MAS) method that can improve the estimation efficiency of existing subsampling-based maximum likelihood estimators.The motivation behind this approach stems from the fact that sample moments can be efficiently computed even if the sample size of the whole data set is huge.Through the generalized method of moments, the proposed method incorporates informative sample moments of the whole data. The MAS estimator can be computed rapidly and is asymptotically normal with a smaller asymptotic variance than the corresponding estimator without incorporating sample moments of the whole data. The asymptotic variance of the proposed estimator depends on the specific sample moments incorporated. We derive the optimal moment that minimizes the resulting asymptotic variance in terms of Loewner order. The proposed MAS estimator can achieve the same estimation efficiency as the whole data-based estimator when the optimal moment is incorporated. Numerical results demonstrate the promising performance of the proposed method in both estimation and computational efficiency compared with existing subsampling methods.

stat.ME↗
arXiv2026

Active Sequential Signal Detection with Asynchronous Decisions

The problem of quickly detecting signals among multiple sequentially observed data streams, subject to constraints on the familywise false-alarm and missed-detection probabilities, is considered under an active sampling setup where only one stream can be sampled at each time instant. Under this setup, it is in general impossible to minimize, even asymptotically, the expected decision time simultaneously for every stream. We therefore consider minimizing the expected time until either the kth signal declaration or termination simultaneously for every k. We first establish non-asymptotic lower bounds for these criteria. For k > |B|, where B denotes the true subset of signals, we show that these lower bounds are attained asymptotically as the error levels vanish by any policy that stops sampling and makes a decision for each stream according to a local Sequential Probability Ratio Test. For 1 <= k <= |B|, however, the performance depends critically on the sampling rule, and we propose one with a simple exploration mechanism that prevents switching away from signals prematurely because of early unfavorable observations. We show that, under a mild relative-rate condition on the two error levels, the resulting policy attains all the lower bounds simultaneously. Numerical studies are presented to compare the proposed policy with an oracle policy and a follow-the-leader policy.

stat.ME↗
arXiv2026

Typical Healthcare Pathways as a Basis for Admixture Modeling of Patient Trajectories

Background: Understanding whether patients follow similar or distinct patterns of care is important for characterizing clinical practice, identifying patient subgroups, and supporting quality improvement. However, routine healthcare trajectories are difficult to compare directly because patients may differ in their diagnostic workup, treatment sequencing, timing of clinical events, and documentation practices. Despite this variation, trajectories often contain recurring patterns at the cohort level. Methods: To address this challenge, we present a framework that explicitly separates cohort-level typical pathway identification from patient-level inference. At the cohort level, we derive an interpretable representation of care processes using a rule-based algorithm to identify typical healthcare pathways, resulting in a compact pathway graph. These pathways are then modeled as Markov chains and used as structured components in an admixture model, allowing each patient to be represented as a probabilistic mixture of typical pathways rather than being assigned to a single pathway component. The resulting admixture weights provide a compact representation of patient trajectories for subgroup characterization. We further assess the stability of the identified pathways and inferred admixture representations across multiple train-test splits. Results: Across train-test splits, the framework demonstrated consistent pathway structures and patient-level mixture patterns. Applied to routine care data from prostate cancer patients undergoing radical prostatectomy, the framework identified interpretable care patterns and supported the identification of patient subgroups with similar clinical event patterns. Conclusion: Overall, the proposed framework provides an interpretable and stable approach for summarizing treatment pathways and characterizing patient subgroups in real-world practice.

stat.ME↗