arXiv ScienceSearch

arXiv subjects

Holger Dette

Publications and source records attributed to Holger Dette.

At least 19 recordsLinked to original sources

Eigenvalue Statistics for Change-Point Detection in Covariance Matrices

Testing for change points in sequences of covariance matrices is an important and equally challenging problem in statistical methodology with applications in various fields. Motivated by the observation that even in cases where the ratio between dimension and sample size is as small as $0.05$, tests based on a fixed-dimension asymptotics do not keep their preassigned level, we propose to derive critical values of test statistics using an asymptotic regime where the dimension diverges at the same rate as the sample size. This paper introduces a novel and well-founded statistical methodology for detecting change points in a sequence of moderately dimensional covariance matrices. Our approach utilizes a min-type statistic based on a sequential process of likelihood ratio statistics. This is used to construct a test for the hypothesis of the existence of a change point with a corresponding estimator for its location. We provide theoretical guarantees by thoroughly analyzing the asymptotic properties of the sequential process of likelihood ratio statistics. In particular, we prove weak convergence towards a Gaussian process under the null hypothesis of no change. To identify the challenging dependency structure between consecutive test statistics, we employ tools from random matrix theory and stochastic processes.

math.ST

Testing similarity of competing risks models by comparing transition probabilities

Assessing whether patient populations exhibit comparable event dynamics is important for evaluating treatment equivalence, pooling cohorts and comparing clinical pathways. Existing similarity tests for competing risks models measure distances between transition intensities, which describe instantaneous event rates. In biomedical applications, similarity may be more naturally formulated through transition probabilities, which quantify cumulative event risks over a clinically relevant horizon. Assuming constant cause-specific transition intensities, we develop a framework for testing similarity based on a maximum-type distance between vectors of transition-probability functions. We propose a constrained parametric bootstrap test and establish asymptotic level control and consistency under administrative and independent exponential random right censoring. The constant-intensity formulation is motivated by small-data settings in which few events are observed and nonparametric estimators may be unstable. Simulations across sample sizes, censoring mechanisms and degrees of dissimilarity show that the proposed test can attain larger finite-sample rejection probabilities than an intensity-based benchmark under comparable alternatives. An application to routine prostate cancer data illustrates how the procedure identifies the smallest examined margin for which similarity of 90-day readmission-probability functions can be established under the fitted model. The method provides an interpretable and practically implementable basis for similarity assessment in parametric competing risks models.

stat.ME

Self-normalization for Spectral Density Integrals

Integrals of spectral densities are frequently used to summarize spectral characteristics of linear processes. This work studies self-normalization for estimators of such integrals based on sequential periodograms and establishes weak convergence of the corresponding processes. For linear functionals of the spectral density, self-normalization yields pivotal limiting distributions that are free of unknown spectral quantities. For non-linear functionals, however, additional components with distinct covariance structures may arise in the limiting process. We demonstrate this phenomenon for the integrated squared spectral density.

math.ST

Limiting properties of monotone rearrangements of estimators when the truth is flat

Monotone rearrangements provide a simple way to enforce shape constraints of an estimator, but existing distributional theory does not cover flat regions, where the target induces no local ordering. We study rearranged estimators in two canonical flat settings. First, for a histogram estimator of the uniform density, we establish functional weak convergence of its non-decreasing rearrangement at the parametric rate on compact subsets of the interval $(0,1)$ after an additional deterministic centering. This result is strikingly different from what is known for strictly monotone densities. Second, we consider two rearranged estimators of rearranged copulas under independence, based on empirical-copula increments and on a checkerboard approximation. After appropriate centering and rescaling, both estimators converge weakly on $[0,1]^2$ to an integrated Gaussian process which was not known before. We further use these results to prove asymptotic normality for a broad class of rearranged copula-based dependence measures, which were recently discussed in Strothmann et al. (2024).

math.ST

Safe and Sharp Honest Inference for Nonparametric Estimation via Empirical Bernstein Calibration

Constructing honest confidence intervals often depends on reliable bias control and accurate calibration based on asymptotic normality, including standard-normal and folded-normal calibration. Substantial progress has been made in correcting or controlling smoothing bias, including robust bias correction and bias-aware inference. We first show that, even after smoothing bias has been well corrected or controlled, asymptotic-normality-based calibration may still be a binding source of finite-sample undercoverage. Thus, the resulting intervals may struggle to achieve the minimax shrinkage rate and uniformly small undercoverage error simultaneously. Instead of using distributional approximation, we calibrate the radius directly by combining an empirical Bernstein bound, a data-driven variance proxy, Lepski-type bandwidth selection, and a bias-aware fixed-length-radius criterion. The formal theory covers nonparametric regression and density estimation, with regression results ranging from local-polynomial to sieve estimators. The resulting empirical Bernstein confidence intervals are safe and sharp. Uniformly over functions with $S$-th order local smoothness, both one-sided and two-sided intervals attain nominal coverage up to $o(n^{-2S/(2S+1)})$, or exponential remainders under bounded or sub-Gaussian conditions, while their widths shrink at the minimax rate $n^{-S/(2S+1)}$ (or up to a $\sqrt{\log n}$-level factor). The calibration principle is modular and can also be combined with other existing bias-control strategies, like robust bias correction. Thus, the contribution of this paper is not a bias-control device but a new angle of calibration. Compared with asymptotic-normality-based calibration, empirical Bernstein calibration safely and conveniently converts the specified smoothness into both coverage accuracy and interval-length efficiency. Simulations support the theory.

math.ST

A spectral based coefficient of determination for the fit of an MA(q) model

We develop a spectral based coefficient of determination to measure how well the spectral density of a stationary linear process is represented by the class of MA($q$) models. Using periodogram-based estimators, we establish asymptotic normality, derive tests for the MA($q$) hypothesis, and construct procedures for determining the smallest order $q$ achieving a prescribed approximation quality.

math.ST

Trustworthy Decisions in Reliability Set Estimation under Insufficient Model Information

Reliability set estimation identifies input regions where a response probability exceeds a target level, bridging estimation and safety-critical decisions. Practitioners typically start with a working model, an imperfect approximation of the true response surface. Relying on this imperfect model may incur decision risk, potentially certifying unsafe regions as safe. We develop a unified framework that turns such a working model into a trustworthy decision rule. First, a modeling-then-calibration procedure decouples estimation from decision. Since the true set is unobservable, we introduce an asymmetric, observable surrogate loss and use a separate calibration set to select a bias-correcting threshold, reducing decision risk and achieving $O_P(1/n)$ volume convergence. Second, we leverage conformal risk control with the surrogate loss to control false inclusion risk, which is the most safety-critical error, at a pre-specified level regardless of working model quality. Together, these calibration procedures show that a separate calibration set is necessary for risk control. Third, an adaptive design concentrates observations on the reliability set and its boundary, improving model quality where errors most affect decisions while controlling budget elsewhere. Numerical studies show not only more accurate set estimates but also calibrated finite-sample risk control that classical plug-in methods lack.

math.ST

Testing for subgroup treatment effect consistency in the Cox model

An overall treatment effect in a clinical trial may inadequately represent particular patient subgroups, creating uncertainty about whether a population-level efficacy conclusion can legitimately be transferred to them. Conventional interaction tests only investigate whether subgroup-specific treatment effects are exactly equal or not, but cannot detect whether the differences are small enough to be clinically negligible. We develop a formal framework for assessing subgroup treatment effect consistency for time-to-event outcomes within the Cox proportional hazards model. Consistency is formulated as an equivalence problem based on the (weighted) treatment-by-subgroup interaction coefficient. We consider detecting consistency between two complementary subgroups and consistency of subgroup-specific treatment effects with the overall treatment effect. For each setting, we develop a conventional two one-sided tests procedure (TOST) and a new test motivated by optimal equivalence testing for normally distributed parameters. We prove the asymptotic validity of all procedures and show empirically that the new tests are more powerful than their TOST counterparts. Finally, we apply the new methodology to a case study motivated by the CANTOS cardiovascular outcomes trial.

stat.ME

Testing for correct model specification in copula regression models

We propose a goodness-of-fit test for semiparametric copula regression models. Such models express the regression function in terms of marginal distribution functions and copula densities and therefore provide a flexible way to avoid fully nonparametric estimation in high-dimensional regression problems. Their performance, however, depends crucially on the specification of the parametric copula family. Instead of testing the copula model itself, we assess misspecification directly at the level of the induced regression function. To this end, we introduce a weighted $L^2$-distance between the true regression function and its best approximation within the postulated copula regression model. A kernel-based estimator of this distance is proposed and shown to be consistent and asymptotically normal under both the null hypothesis of correct specification and fixed alternatives. We derive a classical specification test and, using a self-normalized sequential statistic, construct pivotal confidence intervals and tests for relevant deviations from the model. Finite-sample simulations demonstrate accurate level approximation and good power properties of the proposed procedures.

math.ST

A portmanteau test for multivariate non-stationary functional time series with an increasing number of lags

Multivariate locally stationary functional time series provide a flexible framework for modeling functional data exhibiting both temporal and spatial dependencies while allowing for a time-varying data generating mechanism. In this paper, we introduce a portmanteau-type test for assessing white noise assumptions tailored for multivariate locally stationary functional time series without dimension reduction. A simple bootstrap procedure is proposed to implement the test, because it is not clear if the limiting distribution of the test statistic exists. Our approach is based on a Gaussian approximation result for a degenerate $U$-statistic of second-order functional time series involving an increasing number of lags, which is of independent interest. Through theoretical analysis, simulation studies, and real data analysis of energy consumptions, we demonstrate the efficacy and adaptability of the proposed method in detecting departures from white noise assumptions in multivariate locally stationary functional time series. Finally, the R package corresponding to our method can be downloaded from https://github.com/Lujia-Bai/nftsport.

stat.ME

Testing for Single-Population Ancestry in the Admixture Model

The Admixture Model describes genetic marker data by representing each individual's genome as a mixture of contributions from $K$ ancestral populations, with the individual admixture vector summarizing the corresponding ancestry proportions. In population and forensic genetics, a key question is whether an individual's genome supports a predominantly single-ancestry interpretation or whether an admixed interpretation is more appropriate. We propose a statistical test for single-population ancestry in the supervised Admixture Model, where ancestral allele frequencies are treated as known. The test assesses whether the largest admixture component exceeds a practitioner-chosen dominance threshold, giving precise meaning to the notion of a sufficiently strong single-population contribution. To calibrate the test, we develop a constrained parametric bootstrap procedure that generates data under a null-constrained maximum likelihood estimator, accounting for the constrained hypothesis structure, the marker-wise heterogeneity and small sample sizes. Under standard regularity conditions, we prove that the proposed test has asymptotic level $α$ and is consistent, ensuring control of false single-ancestry declarations while reliably detecting dominant ancestry components. Simulation studies demonstrate good finite-sample performance across different numbers of ancestral populations, marker-panel sizes, dominance thresholds, and allele-frequency distributions. We further illustrate the practical utility of the method using data from the 1000 Genomes Project. The proposed framework delivers interpretable, threshold-based ancestry assessment with rigorous error control, and extends constrained bootstrap methodology to the independent but non-identically distributed setting of genetic marker data.

stat.ME

Pivotal inference for linear predictions in stationary processes

In this paper we develop pivotal inference for the final (FPE) and relative final prediction error (RFPE) of linear forecasts in stationary processes. Our approach is based on a self-normalizing technique and avoids the estimation of the asymptotic variances of the empirical autocovariances. We provide pivotal confidence intervals for the (R)FPE, develop estimates for the minimal order of a linear prediction that is required to obtain a prespecified forecasting accuracy and also propose (pivotal) statistical tests for the hypotheses that the (R)FPE exceeds a given threshold. Additionally, we provide pivotal uncertainty quantification for the commonly used coefficient of determination $R^2$ obtained from a linear prediction based on the past $p \geq 1$ observations and develop new (pivotal) inference tools for the partial autocorrelation, which do not require the assumption of an autoregressive process.

math.ST

Validating spatial-temporal separability for stationary processes

A crucial assumption to reduce computational complexity in spatial-temporal data analysis is separability, which factors the covariance structure into a purely spatial and a purely temporal component. In this paper, we develop statistical inference tools for validating this assumption for a second-order stationary process under both domain-expanding-infill asymptotics and domain-expanding asymptotics. In contrast to previous work on this subject, the methodology neither requires the assumption of normally distributed data, nor uses spectral methods. Our approach is based on nonparametric estimates of measures for the deviation between the covariance matrix and separable approximations, which vanish if and only if the assumption of separability is satisfied. We derive the asymptotic distributions of appropriate estimators for these measures with non-standard limiting distributions and use these results to develop inference tools for validating the assumption of separability. More specifically, we derive confidence intervals for the deviation measures, tests for the hypothesis of exact separability, and for the hypothesis that the deviation from separability is smaller than a prespecified threshold.

math.ST

Differentially private testing for relevant dependencies in high dimensions

We investigate the problem of detecting dependencies between the components of a high-dimensional vector. Our approach advances the existing literature in two important respects. First, we consider the problem under privacy constraints. Second, instead of testing whether the coordinates are pairwise independent, we are interested in determining whether certain pairwise associations between the components (such as all pairwise Kendall's $τ$ coefficients) do not exceed a given threshold in absolute value. Considering hypotheses of this form is motivated by the observation that in the high-dimensional regime, it is rare and perhaps impossible to have a null hypothesis that can be modeled exactly by assuming that all pairwise associations are precisely equal to zero. The formulation of the null hypothesis as a composite hypothesis makes the problem of constructing tests already non-standard in the non-private setting. Additionally, under privacy constraints, state of the art procedures rely on permutation approaches that are rendered invalid under a composite null. We propose a novel bootstrap based methodology that is especially powerful in sparse settings, develop theoretical guarantees under mild assumptions and show that the proposed method enjoys good finite sample properties even in the high privacy regime. Additionally, we present applications in medical data that showcase the applicability of our methodology.

math.ST

Computationally tractable nonparametric bootstrap of high-dimensional sample covariance matrices

We introduce a new ``$(m,mp/n)$ out of $(n,p)$'' sampling-with-replace\-ment bootstrap for eigenvalue statistics of high-dimensional sample covariance matrices based on $n$ independent $p$-dimensional random vectors. As it only uses $q=\lfloor mp/n\rfloor $ coordinates of the observations in a subsample of size $m \ll n $ from the original data, it is computationally tractable for large scale data. In the high-dimensional scenario $p/n\rightarrow c\in (0,\infty)$, this fully nonparametric bootstrap is shown to consistently reproduce the empirical spectral measure if $m/n\rightarrow 0$. If $m^2/n\rightarrow 0$, it approximates correctly the distribution of linear spectral statistics. The crucial component is a suitably defined Representative Subpopulation Condition which is shown to be verified in a large variety of situations. Our proofs are conducted under minimal moment requirements and incorporate delicate results on non-centered quadratic forms, combinatorial trace moments estimates as well as a conditional bootstrap martingale CLT which may be of independent interest.

math.ST

Kernel Estimation Of Chatterjee's Dependence Coefficient

Dette, Siburg, and Stoimenov (2013) introduced a copula-based measure of dependence, which implies independence if it vanishes and is equal to 1 if one variable is a measurable function of the other. For continuous distributions, the dependence measure also appears as stochastic limit of Chatterjee's rank correlation (Chatterjee, 2021). They proved asymptotic normality of a corresponding kernel estimator with a parametric rate of convergence. In recent work Shi, Drton, and Han (2022) revealed empirically and theoretically that under independence the asymptotic variance degenerates. In this note, we derive the correct asymptotic distribution of the kernel estimator under the null hypothesis of independence. We show that after a suitable centering and rescaling at a rate larger than $\sqrt{n}$ (where $n$ is the sample size), the estimator is asymptotically normal. The analysis relies on a refined central limit theorem for double-indexed linear permutation statistics and accounts for boundary effects that are asymptotically non-negligible. As a consequence, we obtain a valid basis for independence testing without relying on permutations and argue that tests based on the kernel estimator detect local alternatives converging to the null at a faster rate than those detectable by Chatterjee's rank correlation.

math.ST

Measuring deviations from spherical symmetry

Most of the work on checking spherical symmetry assumptions on the distribution of the $p$-dimensional random vector $Y$ has its focus on statistical tests for the null hypothesis of exact spherical symmetry. In this paper, we take a different point of view and propose a measure for the deviation from spherical symmetry, which is based on the minimum distance between the distribution of the vector $\big (\|Y\|, Y/ \|Y\| )^\top $ and its best approximation by a distribution of a vector $\big (\|Y_s\|, Y_s/ \|Y_s \| )^\top $ corresponding to a random vector $Y_s$ with a spherical distribution. We develop estimators for the minimum distance with corresponding statistical guarantees (provided by asymptotic theory) and demonstrate the applicability of our approach by means of a simulation study and a real data example.

stat.ME

Inference for Multiple Change-points in Piecewise Locally Stationary Time Series

Change-point detection and locally stationary time series modeling are two major approaches for the analysis of non-stationary data. The former aims to identify stationary phases by detecting abrupt changes in the dynamics of a time series model, while the latter employs (locally) time-varying models to describe smooth changes in dependence structure of a time series. However, in some applications, abrupt and smooth changes can co-exist, and neither of the two approaches alone can model the data adequately. In this paper, we propose a novel likelihood-based procedure for the inference of multiple change-points in locally stationary time series. In contrast to traditional change-point analysis where an abrupt change occurs in a real-valued parameter, a change in locally stationary time series occurs in a parameter curve, and can be classified as a jump or a kink depending on whether the curve is discontinuous or not. We show that the proposed method can consistently estimate the number, locations, and the types of change-points. Two different asymptotic distributions corresponding respectively to jump and kink estimators are also established. Extensive simulation studies and a real data application to financial time series are provided.

stat.ME