arXiv subjects

Han Lin Shang

Publications and source records attributed to Han Lin Shang.

At least 19 recordsLinked to original sources

arXiv2026

Metric Subtraction for Non-Euclidean Random Objects with Applications to Residual Analysis

Residual analysis for non-Euclidean regression models is challenging due to the lack of a natural subtraction operation and a corresponding notion of conditionally centered residuals. We introduce a metric subtraction framework that yields residuals in the original response space, with their conditional Fréchet mean equal to a fixed reference object when the model is correctly specified. We establish sufficient conditions for this conditional centering property, together with a separate condition ensuring distance preservation, providing a construction principle for suitable metric subtraction operators. The proposed metric subtraction framework accommodates several important classes of metric spaces, including spheres, symmetric positive definite matrices, simplices, and Wasserstein spaces. Building on this framework, we develop a randomized test for detecting remaining conditional location effects and establish its asymptotic size and power against fixed alternatives. We further introduce two complementary diagnostic tools, signed distance plots and distance-profile-based boxplots, to visualize systematic residual patterns and identify atypical observations. For uncertainty quantification, we construct residual-based prediction sets and establish their asymptotic validity. Simulation studies and applications to New York City bicycle usage distributions and U.S. electricity generation compositions demonstrate the practical utility of the proposed framework for model assessment, residual diagnostics, and prediction in non-Euclidean regression.

stat.ME↗

arXiv2026

Quantifying Long-Range Dependence in Object-Valued Time Series

Object-valued time series, including distributions, covariance matrices, networks, and compositions, lack the linear structure required by conventional autocovariance-based definitions of long memory. We develop an intrinsic framework for defining and estimating long memory in metric spaces of negative type. An isometric embedding yields a centered Hilbert-valued process, and the trace of its lag-covariance operator provides a signed, additive measure of temporal dependence. Crucially, this trace equals the difference between the marginal mean pairwise distance and expected lagged distance, so the framework and its estimators use only distances between the original objects. We define the memory parameter through the nonsummable decay of this trace and show that it coincides with the usual parameter in Hilbert-valued settings. Estimation uses Bartlett aggregates of distance-based lag measures. Estimating the common marginal distance from the same dependent sample induces a common-centering bias in these aggregates. We derive its finite-sample form and propose iterated block-difference corrections, log-ratio and multi-bandwidth log-slope estimators, and a localized self-consistency refinement. We establish consistency, document substantial bias reduction in simulations, and find long-memory evidence in foreign-exchange return distributions and U.S. electricity-generation compositions.

stat.ME↗

arXiv2026

Forecasting multiple density-valued time series

Density-valued time series are common in practice. However, since these functions are non-negative and must integrate to one, they do not form a linear vector space, which complicates the direct application of standard time-series methods. To address these constraints, we introduce two transformations, based on the centered log-ratio transformation and the cumulative distribution function, for modeling and forecasting balanced panel data represented by multiple densities. Both transformations map densities bijectively into an unconstrained function space in which cross-sectional and serial dependence can be modeled directly. Within the resulting unconstrained space, we consider two approaches: multi-population functional time-series modeling to account for correlations among populations, and gap modeling between gender and region. Using Japanese subnational age-specific life-table death counts from 1973 to 2024, we evaluate and compare the one- to 20-step-ahead point and interval forecast accuracy of the two approaches for each transformation, and offer some general recommendations.

stat.ME↗

arXiv2026

Spherically Embedded Time Series with Unknown Trend and Periodic Components

Spherically embedded time series are time series with values naturally residing on or can be equivalently mapped to the sphere. Despite their ubiquity in diverse scientific fields, these data frequently exhibit complex non-stationarity driven by latent trend and periodic components. Traditional Euclidean time series methods fail to account for the intrinsic non-Euclidean geometry of the sphere, leaving a critical gap in rigorous methodologies for modelling and forecasting nonstationary spherically embedded time series. To address this methodological gap, we propose a unified geometric framework to analyse nonstationary spherically embedded time series. Central to our approach is a novel nonparametric spherical trend-periodicity decomposition model that uses an optimal-transport-based removal operation to sequentially extract the smooth trend and periodic components while preserving spherical topology. The resulting de-trended and de-seasonalised stationary residuals can be further modelled using a spherical autoregressive model, formalising a novel trend-periodic spherical autoregressive model. Theoretical foundations for the modelling procedure are established on the consistency under temporal dependence. Extensive simulations corroborate these theoretical guarantees and demonstrate the superior finite-sample predictive performance of the trend-periodic spherical autoregressive model. Finally, we validate the practical utility of our methodology through applications to electricity generation compositions and bike trip volume profiles, yielding significantly enhanced forecasting accuracy while providing interpretable insights into the underlying structural dynamics.

stat.ME↗

arXiv2026

Bias Correction of Long-memory Estimator of Functional Time Series via the Prefiltered Sieve Bootstrap

We investigate a bias correction procedure based on sieve bootstrapping to estimate the long-memory parameter d in stationary or nonstationary fractionally integrated processes. The resampling method implements a sieve bootstrap method on data prefiltered by a preliminary estimate of the long-memory parameter. For the initial estimate, we recommend the local polynomial Whittle with noise (LPWN) estimator in Frederiksen et al. (2012) to reduce bias, especially in the presence of a strong short-range autoregressive dependence. Through a series of simulation studies, we first highlight the issue of bias, using the local Whittle or detrended fluctuation analysis estimator. Then, we consider the LPWN estimator and show the potential improvement in bias achieved by its sieve bootstrap enhancement. As a byproduct, the sieve bootstrap can also provide confidence intervals of the memory parameter.

stat.ME↗

arXiv2026

Double Descent in High-dimensional Linear Discriminant Analysis

The double-descent phenomenon observed in modern machine learning models has challenged the classical bias-variance trade-off by showing that prediction error can decrease again as model complexity exceeds the interpolation threshold. Although this phenomenon has been extensively studied for deep neural networks and other high-capacity models, its implications for classical statistical methods remain less well understood. In this paper, we investigate double descent in linear discriminant analysis (LDA), a fundamental classification method for binary classification. Using random matrix theory, we derive the asymptotic misclassification risk of LDA in the high-dimensional regime where the dimension (p) and sample size (n) satisfy (p/n \to γ). Specifically, we obtain an explicit asymptotic expression for the LDA risk with a general population covariance matrix when (γ\in (0,1)), and for the Moore-Penrose pseudo-inverse LDA classifier under an isotropic covariance model when (γ\in (1,\infty)). Together, these results provide a complete characterization of the double-descent risk curve of LDA across both the under- and over-parameterized regimes. Simulation studies and experiments on the ARCENE cancer classification dataset demonstrate close agreement with the theoretical predictions.

stat.ME↗

arXiv2026

Forecast reconciliation meets compositional data analysis: A study of Italian cause-specific age-at-death

We introduce a novel approach that integrates compositional data analysis (CoDA) in the forecasting of hierarchical and grouped time series and apply it to improve mortality forecasts. By leveraging the inherent decomposition of mortality data by cause of death and sex, our framework addresses key challenges in mortality forecasting, ensuring internal coherence across different levels of aggregation while maintaining high predictive accuracy. Our empirical results on Italian cause-specific mortality data demonstrate that the proposed approach significantly improves the reliability of forecasts compared to the traditional CoDA, especially for total mortality forecasts. Consistent improvements in forecasting accuracy are also found for total deaths by cause and by sex.

stat.ME↗

arXiv2026

Interpretable models for forecasting high-dimensional functional time series

We study the modeling and forecasting of high-dimensional functional time series, which can be temporally dependent and cross-sectionally correlated. Central to our implementation is a functional analysis of variance by decomposing high-dimensional functional time series, such as subnational age- and sex-specific mortality observed over years, into two distinct components: a deterministic mean structure and a residual process varying over time. Unlike purely statistical dimensionality-reduction techniques, the functional analysis of variance decomposition provides an interpretable framework by partitioning the series into effects attributable to data-specific factors, such as regional and sex-level variations, and a grand functional mean. From the residual process, we implement a functional factor model to capture the remaining stochastic trends. By combining the forecasts of the residual component with the estimated deterministic structure, we obtain the forecasted curves for high-dimensional functional time series. Illustrated by the age-specific Japanese subnational mortality rates from 1975 to 2023, we evaluate and compare the accuracy of the point and interval forecasts across various forecast horizons. The results demonstrate that leveraging these interpretable components not only clarifies the underlying drivers of the data, but also improves point forecast accuracy by about 25% to 45% compared to an existing method, providing more transparent insights for evidence-based policy decisions, such as accurate modeling of financial costs of length of stay in the old-aged care facilities.

stat.ME↗

arXiv2026

Age-specific demographic modeling and forecasting: Rolling window, expanding window, or both?

Rolling and expanding windows are widely used in age-specific demographic modeling and forecasting. Building on these approaches, we propose a simple combination method that assigns equal weight to the forecasts from both schemes. Our focus is on evaluating and comparing the forecast accuracy of the two window types in modeling age-specific mortality and fertility. Based on the multi-country comparison, the superior performance of one method often persists across different forecast horizons. In the absence of prior information, our combined approach offers a robust and practical alternative.

stat.AP↗

arXiv2026

Visualizing and forecasting subnational life-table death counts: Gap forecasting methods

Subnational life-table death counts are highly correlated across time and space and differ by gender. While these associations are helpful in improving forecasts through joint modeling, less attention has been paid to identifying and understanding mortality disparities in gender and regional gaps. We propose a forecasting framework to model and forecast female life-table death counts at the national or subnational level, and to model and forecast the associated gender gap. For either females or males, we could forecast national life-table death counts and the regional gap relative to national data. By combining gender and regional gaps, we also explore the double gap by prioritizing national data and female data. Life-table death counts are unique due to their non-negativity and summability constraints. To address the constraints, we apply a one-to-one transformation, termed cumulative distribution function transformation, to obtain one- to 15-step-ahead forecasts. Using Japanese age-specific life-table death counts between ages 0 and 110+ from 1947 to 2023, we evaluate and compare point and interval forecast accuracy across gender, region, and double gaps. By focusing on these gaps, we can deepen our understanding of the possible factors driving gender and regional mortality variations.

stat.ME↗

arXiv2026

Climate sensitivity analysis -- A case study from forty years of US compositional cause of death data

Emerging climate risks pose pressing challenges for insurers, governments, and businesses worldwide, as they face growing uncertainty in quantifying the impacts of climate change from physical risks. This paper aims to understand the impact of specific climate factors on mortality by cause for subgroups of the United States (US) population, through sensitivity and scenario analysis based on an increase in temperature and sea level extremes. We apply compositional data analysis (CODA) techniques to examine cause-specific deaths, treating the density of deaths as a set of dependent, non-negative values that sum to one. We couple CODA with principal component analysis on the climate factors as a means of dimension reduction, and fit generalised additive models to better reflect the non-linear relationships between the dimension-reduced principal scores and mortality by cause. The results of our analysis indicate climate-related factors have varying impacts by cause and ages within each cause, with more pronounced increases in the proportions of deaths from hypertensive heart disease as temperature and sea level extremes increase. Scenario analysis also indicates that an increase in temperature high extremes, sea level, and rainfall, in conjunction with a decrease in low temperature extremes, lead to offsetting impacts on the proportion of deaths between climate-related causes, but less offsetting by age within causes. Furthermore, the impacts on the proportions of death are more pronounced for ages between 55 and 95, reinforcing the observation that climate-related risks have a greater impact on older (and potentially more vulnerable) subgroups of the population. For life insurers specifically, these results are consistent with the natural hedge that arises between annuity and protection products, in light of increasing climate risk.

stat.AP↗

arXiv2026

Insurance perspectives on climate-related risks

Emerging climate risks are bringing pressing challenges for insurers globally, as they face growing uncertainty in quantifying climate impacts from physical and transition risks. Climate risks are broad and can manifest through short-term changes and events, as well as long-term and sometimes irreversible changes, such as sea level rise and temperature increases. Significant economic changes are inevitable as countries transition and adapt to mitigate climate-related risks. Given the wide range of potential implications for insurers, we conducted a series of interviews with insurance practitioners and academic researchers to explore the diverse landscape of current perspectives on climate-related risks. The interviews considered how insurers can manage emerging climate risks and the role of pricing as a signal of risk. In the context of unknown unknowns for insurers in understanding long-term impacts of climate change, we identify the importance of scenario analysis in informing forward-looking drivers of change and the importance it plays in meeting evolving regulatory requirements.

physics.soc-ph↗

arXiv2026

Conformal prediction for functional time series: Application to age-specific mortality rates

In demographic literature, forecast uncertainty is often quantified with a statistical model. This model-based approach may potentially suffer from drawbacks, namely model misspecification, selection effect, and lack of finite-sample validity. We introduce a model-agnostic and distribution-free procedure, conformal prediction, for constructing prediction intervals for a functional time series. In the family of conformal prediction, split conformal prediction divides the data into training, validation, and test sets. Within the validation set, we can select optimal tuning parameters by calibrating the empirical coverage probabilities to match their nominal values. With the selected optimal tuning parameters, we then construct the prediction intervals using the same forecasting model for the holdout data in the testing set. Without sample splitting, sequential conformal prediction sequentially updates the predicted quantiles via an autoregressive process. Using Australian age- and sex-specific log mortality rates, we evaluate and compare the interval forecast accuracy, as measured by empirical coverage probability, coverage probability difference and mean interval score, between the two variants of conformal prediction.

stat.AP↗

arXiv2026

A Beta-GAM Hidden Markov Model for Proportion Time Series

We propose a hidden Markov model for univariate proportion time series taking values in (0,1), where regime switching captures latent structural changes and the emission distribution belongs to the Beta family. In each latent state, the Beta mean is linked to covariates through a generalized additive model (GAM) with spline-based smooth functions, while the Beta precision is state-specific, enabling flexible modeling of both nonlinear covariate effects and regime-dependent variability. Estimation is carried out via a penalized expectation--maximization algorithm, combining smoothing with numerical maximization of the penalized emission likelihood. To select the number of latent states and the smoothing penalty, we implement a grid search guided by standard information criteria (Akaike Information Criterion/Bayesian Information Criterion/Integrated Completed Likelihood) with a diagnostic filter that removes degenerate solutions characterized by explosive precision estimates. Uncertainty is quantified through a parametric bootstrap procedure for transition probabilities and state-dependent parameters. Simulation results demonstrate accurate recovery of transition dynamics, state precisions, and latent-state decoding. A motivating application to Russian age-specific mortality data (1960--2014, ages 0--40) illustrates how the proposed model summarizes smooth age patterns in female-to-total mortality ratios while identifying two persistent latent regimes that admit a substantive demographic interpretation in light of the country's well-documented mortality shocks that occurred over the second half of the twentieth century.

stat.ME↗

arXiv2026

Robust spatial scalar-on-function regression: A Fisher-consistent redescending M-estimation approach

We develop a Fisher-consistent redescending robust estimator for the spatial scalar-on-function regression model, where a scalar response depends on both a functional predictor and a spatial autoregressive lag. Existing estimation procedures for this model are typically based on likelihood methods or monotone-loss robust M-estimators. They may be highly sensitive to vertical outliers, leverage points in the functional predictor, and numerical instability induced by strong spatial dependence. To address these issues, we propose a new estimation framework that first applies robust functional principal component analysis to obtain a contamination-resistant finite-dimensional representation of the functional predictor and then estimates the resulting spatial regression model through a bias-corrected system of M-estimating equations. The proposed method allows redescending loss functions, including Andrews' sine and Danish losses, and jointly estimates the regression coefficients, spatial dependence parameter, and scale parameter within a unified Fisher-consistent framework. For computation, we develop a hybrid IRLS-Newton algorithm that combines weighted least-squares updates for the regression parameters with a Newton-Raphson update for the spatial parameter. We establish Fisher consistency, consistency, asymptotic normality, and the asymptotic distribution of the reconstructed slope function. Monte Carlo experiments show that the proposed estimators remain competitive under clean data and substantially outperform classical and Huber-type robust competitors under contamination, particularly in severe outlier settings. An application to French air-quality data further demonstrates improved predictive performance and stable estimation of spatial dependence. Our method has been implemented in the fcsar R package.

stat.ME↗

arXiv2026

Modeling and forecasting subnational age distribution of death counts

Existing mortality forecasting methods focus on age-specific mortality rates, which lie in an unconstrained space and overlook the distributional nature of life-table death counts. Few studies have developed and compared forecasting methods that model the shape and dynamics of the age distribution of deaths, especially at the subnational level, where data quality varies greatly. This paper presents several forecasting methods to model and forecast the subnational age distribution of death counts. The age distribution of death counts has many similarities to probability density functions, which are non-negative and have a constrained integral, and thus live in a constrained nonlinear space. To address the nonlinear nature of objects, we implement a cumulative distribution function transformation that is scale-free and has additional monotonicity. Using subnational Japanese life-table death counts from the Japanese Mortality Database (2025), we evaluate the forecast accuracy of the transformation and forecasting methods. The improved forecast accuracy of life-table death counts implemented here will be of great interest to demographers in estimating regional age-specific survival probabilities and life expectancy, and to actuaries as a foundation for exploring potential applications in determining annuity prices for various ages and maturities.

stat.ME↗

arXiv2026

Attribution of Spurious Factors from High-Dimensional Functional Time Series

This article explores a general factor structure for high-dimensional nonstationary functional time series, encompassing a wide range of factor models studied in the existing literature. We investigate the asymptotic spectral behaviors of the sample covariance operator under this general data structure. A novel fundamental sufficient condition, formulated in terms of a newly introduced effective rank tailored to this setup, is established under which empirical eigen-analysis yields spurious results, rendering sample eigenvalues and eigenvectors unreliable for accurately recovering the underlying factor structure. This generalizes the results of Onatski and Wang [2021] from typical high-dimensional time series (HDTS) to the more intricate functional framework. The newly defined effective rank is rigorously analyzed through a decomposition of the effects attributable to functional factor loadings and functional factors. Contrary to the findings in the HDTS setting, empirical eigen-analysis of models with only a small number of strong non-stationary factors may still produce spurious limits in the functional framework. Therefore, additional caution is warranted when applying covariance-based statistical methods to potentially nonstationary functional data. Simulation studies are performed to determine conditions under which spurious limits occur. Real data analysis on age-specific mortality rate data from multiple locations is conducted for evidence of spurious factors induced by empirical eigen-analysis.

stat.ME↗

arXiv2026

Forecasting age distribution of deaths across countries: Life expectancy and annuity valuation

In this paper, we provide a comprehensive cross-country validation study of compositional mortality modeling and forecasting methods. Thus, we consider two one-to-one transformations: the cumulative distribution function and the centered log-ratio transformation in compositional data analysis. Between the two transformations, the cumulative distribution function provides a scale-free way to visualize the gender gap and cross-country heterogeneity in the probability of dying by sex and country. Drawing on age-specific period life-table death counts from 24 countries in the Human Mortality Database (2025), we assess and compare the point and interval forecast accuracy of the two transformations, using the same forecasting method. Enhancing the forecast accuracy of period life-table death counts is of significant value to demographers, who rely on such forecasts to estimate survival probabilities and life expectancy, and to actuaries, who use them to price annuities across various entry ages and maturities.

stat.AP↗