arXiv subjects

Wan Tian

Publications and source records attributed to Wan Tian.

15 recordsLinked to original sources

arXiv2026

Projection-Based Outlier Detection in Interval-Valued Functional Data

Outlier detection is a fundamental task for ensuring reliable statistical modeling and inference. Interval-valued functional data (IVFD), in which each observation is represented by an interval-valued curve that preserves the variability and uncertainty within the observation, have attracted increasing attention in statistics and related applications. Developing effective outlier detection procedures for IVFD is therefore an important methodological problem. To address this issue, we develop a robust projection-based outlier detection framework. We first represent each interval-valued functional observation through its center and log-radius functions and apply interval-valued functional principal component analysis (IFPCA) to obtain a joint low-dimensional representation. We then introduce the interval-valued least trimmed functional scores (ILTFS) method, which identifies a robust reference subset by minimizing a trimmed aggregate of standardized IFPCA score distances. Finally, we proposed the ILTFS-FDR outlier detection procedure by converting the resulting projection distances into empirical $p$-values and adjusting using the Benjamini--Hochberg procedure at a prespecified target false discovery rate level. Theoretically, we derive the finite-sample breakdown point of the ILTFS mean estimator and establish the descent property of the concentration-step algorithm. Simulation studies and an empirical application to high-frequency ETF data demonstrate the effectiveness and robustness of the proposed ILTFS-FDR procedure in detecting abnormal interval-valued functional observations.

stat.ME↗

arXiv2026

Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards

Classical information-theoretic learning bounds typically rely on KL mutual information and moment-generating-function (MGF) arguments, which are well matched to bounded or sub-Gaussian losses but can be ineffective when losses or rewards are heavy-tailed. We develop a tail-aware information-theoretic framework for sub-Weibull data, where the tail parameter $θ$ controls the tail heaviness: $θ=2$ corresponds to sub-Gaussian, $θ=1$ to sub-exponential, and $0<θ<1$ to genuinely heavy tails. Our key technical ingredient is a decorrelation lemma that bounds change-of-measure expectations using a shifted-log $f_θ$-divergence, which admits explicit comparisons to Rényi divergence without MGF arguments. On the empirical-process side, we establish sharp maximal inequalities and a Dudley-type chaining bound for sub-Weibull processes, with logarithmic and entropy terms raised to the power $1/θ$. These tools yield tail-adaptive selection bounds and a multiscale information-theoretic Dudley inequality based on shifted-log and Rényi mutual information. We apply our theory to large language models (LLMs) in the context of reward hacking within reinforcement learning from human feedback (RLHF). We show that Rényi-regularized alignment provides finite reward guarantees and ensures that best-of-N policies remain well-controlled, thereby mitigating the catastrophic Goodhart effects where standard KL-regularization fails. We illustrate Rényi-regularized RLHF by experiments, including controlled heavy-tailed rewards and token-space reward attacks.

stat.ML↗

arXiv2026

A Unified Detection Framework for AI-Related Content and Artifacts

Artificial intelligence (AI) is a double-edged sword: while it has achieved remarkable success across a wide range of domains, its deployment also calls for effective oversight and regulation, for which the detection of AI-related content and artifacts is perhaps the most direct and cost-effective approach. To this end, we propose a unified detection framework based on Mahalanobis distance scores (MDS), applicable to several important settings, including the detection of large language model (LLM) generated text, hallucination, watermark, and adversarial examples. A key component of the proposed method is to accurately characterize the positive class--such as human-generated text, factual statements, unwatermarked text, or non-adversarial samples--which requires an efficient and robust estimator of the covariance matrix of deep representations of positive samples before computing the MDS. Since the positive samples typically consist of multiple classes, and these classes may exhibit both homogeneity and heterogeneity, we develop joint estimation methods for both the casewise and cellwise minimum covariance determinant (MCD) estimators. We provide efficient optimization algorithms for both estimators and prove their convergence. We provide a reasonable definition of the breakdown point for the joint estimators and prove their corresponding high breakdown point properties. Empirical evaluations confirm the effectiveness of the proposed detection framework.

stat.ML↗

arXiv2026

Counterfactual Credit Policy Optimization for Multi-Agent Collaboration

Collaborative multi-agent large language models (LLMs) can solve complex reasoning tasks by decomposing roles, but reinforcement learning for such systems is limited by credit assignment: shared terminal rewards obscure individual contributions and can encourage free-riding. We introduce two optimizer-agnostic credit assignment methods for converting joint outcomes into agent-specific learning signals. Counterfactual Credit for Policy Optimization (CCPO) estimates an agent's marginal contribution by comparing the realized joint outcome with a counterfactual outcome where that agent is removed. Self-Evaluated Credit for Policy Optimization (SEPO) uses constrained self- and peer-evaluations as a verifier-anchored credit signal while keeping the external task outcome dominant. Both operate at the reward-construction layer rather than as policy optimizers, producing role-specific rewards or advantages for GRPO, GSPO, or REINFORCE++. We instantiate these credit signals in a sequential Think--Solve setting and evaluate them on mathematical reasoning benchmarks. Results show that explicit credit assignment often improves dual-agent reasoning, especially on MATH500 and several out-of-distribution settings, while gains vary across models and datasets. Our code is available at: https://github.com/bhai114/ccpo.

cs.AI↗

arXiv2026

Estimating Precision Matrices for High-Dimensional Interval-Valued Data

In the field of statistical learning and data analysis, estimating precision matrices (i.e., the inverse of covariance matrices) is a critical task, particularly for understanding dependency structures among variables. However, traditional methods often fall short when dealing with high-dimensional interval-valued data, where each observation is represented as an interval rather than a single point. This paper proposes a novel framework for estimating precision matrices in such contexts, addressing the unique challenges posed by the interval nature of the data. Specifically, we assume that the upper and lower bounds of the intervals share the same conditional dependency structure, and then formulate the interval graphical lasso optimization objective to estimate the precision matrix. At the optimization level, we provide an efficient computational approach, while at the theoretical level, we prove the sparsity and consistency of the estimator. Experimental results on simulated studies and real data applications demonstrate the superiority of the proposed method in terms of estimation precision and interpretability.

stat.ME↗

arXiv2026

Covariance Matrix Estimation for High-Dimensional Interval-Valued Data with Positive Definiteness

In the realm of high-dimensional data analysis, the estimation of covariance matrices is a fundamental task, and this holds true for interval-valued data as well. However, there is no unified definition for the covariance matrix of interval-valued data, let alone established estimation methods in high-dimensional settings. This paper presents a novel approach to estimating covariance matrices for high-dimensional interval-valued data while ensuring positive definiteness. We begin by assuming that the upper and lower bounds of interval-valued variables share the same dependency structure. Based on this assumption, we extend the classical soft-thresholding covariance matrix estimator to the interval-valued scenario, referred to as the Interval-valued Soft-Thresholding (IST) estimator. Subsequently, to ensure the positive definiteness of the estimator, we impose a positive definiteness constraint on the IST estimator. We derive an alternating direction method to solve the proposed problem and establish its convergence. Under some very mild conditions, we develop a non-asymptotic statistical theory for the proposed estimator. Simulation studies and applications to high-frequency financial data from the CSI 300 Index demonstrated the effectiveness of the proposed estimator.

stat.ME↗

arXiv2026

Machine Learning-Assisted High-Dimensional Matrix Estimation

Efficient estimation of high-dimensional matrices-including covariance and precision matrices-is a cornerstone of modern multivariate statistics. Most existing studies have focused primarily on the theoretical properties of the estimators (e.g., consistency and sparsity), while largely overlooking the computational challenges inherent in high-dimensional settings. Motivated by recent advances in learning-based optimization method-which integrate data-driven structures with classical optimization algorithms-we explore high-dimensional matrix estimation assisted by machine learning. Specifically, for the optimization problem of high-dimensional matrix estimation, we first present a solution procedure based on the Linearized Alternating Direction Method of Multipliers (LADMM). We then introduce learnable parameters and model the proximal operators in the iterative scheme with neural networks, thereby improving estimation accuracy and accelerating convergence. Theoretically, we first prove the convergence of LADMM, and then establish the convergence, convergence rate, and monotonicity of its reparameterized counterpart; importantly, we show that the reparameterized LADMM enjoys a faster convergence rate. Notably, the proposed reparameterization theory and methodology are applicable to the estimation of both high-dimensional covariance and precision matrices. We validate the effectiveness of our method by comparing it with several classical optimization algorithms across different structures and dimensions of high-dimensional matrices.

cs.LG↗

arXiv2026

Sharper Generalization Bounds for Transformer

This paper studies generalization error bounds for Transformer models. Based on the offset Rademacher complexity, we derive sharper generalization bounds for different Transformer architectures, including single-layer single-head, single-layer multi-head, and multi-layer Transformers. We first express the excess risk of Transformers in terms of the offset Rademacher complexity. By exploiting its connection with the empirical covering numbers of the corresponding hypothesis spaces, we obtain excess risk bounds that achieve optimal convergence rates up to constant factors. We then derive refined excess risk bounds by upper bounding the covering numbers of Transformer hypothesis spaces using matrix ranks and matrix norms, leading to precise, architecture-dependent generalization bounds. Finally, we relax the boundedness assumption on feature mappings and extend our theoretical results to settings with unbounded (sub-Gaussian) features and heavy-tailed distributions.

cs.LG↗

arXiv2026

Adaptive Robust Estimator for Multi-Agent Reinforcement Learning

Multi-agent collaboration has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models, yet it suffers from interaction-level ambiguity that blurs generation, critique, and revision, making credit assignment across agents difficult. Moreover, policy optimization in this setting is vulnerable to heavy-tailed and noisy rewards, which can bias advantage estimation and trigger unstable or even divergent training. To address both issues, we propose a robust multi-agent reinforcement learning framework for collaborative reasoning, consisting of two components: Dual-Agent Answer-Critique-Rewrite (DACR) and an Adaptive Robust Estimator (ARE). DACR decomposes reasoning into a structured three-stage pipeline: answer, critique, and rewrite, while enabling explicit attribution of each agent's marginal contribution to its partner's performance. ARE provides robust estimation of batch experience means during multi-agent policy optimization. Across mathematical reasoning and embodied intelligence benchmarks, even under noisy rewards, our method consistently outperforms the baseline in both homogeneous and heterogeneous settings. These results indicate stronger robustness to reward noise and more stable training dynamics, effectively preventing optimization failures caused by noisy reward signals.

cs.AI↗

arXiv2026

Omni-Masked Gradient Descent: Memory-Efficient Optimization via Mask Traversal with Improved Convergence

Memory-efficient optimization methods have recently gained increasing attention for scaling full-parameter training of large language models under the GPU-memory bottleneck. Existing approaches either lack clear convergence guarantees, or only achieve the standard ${\mathcal{O}}(ε^{-4})$ iteration complexity in the nonconvex settings. We propose Omni-Masked Gradient Descent (OMGD), an optimization method based on mask traversal for memory efficient training, and provide a nonconvex convergence analysis that establishes a strictly improved iteration complexity of $\tilde{\mathcal{O}}(ε^{-3})$ for finding an $ε$-approximate stationary point. Empirically, OMGD is a lightweight, plug-and-play approach that integrates seamlessly into most mainstream optimizers, yielding consistent improvements over competitive baselines in both fine-tuning and pre-training tasks.

cs.LG↗

arXiv2025

RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training

Reinforcement learning with verifiable reward has recently emerged as a central paradigm for post-training large language models (LLMs); however, prevailing mean-based methods, such as Group Relative Policy Optimization (GRPO), suffer from entropy collapse and limited reasoning gains. We argue that these issues stem from overemphasizing high-probability output sequences while neglecting rare but informative reasoning paths. To address these challenges, we propose Risk-based Policy Optimization (RiskPO), which substitutes classical mean-based objectives with principled risk measures. Specifically, we introduce a Mixed Value-at-Risk objective that integrates weighted attention over multiple regions of the reward distribution, thereby amplifying gradient signals on challenging instances and preventing overconfident convergence. We further design a bundling scheme that aggregates multiple questions into bundles, thus enriching the feedback signal and yielding more stable and informative training dynamics. Theoretically, we prove that the risk-averse update alleviates entropy collapse and promotes exploration. Numerically, RiskPO achieves consistent and significant improvements in mathematical reasoning, multi-modal reasoning, and code generation benchmarks, surpassing GRPO and its variants on both Pass@1 and Pass@k metrics. Our results demonstrate that risk-based optimization provides a rigorous and effective paradigm for enhancing LLM reasoning capabilities.

cs.LG↗

arXiv2025

Interval-Valued Time Series Classification Using $D_K$-Distance

In recent years, modeling and analysis of interval-valued time series have garnered increasing attention in econometrics, finance, and statistics. However, these studies have predominantly focused on statistical inference in the forecasting of univariate and multivariate interval-valued time series, overlooking another important aspect: classification. In this paper, we introduce a classification approach that treats intervals as unified entities, applicable to both univariate and multivariate interval-valued time series. Specifically, we first extend the point-valued time series imaging methods to interval-valued scenarios using the $D_K$-distance, enabling the imaging of interval-valued time series. Then, we employ suitable deep learning model for classification on the obtained imaging dataset, aiming to achieve classification for interval-valued time series. In theory, we derived a sharper excess risk bound for deep multiclassifiers based on offset Rademacher complexity. Finally, we validate the superiority of the proposed method through comparisons with various existing point-valued time series classification methods in both simulation studies and real data applications.

stat.ML↗

arXiv2025

A model-free feature extraction procedure for interval-valued time series prediction

In this paper, we present a novel feature extraction procedure to predict interval-valued time series by combing transfer learning and imaging approaches. Initially, we represent interval-valued time series using a bivariate point-valued time series, which serves as a representative form. We first transform each time series into images by employing various imaging approaches such as recurrence plot, gramian angular summation/difference field, and Markov transition field, and construct an image dataset by treating each imaging method's output as a separate class. Based on this dataset, we train several candidates for a feature extraction network (FEN), specifically ResNet with varying layers. Then we choose the penultimate layer of the FEN to extract the most relevant features from the transformed images. We integrate the extracted features into conventional predictive models to formulate the corresponding prediction models. To formulate prediction, we integrate the extracted features into a regular prediction model. The proposed methods are evaluated based on the S\&P 500 index and three data-generating processes (DGPs), and the experimental results demonstrate a notable improvement in prediction performance compared to existing methods.

stat.AP↗

arXiv2025

Adaptive Classification of Interval-Valued Time Series

In recent years, the modeling and analysis of interval-valued time series have garnered significant attention in the fields of econometrics and statistics. However, the existing literature primarily focuses on regression tasks while neglecting classification aspects. In this paper, we propose an adaptive approach for interval-valued time series classification. Specifically, we represent interval-valued time series using convex combinations of upper and lower bounds of intervals and transform these representations into images based on point-valued time series imaging methods. We utilize a fine-grained image classification neural network to classify these images, to achieve the goal of classifying the original interval-valued time series. This proposed method is applicable to both univariate and multivariate interval-valued time series. On the optimization front, we treat the convex combination coefficients as learnable parameters similar to the parameters of the neural network and provide an efficient estimation method based on the alternating direction method of multipliers (ADMM). On the theoretical front, under specific conditions, we establish a margin-based multiclass generalization bound for generic CNNs composed of basic blocks involving convolution, pooling, and fully connected layers. Through simulation studies and real data applications, we validate the effectiveness of the proposed method and compare its performance against a wide range of point-valued time series classification methods.

stat.ML↗

arXiv2025

Block Toeplitz Sparse Precision Matrix Estimation for Large-Scale Interval-Valued Time Series Forecasting

Modeling and forecasting interval-valued time series (ITS) have attracted considerable attention due to their growing presence in various contexts. To the best of our knowledge, there have been no efforts to model large-scale ITS. In this paper, we propose a feature extraction procedure for large-scale ITS, which involves key steps such as auto-segmentation and clustering, and feature transfer learning. This procedure can be seamlessly integrated with any suitable prediction models for forecasting purposes. Specifically, we transform the automatic segmentation and clustering of ITS into the estimation of Toeplitz sparse precision matrices and assignment set. The majorization-minimization algorithm is employed to convert this highly non-convex optimization problem into two subproblems. We derive efficient dynamic programming and alternating direction method to solve these two subproblems alternately and establish their convergence properties. By employing the Joint Recurrence Plot (JRP) to image subsequence and assigning a class label to each cluster, an image dataset is constructed. Then, an appropriate neural network is chosen to train on this image dataset and used to extract features for the next step of forecasting. Real data applications demonstrate that the proposed method can effectively obtain invariant representations of the raw data and enhance forecasting performance.

stat.ML↗