arXiv · 2611.20642
Semiparametric regression with stagewise empirical risk minimization
Abstract
Maintaining both interpretability and accuracy remains a fundamental challenge in regression analysis. This paper addresses these two objectives through a semiparametric approach that combines a parametric crude guess with its nonparametric adjustment. In particular, the nonparametric adjustment is implemented via a machine learning technique called ``stagewise empirical risk minimization algorithm.'' Once a set (dictionary) of functions (words) is prepared, this algorithm constructs a convex combination of words selected iteratively from the dictionary. This selection is based on the empirical risk derived from the Bregman divergence, which enables the proposed estimators to be applied in broad settings, including generalized linear models. This paper examines these proposed semiparametric estimators theoretically and numerically. In the theoretical study, we derive a non-asymptotic error bound for the estimators constructed from a general data-dependent dictionary. Among the concrete dictionaries proposed in this paper, some can be utilized to correct variable selection in LASSO-based parametric regression, which is investigated in the numerical study. Moreover, simulation studies investigate practical performance of the proposed estimators, while real data applications further confirm their predictive performance. These studies indicate that the proposed semiparametric estimators can address the above challenge in several settings.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Daisuke Matsuno, Kanta Naito. 2026-11-16. Semiparametric regression with stagewise empirical risk minimization. https://arxiv.science/abs/2611.20642
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.