arXiv subjects

Ryan Burn

Publications and source records attributed to Ryan Burn.

3 recordsLinked to original sources

arXiv2025

Evaluation and Optimization of Leave-one-out Cross-validation for the Lasso

I develop an algorithm to produce the piecewise quadratic that computes leave-one-out cross-validation for the lasso as a function of its hyperparameter. The algorithm can be used to find exact hyperparameters that optimize leave-one-out cross-validation either globally or locally, and its practicality is demonstrated on real-world data sets. I also show how the algorithm can be modified to compute approximate leave-one-out cross-validation, making it suitable for larger data sets.

stat.ML↗

arXiv2023

Deterministic Objective Bayesian Analysis for Spatial Models

Berger et al. (2001) and Ren et al. (2012) derived noninformative priors for Gaussian process models of spatially correlated data using the reference prior approach (Berger, Bernardo, 1991). The priors have good statistical properties and provide a basis for objective Bayesian analysis (Berger, 2006). Using a trust-region algorithm for optimization with exact equations for posterior derivatives and an adaptive sparse grid at Chebyshev nodes, this paper develops deterministic algorithms for fully Bayesian prediction and inference with the priors. Implementations of the algorithms are available at https://github.com/rnburn/bbai.

stat.ME↗

arXiv2020

Optimizing Approximate Leave-one-out Cross-validation to Tune Hyperparameters

For a large class of regularized models, leave-one-out cross-validation can be efficiently estimated with an approximate leave-one-out formula (ALO). We consider the problem of adjusting hyperparameters so as to optimize ALO. We derive efficient formulas to compute the gradient and hessian of ALO and show how to apply a second-order optimizer to find hyperparameters. We demonstrate the usefulness of the proposed approach by finding hyperparameters for regularized logistic regression and ridge regression on various real-world data sets.

stat.ML↗