Prior-data fitted networks for variable selection

Omar Melikechi

Home/Archive/arXiv

arXiv · 2611.05966

Prior-data fitted networks for variable selection

Omar Melikechi

View original source ↗Read source document ↗Export metadata ↓

Abstract

Prior-data fitted networks (PFNs) have become state-of-the-art for tabular prediction but remain underexplored in other areas of statistics. In this work, we present two PFNs for Bayesian variable selection: CD-PFN identifies conditional dependencies, and PC-PFN identifies parents and children in a causal graph. Both networks estimate posterior inclusion probabilities (PIPs) without posterior samples or data-specific modeling. In semi-synthetic and real data experiments, CD-PFN and PC-PFN often outperform existing methods, demonstrating the value of PFNs for variable selection and their potential in statistics more broadly. Open-source software is available.

Explore related subjects

stat.ME stat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Omar Melikechi. 2026-11-04. Prior-data fitted networks for variable selection. https://arxiv.org/abs/2611.05966

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Bayesian MI-LASSO for Variable Selection on Multiply-Imputed Data

Multiple imputation is widely used for handling missing data in real-world applications. For variable selection on multiply-imputed datasets, however, if selection is performed on each imputed dataset separately, it can result in different sets of selected variables across datasets. MI-LASSO, one of the most commonly used approaches to this problem, regards the same variable across all separate imputed datasets as a group variable and exploits the group LASSO to yield a consistent variable selection across all the multiply-imputed datasets. In this paper, we extend MI-LASSO into the Bayesian framework and propose four Bayesian MI-LASSO models for variable selection on multiply-imputed data, including three shrinkage prior-based and one Spike-and-Slab prior-based methods. To overcome the limitations of traditional, ad hoc threshold-based model selection, we develop a principled four-step projection predictive procedure that provides robust variable selection. This procedure also facilitates approximately calibrated post-selection inference and attains near-nominal coverage under a correct selection model. Simulation studies showed that the Bayesian MI-LASSO outperformed MI-LASSO and other alternative approaches, achieving higher specificity and lower mean squared error across a range of settings. We further demonstrated these methods via a case study using a multiply-imputed dataset from the University of Michigan Dioxin Exposure Study. The R package BMIselect is available on CRAN.

stat.ME↗

arXiv2026

Online conformal inference for multi-step time series forecasting

We study the construction of distribution-free prediction intervals for multi-step time series forecasting. In this setting, forecast errors at different horizons are serially dependent, and feedback is delayed, since the error of an h-step-ahead forecast becomes available only after h periods. We propose autocorrelated multi-step conformal prediction (AcMCP), an online procedure that accommodates delayed feedback and explicitly incorporates the dependence among multi-step forecast errors into conformal calibration. To characterize this dependence, we derive first-order representations of the optimal forecast errors under a general nonstationary autoregressive data-generating process, with explicit bounds on remainder terms. AcMCP guarantees long-run coverage of at least the nominal level at every horizon, without requiring exchangeability or any assumption on how the data distribution evolves over time. We also establish a finite-sample bound on the empirical coverage shortfall, which depends on the forecast horizon and the evaluation sample size. Notably, these guarantees do not rely on the accuracy of the linear approximations. Simulations and real-data applications show that AcMCP adapts to time-varying dynamics and maintains rolling coverage close to the nominal level. By exploiting the dependence among multi-step errors, it often yields narrower intervals than its counterpart without this dependence, with similar coverage, particularly at longer horizons.

stat.ME↗

arXiv2026

Likelihood-free Posterior Density Learning for Uncertainty Quantification in Inference Problems

Generative models and those with computationally intractable likelihoods are widely used to describe complex systems in the natural sciences, social sciences, and engineering. Fitting these models to data requires likelihood-free inference methods that explore the parameter space without explicit likelihood evaluations, relying instead on sequential simulation, which comes at the cost of computational efficiency and extensive tuning. We develop an alternative framework called kernel-adaptive synthetic posterior estimation (KASPE) that uses deep learning to directly reconstruct the mapping between the observed data and a finite-dimensional parametric representation of the posterior distribution, trained on a large number of simulated datasets. We provide theoretical justification for KASPE and a formal connection to the likelihood-based approach of expectation propagation. Simulation experiments demonstrate KASPE's flexibility and performance relative to existing likelihood-free methods including approximate Bayesian computation in challenging inferential settings involving posteriors with heavy tails, multiple local modes, and over the parameters of a nonlinear dynamical system.

stat.ME↗