On proportional volume sampling for experimental design in general spaces

Arnaud Poinas; Rémi Bardenet

Home/Archive/arXiv

arXiv · 2011.04562

On proportional volume sampling for experimental design in general spaces

Arnaud Poinas·Rémi Bardenet

View original source ↗Read source document ↗Export metadata ↓

Abstract

Optimal design for linear regression is a fundamental task in statistics. For finite design spaces, recent progress has shown that random designs drawn using proportional volume sampling (PVS) lead to approximation guarantees for A-optimal design. PVS strikes the balance between design nodes that jointly fill the design space, while marginally staying in regions of high mass under the solution of a relaxed convex version of the original problem. In this paper, we examine some of the statistical implications of a new variant of PVS for (possibly Bayesian) optimal design. Using point process machinery, we treat the case of a generic Polish design space. We show that not only are the A-optimality approximation guarantees preserved, but we obtain similar guarantees for D-optimal design that tighten recent results. Moreover, we show that PVS can be sampled in polynomial time. Unfortunately, in spite of its elegance and tractability, we demonstrate on a simple example that the practical implications of general PVS are likely limited. In the second part of the paper, we focus on applications and investigate the use of PVS as a subroutine for stochastic search heuristics. We demonstrate that PVS is a robust addition to the practitioner's toolbox, especially when the regression functions are nonstandard and the design space, while low-dimensional, has a complicated shape (e.g., nonlinear boundaries, several connected components).

Explore related subjects

stat.CO

Keep this discovery

BibTeX RIS

Arnaud Poinas, Rémi Bardenet. 2020-11-09. On proportional volume sampling for experimental design in general spaces. https://arxiv.org/abs/2011.04562

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Extended One-Liners for the Gamma, Poisson, and Binomial Distributions

We demonstrate explicit transformations of three independent uniforms with exact Gamma$(a,1)$, $a>0$, Poisson$(\lambda)$, $\lambda>0$, and Binomial$(N,p)$, $N\geq 1$, $0\leq p \leq 1$ laws using only elementary operations.

stat.CO↗

arXiv2026

Estimating Hierarchically Rank Structured Covariance Matrices

We consider the problem of estimating a high-dimensional covariance matrix from a very limited number of samples. This problem is ubiquitous in computational fluid dynamics, where a small number of fluid snapshots must be used to construct a Gramian matrix determining a reduced-order model, as well as in computational geoscience, where a small ensemble of Earth system forecasts must be used to estimate the covariance matrix associated with the forecast uncertainty. It is common practice to regularize the small-sample covariance by imposing a "localization" structure that enforces a physically realistic correlation length scale, imposing a sparsity constraint, "shrinking" towards a prescribed target, or attenuating small correlations. We propose an alternate technique that regularizes the small-sample covariance by imposing hierarchical rank structure. Compared to regularization methods that assume sparsity such as spatial localization, hierarchical rank structure accommodates a wider range of covariance matrices, roughly corresponding to situations where long-range correlations vary more smoothly than short-range ones. It also results in a data-sparse matrix format that permits highly efficient matrix-vector products. We present theory and algorithms which show how to efficiently estimate a high-dimensional, hierarchically rank structured covariance matrix from limited samples. Through an error analysis and numerical experiments with a variety of model problems, we demonstrate that these techniques are effective at reducing sampling errors, and that in many cases they achieve smaller estimation error than conventional techniques.

stat.CO↗

arXiv2026

Optimal Slice-Adaptive Tuning of Hybrid Slice Sampling

Slice sampling is a Markov chain Monte Carlo algorithm that draws its next state uniformly from a "slice"---a super-level set of the target density function---at each iteration, thereby providing automatic local adaptivity to the scale of the target. In practice the exact slice is not known, so general-purpose implementations use an approximate slice that is grown from a starting interval of length $w>0$, with a computational cost that depends on $w$. This work presents an analysis of the average per-iteration number of target density evaluations, as a function of $w$, of hybrid slice sampling with various slice-finding schemes for targets with contiguous slices. The paper uses the results of the analysis to develop automated, slice-adaptive tuning schemes along with suboptimality bounds and asymptotic convergence guarantees. Simulations demonstrate that the tuning schemes reliably yield near-optimal slice-adaptive tuning with essentially no dependence on the initial setting of $w$.

stat.CO↗