Sparse Experimental Design for Nonsmooth Estimators via Bilevel Optimization

Abstract

Optimal experimental design (OED) decides which measurements to acquire for downstream estimation. Classically, this is done by optimizing an information criterion derived from a linear-Gaussian model, for which the estimator is available in closed form. Many modern estimators such as Lasso, elastic net, or total-variation-based estimators, however, are nonsmooth and lack a closed-form solution. To extend OED to these scenarios, we recast the problem as what it implicitly is: a bilevel program whose lower level computes the deployed nonsmooth estimator and whose upper level scores its validation prediction risk. Rather than fixing the number of measurements in advance, we charge each continuous acquisition weight a concave sparsity price. As a result, how many and which measurements to keep are both outcomes of the optimization. A measurement survives only if its estimator-aware value exceeds its price, and progressively increasing the price yields a sequence of designs that explores the trade-off between measurement cardinality and estimation accuracy. Using a value-function penalty reformulation, we develop a single-loop proximal-gradient algorithm that avoids differentiating the nonsmooth solution map and establish its convergence to an (approximate) stationary point. Experiments on synthetic sparse recovery and image reconstruction demonstrate the benefits of the proposed method.