arXiv Science⌕ Search

arXiv subjects

Philipp Wacker

Publications and source records attributed to Philipp Wacker.

At least 19 recordsLinked to original sources
arXiv2025

An optimal experimental design approach to sensor placement in continuous stochastic filtering

Sequential filtering and spatial inverse problems assimilate data points distributed either temporally (in the case of filtering) or spatially (in the case of spatial inverse problems). Sometimes it is possible to choose the position of these data points (which we call sensors here) in advance, with the goal of maximising the expected information gain (or a different metric of performance) from future data, and this leads to an Optimal Experimental Design (OED) problem. Here we revisit an interpretation of optimising sensor placement as an integration with respect to a general probability measure $\xi$. This generalises the problem of discrete-time sensor placement (which corresponds to the special case where the probability measure is a mixture of Diracs) to an infinite-dimensional, but mathematically more well-behaved setting. We focus on the continuous-time stochastic filtering setting, whose solution is governed by the Zakai equation. We derive an expression for the Fr\'echet derivative of a general OED utility functional, the key to which is an adjoint (backwards in time) differential equation. This paves the way for utilising new gradient-based methods for solving the corresponding optimisation problem, as a potentially more efficient alternative to (semi-)discrete optimisation methods, e.g. based on greedy insertion and deletion of sensor placements.

math.ST↗
arXiv2025

On MAP estimates and source conditions for drift identification in SDEs

We consider the inverse problem of identifying the drift in an SDE from $n$ observations of its solution at $M+1$ distinct time points. We derive a corresponding MAP estimate, we prove differentiability properties as well as a so-called tangential cone condition for the forward operator, and we review the existing theory for related problems, which under a slightly stronger tangential cone condition would additionally yield convergence rates for the MAP estimate as $n\to\infty$. Numerical simulations in 1D indicate that such convergence rates indeed hold true.

math.NA↗
arXiv2025

Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies

Consensus-based optimization (CBO) has established itself as an efficient gradient-free optimization scheme, with attractive mathematical properties, such as mean-field convergence results for non-convex loss functions. In this work, we study CBO in the context of closed-box adversarial attacks, which are imperceptible input perturbations that aim to fool a classifier, without accessing its gradient. Our contribution is to establish a connection between the so-called consensus hopping as introduced by Riedl et al. and natural evolution strategies (NES) commonly applied in the context of adversarial attacks and to rigorously relate both methods to gradient-based optimization schemes. Beyond that, we provide a comprehensive experimental study that shows that despite the conceptual similarities, CBO can outperform NES and other evolutionary strategies in certain scenarios.

math.OC↗
arXiv2025

Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems

We introduce a gradient-free framework for Bayesian Optimal Experimental Design (BOED) in sequential settings, aimed at complex systems where gradient information is unavailable. Our method combines Ensemble Kalman Inversion (EKI) for design optimization with the Affine-Invariant Langevin Dynamics (ALDI) sampler for efficient posterior sampling-both of which are derivative-free and ensemble-based. To address the computational challenges posed by nested expectations in BOED, we propose variational Gaussian and parametrized Laplace approximations that provide tractable upper and lower bounds on the Expected Information Gain (EIG). These approximations enable scalable utility estimation in high-dimensional spaces and PDE-constrained inverse problems. We demonstrate the performance of our framework through numerical experiments ranging from linear Gaussian models to PDE-based inference tasks, highlighting the method's robustness, accuracy, and efficiency in information-driven experimental design.

stat.ML↗
arXiv2024

Mathematical description of continuous time and space replicator-mutator equations for quadratic fitness landscapes

The replicator-mutator equation is a model for populations of individuals carrying different traits, with a fitness function mediating their ability to replicate, and a stochastic model for mutation. We derive analytical solutions for the replicator-mutator equation in continuous time and for continuous traits for a quadratic fitness function. Using these results we can explain and quantify (without the need for numerical in-silico simulations) a series of evolutionary phenomena, in particular the flying kite effect, survival of the flattest, and the ability of a population to sustain itself while tracking an optimal feature which may be fixed, moving with bounded velocity in trait space, oscillating, or randomly fluctuating.

q-bio.PE↗
arXiv2024

Connections between sequential Bayesian inference and evolutionary dynamics

It has long been posited that there is a connection between the dynamical equations describing evolutionary processes in biology and sequential Bayesian learning methods. This manuscript describes new research in which this precise connection is rigorously established in the continuous time setting. Here we focus on a partial differential equation known as the Kushner-Stratonovich equation describing the evolution of the posterior density in time. Of particular importance is a piecewise smooth approximation of the observation path from which the discrete time filtering equations, which are shown to converge to a Stratonovich interpretation of the Kushner-Stratonovich equation. This smooth formulation will then be used to draw precise connections between nonlinear stochastic filtering and replicator-mutator dynamics. Additionally, gradient flow formulations will be investigated as well as a form of replicator-mutator dynamics which is shown to be beneficial for the misspecified model filtering problem. It is hoped this work will spur further research into exchanges between sequential learning and evolutionary biology and to inspire new algorithms in filtering and sampling.

math.PR↗
arXiv2024

How to survive the Squid Games using probability theory

In this paper, we consider how probability theory can be used to determine the survival strategy in two of the ``Squid Game" and ``Squid Game: The Challenge" challenges: the Hopscotch and the Warships. We show how Hopscotch can be easily tackled with the knowledge of the binomial distribution, taught in introductory statistics courses, while Warships is a much more complex problem, which can be tackled at different levels.

stat.OT↗
arXiv2024

Perspectives on locally weighted ensemble Kalman methods

This manuscript derives locally weighted ensemble Kalman methods from the point of view of ensemble-based function approximation. This is done by using pointwise evaluations to build up a local linear or quadratic approximation of a function, tapering off the effect of distant particles via local weighting. This introduces a candidate method (the locally weighted Ensemble Kalman method for inversion) with the motivation of combining some of the strengths of the particle filter (ability to cope with nonlinear maps and non-Gaussian distributions) and the Ensemble Kalman filter (no filter degeneracy). We provide some numerical evidence for the accuracy of locally weighted ensemble methods, both in terms of approximation and inversion.

math.NA↗
arXiv2023

Nested Sampling for Uncertainty Quantification and Rare Event Estimation

Nested Sampling is a method for computing the Bayesian evidence, also called the marginal likelihood, which is the integral of the likelihood with respect to the prior. More generally, it is a numerical probabilistic quadrature rule. The main idea of Nested Sampling is to replace a high-dimensional likelihood integral over parameter space with an integral over the unit line by employing a push-forward with respect to a suitable transformation. Practically, a set of active samples ascends the level sets of the integrand function, with the measure contraction of the super-level sets being statistically estimated. We justify the validity of this approach for integrands with non-negligible plateaus, and demonstrate Nested Sampling's practical effectiveness in estimating the (log-)probability of rare events.

stat.CO↗
arXiv2022

Polarized consensus-based dynamics for optimization and sampling

In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several global minima or distributions with many modes, respectively. For this, we ``polarize'' the dynamics with a localizing kernel and the resulting model can be viewed as a bounded confidence model for opinion formation in the presence of common objective. Instead of being attracted to a common weighted mean as in the original consensus-based methods, which prevents the detection of more than one minimum or mode, in our method every particle is attracted to a weighted mean which gives more weight to nearby particles. We prove that in the mean-field regime the polarized CBS dynamics are unbiased for Gaussian targets. We also prove that in the zero temperature limit and for sufficiently well-behaved strongly convex objectives the solution of the Fokker--Planck equation converges in the Wasserstein-2 distance to a Dirac measure at the minimizer. Finally, we propose a computationally more efficient generalization which works with a predefined number of clusters and improves upon our polarized baseline method for high-dimensional optimization.

math.OC↗
arXiv2022

Ensemble-based gradient inference for particle methods in optimization and sampling

We propose an approach based on function evaluations and Bayesian inference to extract higher-order differential information of objective functions {from a given ensemble of particles}. Pointwise evaluation $\{V(x^i)\}_i$ of some potential $V$ in an ensemble $\{x^i\}_i$ contains implicit information about first or higher order derivatives, which can be made explicit with little computational effort (ensemble-based gradient inference -- EGI). We suggest to use this information for the improvement of established ensemble-based numerical methods for optimization and sampling such as Consensus-based optimization and Langevin-based samplers. Numerical studies indicate that the augmented algorithms are often superior to their gradient-free variants, in particular the augmented methods help the ensembles to escape their initial domain, to explore multimodal, non-Gaussian settings and to speed up the collapse at the end of optimization dynamics.} The code for the numerical examples in this manuscript can be found in the paper's Github repository (https://github.com/MercuryBench/ensemble-based-gradient.git).

stat.ML↗
arXiv2022

Maximum a posteriori estimators in $\ell^p$ are well-defined for diagonal Gaussian priors

We prove that maximum a posteriori estimators are well-defined for diagonal Gaussian priors $\mu$ on $\ell^p$ under common assumptions on the potential $\Phi$. Further, we show connections to the Onsager--Machlup functional and provide a corrected and strongly simplified proof in the Hilbert space case $p=2$, previously established by Dashti et al (2013) and Kretschmann (2019). These corrections do not generalize to the setting $1 \leq p < \infty$, which requires a novel convexification result for the difference between the Cameron--Martin norm and the $p$-norm.

math.ST↗
arXiv2022

Nested sampling for physical scientists

We review Skilling's nested sampling (NS) algorithm for Bayesian inference and more broadly multi-dimensional integration. After recapitulating the principles of NS, we survey developments in implementing efficient NS algorithms in practice in high-dimensions, including methods for sampling from the so-called constrained prior. We outline the ways in which NS may be applied and describe the application of NS in three scientific fields in which the algorithm has proved to be useful: cosmology, gravitational-wave astronomy, and materials science. We close by making recommendations for best practice when using NS and by summarizing potential limitations and optimizations of NS.

stat.CO↗
arXiv2021

Continuous time limit of the stochastic ensemble Kalman inversion: Strong convergence analysis

The Ensemble Kalman inversion (EKI) method is a method for the estimation of unknown parameters in the context of (Bayesian) inverse problems. The method approximates the underlying measure by an ensemble of particles and iteratively applies the ensemble Kalman update to evolve (the approximation of the) prior into the posterior measure. For the convergence analysis of the EKI it is common practice to derive a continuous version, replacing the iteration with a stochastic differential equation. In this paper we validate this approach by showing that the stochastic EKI iteration converges to paths of the continuous-time stochastic differential equation by considering both the nonlinear and linear setting, and we prove convergence in probability for the former, and convergence in moments for the latter. The methods employed can also be applied to the analysis of more general numerical schemes for stochastic differential equations in general.

math.NA↗
arXiv2021

Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion

The ensemble Kalman inversion (EKI) for the solution of Bayesian inverse problems of type $y = A u +\varepsilon$, with $u$ being an unknown parameter, $y$ a given datum, and $\varepsilon$ measurement noise, is a powerful tool usually derived from a sequential Monte Carlo point of view. It describes the dynamics of an ensemble of particles $\{u^j(t)\}_{j=1}^J$, whose initial empirical measure is sampled from the prior, evolving over an artificial time $t$ towards an approximate solution of the inverse problem, with $t=1$ emulating the posterior, and $t\to\infty$ corresponding to the under-regularized minimum-norm solution of the inverse problem. Using spectral techniques, we provide a complete description of the deterministic dynamics of EKI and its asymptotic behavior in parameter space. In particular, we analyze the dynamics of naive EKI and mean-field EKI with a special focus on their time asymptotic behavior. Furthermore, we show that -- even in the deterministic case -- residuals in parameter space do not decrease monotonously in the Euclidean norm and suggest a problem-adapted norm, where monotonicity can be proved. Finally, we derive a system of ordinary differential equations governing the spectrum and eigenvectors of the covariance matrix. While the analysis is aimed at the EKI, we believe that it can be applied to understand more general particle-based dynamical systems.

math.NA↗
arXiv2020

The lion in the attic -- A resolution of the Borel--Kolmogorov paradox

The Borel--Kolmogorov paradox of conditioning with respect to events of prior probability zero has fascinated students and researchers since its discovery more than 100 years ago. Classical conditioning is only valid with respect to events of positive probability. If we ignore this constraint and condition on such sets, for example events of type $\{Y=y\}$ for a continuously distributed random variable $Y$, almost any probability measure can be chosen as the conditional measure on such sets. There have been numerous descriptions and explanations of the paradox' appearance in the setting of conditioning on a subset of probability zero. However, most treatments don't supply explicit instructions on how to avoid it. We propose to close this gap by defining a version of conditional measure which utilizes the Hausdorff measure. This makes the choice canonical in the sense that it only depends on the geometry of the space, thus removing any ambiguity. We describe the set of possible measures arising in the context of the Borel--Kolmogorov paradox and classify those coinciding with the canonical measure. The objective of this manuscript is to provide a manual for singular conditional probability: We give an explicit explanation in which settings ambiguity arises (and where not) and how to get rid of this ambiguity once and for all by a canonical choice.

math.PR↗
arXiv2020

MAP estimators for nonparametric Bayesian inverse problems in Banach spaces

In order to rigorously define maximum-a-posteriori estimators for nonparametric Bayesian inverse problems for general Banach space valued parameters, we derive and prove certain previously postulated but unproven bounds on small ball probabilities. This allows us to prove existence of MAP estimators in the Banach space setting under very mild assumptions on the loglikelihood. As a similar statement so far (as far as the author is aware) only existed in the Hilbert space setting, this closes an important gap in the literature.

math.PR↗