arXiv subjects

Tim Jahn

Publications and source records attributed to Tim Jahn.

16 recordsLinked to original sources

arXiv2026

Minimax-Optimal Early Stopping for Continuous-Time SGD via the Discrepancy Principle

We study early stopping for a continuous-time model of stochastic gradient descent (SGD) in ill-posed linear inverse problems. We consider an a posteriori stopping rule based on the discrepancy principle, which stops the dynamics once the residual reaches the noise level. Unlike deterministic gradient flow, the stochastic dynamics exhibits persistent multiplicative fluctuations whose amplitude scales with the residual itself. Under a source condition on the initial error and a polynomial bound on the spectrum of the empirical covariance operator, we prove that, with high probability, the error at this random stopping time achieves the minimax rate over the corresponding source class, up to a logarithmic factor. Our results therefore establish the discrepancy principle as an adaptive regularization strategy for continuous-time SGD that is minimax-optimal up to a logarithmic factor, despite the persistent fluctuations induced by stochastic sampling. To our knowledge, this is the first convergence-rate result for a continuous-time model of stochastic gradient descent in inverse problems under an a posteriori stopping rule: existing analyses, including those of variance-reduced variants, bound the stopping time but do not quantify the error attained there.

math.NA↗

arXiv2026

On the convergence of an adaptive denoiser driven iterative regularization with early stopping

Solving inverse problems requires appropriate regularization techniques to ensure well-posedness and stability. In recent years, denoiser-driven methods have emerged as effective regularization strategies, achieving state-of-the-art performance in various imaging applications. However, their stability and convergence within iterative regularization frameworks remain largely unexplored. In this work, we extend the framework of Regularization by Denoising (RED) by introducing a novel denoiser-driven iterative regularization scheme, referred to as \texttt{DDIR}, that incorporates a new regularization functional based on averaged denoisers. The proposed approach employs an adaptive step-size strategy together with an \emph{a posteriori} stopping rule to ensure stability while alleviating oscillatory behavior and semi-convergence effects induced by noise. As our main theoretical contribution, we prove that the resulting reconstruction method constitutes a stable and convergent regularization scheme in the classical sense. To the best of our knowledge, this provides the first rigorous justification of \texttt{DDIR} within the framework of regularization theory. Finally, we demonstrate the performance of the proposed method through numerical experiments on image deblurring and phase retrieval Computed Tomography (CT) using three denoisers, namely median, TNRD, and TV proximal. The results highlight the effectiveness of the method in terms of reconstruction accuracy and computational efficiency.

math.NA↗

arXiv2026

Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics

We introduce Prior-Fitted Functional Flows, a generative foundation model for pharmacokinetics that enables zero-shot population synthesis and individual forecasting without manual parameter tuning. We learn functional vector fields, explicitly conditioned on the sparse, irregular data of an entire study population. This enables the generation of coherent virtual cohorts as well as forecasting of partially observed patient trajectories with calibrated uncertainty. We construct a new open-access literature corpus to inform our priors, and demonstrate state-of-the-art predictive accuracy on extensive real-world datasets.

cs.LG↗

arXiv2025

Convergence of generalized cross-validation with applications to ill-posed integral equations

In this article, we rigorously establish the consistency of generalized cross-validation as a parameter-choice rule for solving inverse problems. We prove that the index chosen by leave-one-out GCV achieves a non-asymptotic, order-optimal error bound with high probability for polynomially ill-posed compact operators. Hereby it is remarkable that the unknown true solution need not satisfy a self-similarity condition, which is generally needed for other heuristic parameter choice rules. We quantify the rate and demonstrate convergence numerically on integral equation test cases, including image deblurring and CT reconstruction.

math.NA↗

arXiv2024

Fast Summation of Radial Kernels via QMC Slicing

The fast computation of large kernel sums is a challenging task, which arises as a subproblem in any kernel method. We approach the problem by slicing, which relies on random projections to one-dimensional subspaces and fast Fourier summation. We prove bounds for the slicing error and propose a quasi-Monte Carlo (QMC) approach for selecting the projections based on spherical quadrature rules. Numerical examples demonstrate that our QMC-slicing approach significantly outperforms existing methods like (QMC-)random Fourier features, orthogonal Fourier features or non-QMC slicing on standard test datasets.

math.NA↗

arXiv2024

Early Stopping of Untrained Convolutional Neural Networks

In recent years, new regularization methods based on (deep) neural networks have shown very promising empirical performance for the numerical solution of ill-posed problems, e.g., in medical imaging and imaging science. Due to the nonlinearity of neural networks, these methods often lack satisfactory theoretical justification. In this work, we rigorously discuss the convergence of a successful unsupervised approach that utilizes untrained convolutional neural networks to represent solutions to linear ill-posed problems. Untrained neural networks are particularly appealing for many applications because they do not require paired training data. The regularization property of the approach relies solely on the architecture of the neural network instead. Due to the vast over-parameterization of the employed neural network, suitable early stopping is essential for the success of the method. We establish that the classical discrepancy principle is an adequate method for early stopping of two-layer untrained convolutional neural networks learned by gradient descent, and furthermore, it yields an approximation with minimax optimal convergence rates. Numerical results are also presented to illustrate the theoretical findings.

math.NA↗

arXiv2024

Efficient solution of ill-posed integral equations through averaging

This paper discusses the error and cost aspects of ill-posed integral equations when given discrete noisy point evaluations on a fine grid. Standard solution methods usually employ discretization schemes that are directly induced by the measurement points. Thus, they may scale unfavorably with the number of evaluation points, which can result in computational inefficiency. To address this issue, we propose an algorithm that achieves the same level of accuracy while significantly reducing computational costs. Our approach involves an initial averaging procedure to sparsify the underlying grid. To keep the exposition simple, we focus only on one-dimensional ill-posed integral equations that have sufficient smoothness. However, the approach can be generalized to more complicated two- and three-dimensional problems with appropriate modifications.

math.NA↗

arXiv2022

Noise level free regularisation of general linear inverse problems under unconstrained white noise

In this note we solve a general statistical inverse problem under absence of knowledge of both the noise level and the noise distribution via application of the (modified) heuristic discrepancy principle. Hereby the unbounded (non-Gaussian) noise is controlled via introducing an auxiliary discretisation dimension and choosing it in an adaptive fashion. We first show convergence for completely arbitrary compact forward operator and ground solution. Then the uncertainty of reaching the optimal convergence rate is quantified in a specific Bayesian-like environment.

math.NA↗

arXiv2022

Discretisation-adaptive regularisation of statistical inverse problems

We consider linear inverse problems under white noise. These types of problems can be tackled with, e.g., iterative regularisation methods and the main challenge is to determine a suitable stopping index for the iteration. Convergence results for popular adaptive methods to determine the stopping index often come along with restrictions, e.g. concerning the type of ill-posedness of the problem, the unknown solution or the error distribution. In the recent work \cite{jahn2021optimal} a modification of the discrepancy principle, one of the most widely used adaptive methods, applied to spectral cut-off regularisation was presented which provides excellent convergence properties in general settings. Here we investigate the performance of the modified discrepancy principle with other filter based regularisation methods and we hereby focus on the iterative Landweber method. We show that the method yields optimal convergence rates and present some numerical experiments confirming that it is also attractive in terms of computational complexity. The key idea is to incorporate and modify the discretisation dimension in an adaptive manner.

math.NA↗

arXiv2022

A Probabilistic Oracle Inequality and Quantification of Uncertainty of a modified Discrepancy Principle for Statistical Inverse Problems

In this note we consider spectral cut-off estimators to solve a statistical linear inverse problem under arbitrary white noise. The truncation level is determined with a recently introduced adaptive method based on the classical discrepancy principle. We provide probabilistic oracle inequalities together with quantification of uncertainty for general linear problems. Moreover, we compare the new method to existing ones, namely early stopping sequential discrepancy principle and the balancing principle, both theoretically and numerically.

math.NA↗

arXiv2021

Optimal Convergence of the Discrepancy Principle for polynomially and exponentially ill-posed Operators under White Noise

We consider a linear ill-posed equation in the Hilbert space setting under white noise. Known convergence results for the discrepancy principle are either restricted to Hilbert-Schmidt operators (and they require a self-similarity condition for the unknown solution $\hat{x}$, additional to a classical source condition) or to polynomially ill-posed operators (excluding exponentially ill-posed problems). In this work we show optimal convergence for a modified discrepancy principle for both polynomially and exponentially ill-posed operators (without further restrictions) solely under either H\"older-type or logarithmic source conditions. In particular, the method includes only a single simple hyper parameter, which does not need to be adapted to the type of ill-posedness.

math.NA↗

arXiv2021

A modified discrepancy principle to attain optimal convergence rates under unknown noise

We consider a linear ill-posed equation in the Hilbert space setting. Multiple independent unbiased measurements of the right hand side are available. A natural approach is to take the average of the measurements as an approximation of the right hand side and to estimate the data error as the inverse of the square root of the number of measurements. We calculate the optimal convergence rate (as the number of measurements tends to infinity) under classical source conditions and introduce a modified discrepancy principle, which asymptotically attains this rate.

math.NA↗

arXiv2020

Regularising linear inverse problems under unknown non-Gaussian white noise allowing repeated measurements

We deal with the solution of a generic linear inverse problem in the Hilbert space setting. The exact right hand side is unknown and only accessible through discretised measurements corrupted by white noise with unknown arbitrary distribution. The measuring process can be repeated, which allows to reduce and estimate the measurement error through averaging. We show convergence against the true solution of the infinite-dimensional problem for a priori and a posteriori regularisation schemes as the number of measurements and the dimension of the discretisation tend to infinity under natural and easily verifiable conditions for the discretisation.

math.NA↗

arXiv2020

On the Discrepancy Principle for Stochastic Gradient Descent

Stochastic gradient descent (SGD) is a promising numerical method for solving large-scale inverse problems. However, its theoretical properties remain largely underexplored in the lens of classical regularization theory. In this note, we study the classical discrepancy principle, one of the most popular \textit{a posteriori} choice rules, as the stopping criterion for SGD, and prove the finite iteration termination property and the convergence of the iterate in probability as the noise level tends to zero. The theoretical results are complemented with extensive numerical experiments.

math.NA↗

arXiv2018

Beyond the Bakushinskii veto: Regularising linear inverse problems without knowing the noise distribution

This article deals with the solution of linear ill-posed equations in Hilbert spaces. Often, one only has a corrupted measurement of the right hand side at hand and the Bakushinskii veto tells us, that we are not able to solve the equation if we do not know the noise level. But in applications it is ad hoc unrealistic to know the error of a measurement. In practice, the error of a measurement may often be estimated through averaging of multiple measurements. We integrated that in our anlaysis and obtained convergence to the true solution, with the only assumption that the measurements are unbiased, independent and identically distributed according to an unknown distribution.

math.NA↗

arXiv2015

The sensorimotor loop as a dynamical system: How regular motion primitives may emerge from self-organized limit cycles

We investigate the sensorimotor loop of simple robots simulated within the LPZRobots environment from the point of view of dynamical systems theory. For a robot with a cylindrical shaped body and an actuator controlled by a single proprioceptual neuron we find various types of periodic motions in terms of stable limit cycles. These are self-organized in the sense, that the dynamics of the actuator kicks in only, for a certain range of parameters, when the barrel is already rolling, stopping otherwise. The stability of the resulting rolling motions terminates generally, as a function of the control parameters, at points where fold bifurcations of limit cycles occur. We find that several branches of motion types exist for the same parameters, in terms of the relative frequencies of the barrel and of the actuator, having each their respective basins of attractions in terms of initial conditions. For low drivings stable limit cycles describing periodic and drifting back-and-forth motions are found additionally. These modes allow to generate symmetry breaking explorative behavior purely by the timing of an otherwise neutral signal with respect to the cyclic back-and-forth motion of the robot.

q-bio.NC↗