arXiv subjects

Felix Krahmer

Publications and source records attributed to Felix Krahmer.

At least 19 recordsLinked to original sources

arXiv2026

Informed Sinogram Interpolation for Sparse View Reconstruction

Computed tomography (CT) has been widely used in medical examinations and non-destructive testing. Micro-CT (microfocus X-ray CT system) is an advanced version that can observe the internal structures of small objects. However, secondary radiation can prevent micro-CT from imaging the full structure of the object. Given the same amount of secondary scans as primary scans, simple subtraction solves the problem at the price of doubled acquisition time. To reduce the acquisition time, one aims to limit the secondary scans to as few imaging angles as possible and interpolate to recover the missing data. Different interpolation methods have been explored for limited-angle tomography, such as polynomial or spline interpolation, compressive sensing, deep learning, etc. In contrast to many of these scenarios, the secondary imaging setup considered in this paper often exhibits simple sinusoidal structures that are not exploited directly by any of the aforementioned approaches. This paper aims to fill this gap with a continuous mathematical model describing the secondary sinogram only with information from the projection space. Building on this model, we propose an interpolation method that effectively interpolates secondary scans from limited observations.

eess.IV↗

arXiv2026

Towards Digital Halftoning on Closed Manifolds--An Error Diffusion Scheme for the $2D$ Torus based on Sigma-Delta Quantization along the Rank-one Lattice

Digital halftoning aims to represent continuous-tone images by binary patterns while preserving their visually relevant low-frequency content. Among the many available approaches, error-diffusion methods implement noise shaping through causal feedback filters and can be interpreted as two-dimensional versions of the signal quantization paradigm Sigma--Delta modulation. On closed domains, however, the terminal state of the underlying recurrence relation need not match the initial one, producing boundary artifacts. We study this problem for bandlimited functions on the two-dimensional torus. By arranging all pixels along a single closed rank-one lattice, we replace the multiple mismatches associated with separately processed rows and columns with a single terminal contribution, while retaining exact reconstruction. For a uniform lattice with $N=M^2+1$ points, we obtain first- and second-order error bounds of order $N^{-1/2}$ and $N^{-1}$. A suitable constant update eliminates the terminal mismatch and reduces the spatial localization of the error without changing these asymptotic orders. For fixed-direction rank-one lattices, the corrected first- and second-order reconstructions instead achieve rates $N^{-1}$ and $N^{-2}$. Numerical experiments illustrate a reduction in boundary artifacts compared with classical schemes applied on the Cartesian grid.

cs.IT↗

arXiv2026

KV Cache Compression Through the Lens of Transform Coding

The key-value (KV) cache stores information from past tokens and is a major memory bottleneck in long-context inference. Existing quantization methods address this bottleneck by representing the KV cache uniformly with lower-precision data types and designing quantization schemes to minimize reconstruction error in the cache itself, without accounting for how that error propagates through attention mechanisms. We prove that, under a white-noise quantization model, the expected attention-aware distortion decomposes into additive key and value contributions that factor across tokens and channels. Building on transform coding and reverse water-filling, which are classical tools from signal processing and rate-distortion theory, we introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion. On Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, evaluated across LongBench, RULER, GSM8K, MMLU-Pro, and MATH-500, our method achieves near-lossless accuracy at approximately $5.8\times$ compression, whereas each baseline degrades in at least some settings.

cs.LG↗

arXiv2026

Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier

Real-world sensor-based learning systems require uncertainty estimation that is both reliable and computationally efficient. Evidential Deep Learning (EDL) provides single-pass uncertainty estimation by modeling the class probabilities via Dirichlet distributions, where the Dirichlet parameters are predicted by a learned neural network mapping. However, this approach can lead to computational challenges, as Dirichlet expected objectives are more complex than standard supervised learning losses, complicating their analysis and implementation. We address this issue by approximating the objective of the first-order empirical risk minimization problem induced by EDL with a plug-in loss evaluated at the Dirichlet mean and show that, under mild assumptions, the approximation error decays with growing evidence for a broad class of loss functions, including mean-squared error and cross-entropy loss. As a special case, our analysis provides justification for the use of softmax in the context of uncertainty estimation, since under a particular evidence-to-Dirichlet mapping, our framework includes the standard softmax classifier. We validate the proposed simplified objectives on the Google Speech Commands dataset and show that they achieve predictive accuracy and selective prediction performance comparable to classical EDL, while being simpler to implement using standard deep learning losses and training pipelines. To the best of our knowledge, this empirical analysis is the first to obtain coverage-accuracy trade-offs for speech recognition tasks through EDL.

cs.LG↗

arXiv2026

On the reconstruction of bandlimited signals from random samples quantized via noise-shaping

Noise-shaping quantization techniques are widely used for converting bandlimited signals from the analog to the digital domain. They work by ``shaping" the quantization noise so that it falls close to the reconstruction operator's null space. We investigate the compatibility of two such schemes, specifically $ΣΔ$ quantization and distributed noise-shaping quantization, with random samples of bandlimited functions. Suppose $R>1$ is a real number and assume that $\{x_i\}_{i=1}^m$ is a sequence of i.i.d random variables uniformly distributed on $[-\tilde{R},\tilde{R}]$, where $\tilde{R}>R$ is appropriately chosen. We show that by using a noise-shaping quantizer to quantize the (randomly sign flipped) values of a real-valued $π$-bandlimited function $f$ at $\{x_i\}_{i=1}^m$, a function $f^{\sharp}$ can be reconstructed from these quantized values such that $\|f-f^{\sharp}\|_{L^2[-R, R]}$ decays with high probability as $m$ and $\tilde{R}$ increase. This decay holds uniformly over all bandlimited $f$. We emphasize that the sample points $\{x_i\}_{i=1}^m$ are completely random, that is, they have no predefined structure, which makes our findings the first of their kind.

cs.IT↗

arXiv2026

On Trajectory-Based Stability Analysis for $1$-bit Sigma-Delta Quantization and its Application to the Second-Order Case

A state-of-the-art strategy for digitally representing a bandlimited signal $f$ is $ΣΔ$ quantization. $ΣΔ$ quantization schemes choose a bit sequence $(q_n)$ representing the samples $(y_n)$ of $f$ sequentially based on a state sequence $(u_n)$ defined via a recurrence relation of the form \begin{equation*} u_n = (h*u)_n + y_n - q_n, \end{equation*} where $h_j = 0$ for $j\le 0.$ The effectiveness of a quantization scheme crucially depends on the fact that it is stable, i.e. , the state variable remains uniformly bounded in a given class of signals. Thus, a common strategy is to choose $$q_n = \operatorname{sign}((h*u)_n + y_n).$$ It is well known that a sufficient condition for this quantization rule to induce stability is that $$ \|h\|_{\ell^1}+\|f\|_{\infty}\le 2.$$ At the same time, one empirically observes that this condition is conservative and stability holds significantly beyond this bound. In this paper, we address this gap by establishing the first stability guarantees beyond first order that outperform the $\ell^1$ based stability condition. In contrast to many previous approaches, our analysis describes the trajectories of the state variables rather than characterizing the invariant set, an approach that had previously been performed only in some specific example cases. This viewpoint has the main advantage that it makes it possible to treat longer filters, which are difficult to handle through invariant-set analysis because of the resulting high dimensionality. We apply our technique to second-order $ΣΔ$ schemes with sparse feedback filters as proposed by Günturk \cite{gunturk2003one}, showing that the filter length required to guarantee stability significantly improves from the length $O\left(\frac{1}{1-\|f\|_{\infty}}\right)$ needed to apply the $\ell^1$ based criterion to $O\left(\frac{1}{\sqrt{1-\|f\|_{\infty}}}\right)$.

cs.IT↗

arXiv2026

Fast One-Pass Sparse Approximation of the Top Eigenvectors of Huge Approximately Low-Rank Matrices? Yes, $MAM^*$!

Motivated by applications such as sparse PCA, in this paper we present provably-accurate one-pass algorithms for the sparse approximation of the top eigenvectors of extremely massive matrices based on a single compact linear sketch. The resulting compressive-sensing-based approaches can approximate the leading eigenvectors of huge approximately low-rank matrices that are too large to store in memory based on a single pass over its entries while utilizing a total memory footprint on the order of the much smaller desired sparse eigenvector approximations. Finally, the compressive sensing recovery algorithm itself (which takes the gathered compressive matrix measurements as input, and then outputs sparse approximations of its top eigenvectors) can also be formulated to run in a time which principally depends on the size of the sought sparse approximations, making its runtime sublinear in the size of the large matrix whose eigenvectors one aims to approximate. Preliminary experiments on huge matrices having $\sim 10^{16}$ entries illustrate the developed theory and demonstrate the practical potential of the proposed approach.

cs.IT↗

arXiv2026

Low-Bit Quantization of Bandlimited Graph Signals via Iterative Methods

We study the quantization of real-valued bandlimited signals on graphs, focusing on low-bit representations. We propose iterative noise-shaping algorithms for quantization, including sampling approaches with and without vertex replacement. The methods leverage the spectral properties of the graph Laplacian and exploit graph incoherence to achieve high-fidelity approximations. Theoretical guarantees are provided for the random sampling method, and extensive numerical experiments on synthetic and real-world graphs illustrate the efficiency and robustness of the proposed schemes.

eess.SP↗

arXiv2025

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay

Machine learning methods are commonly used to solve inverse problems, wherein an unknown signal must be estimated from few indirect measurements generated via a known acquisition procedure. In particular, neural networks perform well empirically but have limited theoretical guarantees. In this work, we study an underdetermined linear inverse problem that admits several possible solution operators that map measurements to estimates of the target signal. A standard remedy (e.g., in compressed sensing) for establishing the uniqueness of the solution mapping is to assume the existence of a latent low-dimensional structure in the source signal. We ask the following question: do deep linear neural networks adapt to unknown low-dimensional structure when trained by gradient descent with weight decay regularization? We prove that mildly overparameterized deep linear networks trained in this manner converge to an approximate solution mapping that accurately solves the inverse problem while implicitly encoding latent subspace structure. We show rigorously that deep linear networks trained with weight decay automatically adapt to latent subspace structure in the data under practical stepsize and weight initialization schemes. Our work highlights that regularization and overparameterization improve generalization, while overparameterization also accelerates convergence during training.

cs.LG↗

arXiv2025

Conformal Prediction for Multi-Source Detection on a Network

Detecting the origin of information or infection spread in networks is a fundamental challenge with applications in misinformation tracking, epidemiology, and beyond. We study the multi-source detection problem: given snapshot observations of node infection status on a graph, estimate the set of source nodes that initiated the propagation. Existing methods either lack statistical guarantees or are limited to specific diffusion models and assumptions. We propose a novel conformal prediction framework that provides statistically valid recall guarantees for source set detection, independent of the underlying diffusion process or data distribution. Our approach introduces principled score functions to quantify the alignment between predicted probabilities and true sources, and leverages a calibration set to construct prediction sets with user-specified recall and coverage levels. The method is applicable to both single- and multi-source scenarios, supports general network diffusion dynamics, and is computationally efficient for large graphs. Empirical results demonstrate that our method achieves rigorous coverage with competitive accuracy, outperforming existing baselines in both reliability and scalability.The code is available online.

cs.SI↗

arXiv2025

Phasebook: A Survey of Selected Open Problems in Phase Retrieval

Phase retrieval is an inverse problem that, on one hand, is crucial in many applications across imaging and physics, and, on the other hand, leads to deep research questions in theoretical signal processing and applied harmonic analysis. This survey paper is an outcome of the recent workshop Phase Retrieval in Mathematics and Applications (PRiMA) (held on August 5--9 2024 at the Lorentz Center in Leiden, The Netherlands) that brought together experts working on theoretical and practical aspects of the phase retrieval problem with the purpose to formulate and explore essential open problems in the field.

cs.IT↗

arXiv2025

Spectral method for low-dose Poisson and Bernoulli phase retrieval

We consider the problem of phaseless reconstruction from measurements with Poisson or Bernoulli distributed noise. This is of particular interest in biological imaging experiments where a low dose of radiation has to be used to mitigate potential damage of the specimen, resulting in low observed particle counts. We derive recovery guarantees for the spectral method for these noise models in the case of Gaussian measurements. Our results give a quantitative insight in the trade-off between the employed radiation dose per measurement and the overall sampling complexity.

cs.IT↗

arXiv2024

Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent

We provide a rigorous analysis of implicit regularization in an overparametrized tensor factorization problem beyond the lazy training regime. For matrix factorization problems, this phenomenon has been studied in a number of works. A particular challenge has been to design universal initialization strategies which provably lead to implicit regularization in gradient-descent methods. At the same time, it has been argued by Cohen et. al. 2016 that more general classes of neural networks can be captured by considering tensor factorizations. However, in the tensor case, implicit regularization has only been rigorously established for gradient flow or in the lazy training regime. In this paper, we prove the first tensor result of its kind for gradient descent rather than gradient flow. We focus on the tubal tensor product and the associated notion of low tubal rank, encouraged by the relevance of this model for image data. We establish that gradient descent in an overparametrized tensor factorization model with a small random initialization exhibits an implicit bias towards solutions of low tubal rank. Our theoretical findings are illustrated in an extensive set of numerical simulations show-casing the dynamics predicted by our theory as well as the crucial role of using a small random initialization.

cs.LG↗

arXiv2024

With or Without Replacement? Improving Confidence in Fourier Imaging

Over the last few years, debiased estimators have been proposed in order to establish rigorous confidence intervals for high-dimensional problems in machine learning and data science. The core argument is that the error of these estimators with respect to the ground truth can be expressed as a Gaussian variable plus a remainder term that vanishes as long as the dimension of the problem is sufficiently high. Thus, uncertainty quantification (UQ) can be performed exploiting the Gaussian model. Empirically, however, the remainder term cannot be neglected in many realistic situations of moderately-sized dimensions, in particular in certain structured measurement scenarios such as Magnetic Resonance Imaging (MRI). This, in turn, can downgrade the advantage of the UQ methods as compared to non-UQ approaches such as the standard LASSO. In this paper, we present a method to improve the debiased estimator by sampling without replacement. Our approach leverages recent results of ours on the structure of the random nature of certain sampling schemes showing how a transition between sampling with and without replacement can lead to a weighted reconstruction scheme with improved performance for the standard LASSO. In this paper, we illustrate how this reweighted sampling idea can also improve the debiased estimator and, consequently, provide a better method for UQ in Fourier imaging.

eess.SP↗

arXiv2024

Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning

Uncertainty quantification (UQ) is a crucial but challenging task in many high-dimensional regression or learning problems to increase the confidence of a given predictor. We develop a new data-driven approach for UQ in regression that applies both to classical regression approaches such as the LASSO as well as to neural networks. One of the most notable UQ techniques is the debiased LASSO, which modifies the LASSO to allow for the construction of asymptotic confidence intervals by decomposing the estimation error into a Gaussian and an asymptotically vanishing bias component. However, in real-world problems with finite-dimensional data, the bias term is often too significant to be neglected, resulting in overly narrow confidence intervals. Our work rigorously addresses this issue and derives a data-driven adjustment that corrects the confidence intervals for a large class of predictors by estimating the means and variances of the bias terms from training data, exploiting high-dimensional concentration phenomena. This gives rise to non-asymptotic confidence intervals, which can help avoid overestimating uncertainty in critical applications such as MRI diagnosis. Importantly, our analysis extends beyond sparse regression to data-driven predictors like neural networks, enhancing the reliability of model-based deep learning. Our findings bridge the gap between established theory and the practical applicability of such debiased methods.

cs.LG↗

arXiv2024

High-Dimensional Confidence Regions in Sparse MRI

One of the most promising solutions for uncertainty quantification in high-dimensional statistics is the debiased LASSO that relies on unconstrained $\ell_1$-minimization. The initial works focused on real Gaussian designs as a toy model for this problem. However, in medical imaging applications, such as compressive sensing for MRI, the measurement system is represented by a (subsampled) complex Fourier matrix. The purpose of this work is to extend the method to the MRI case in order to construct confidence intervals for each pixel of an MR image. We show that a sufficient amount of data is $n \gtrsim \max\{ s_0\log^2 s_0\log p, s_0 \log^2 p \}$.

eess.SP↗

arXiv2024

The Mathematics of Dots and Pixels: On the Theoretical Foundations of Image Halftoning

The evolution of image halftoning, from its analog roots to contemporary digital methodologies, encapsulates a fascinating journey marked by technological advancements and creative innovations. Yet the theoretical understanding of halftoning is much more recent. In this article, we explore various approaches towards shedding light on the design of halftoning approaches and why they work. We discuss both halftoning in a continuous domain and on a pixel grid. We start by reviewing the mathematical foundation of the so-called electrostatic halftoning method, which departed from the heuristic of considering the back dots of the halftoned image as charged particles attracted by the grey values of the image in combination with mutual repulsion. Such an attraction-repulsion model can be mathematically represented via an energy functional in a reproducing kernel Hilbert space allowing for a rigorous analysis of the resulting optimization problem as well as a convergence analysis in a suitable topology. A second class of methods that we discuss in detail is the class of error diffusion schemes, arguably among the most popular halftoning techniques due to their ability to work directly on a pixel grid and their ease of application. The main idea of these schemes is to choose the locations of the black pixels via a recurrence relation designed to agree with the image in terms of the local averages. We discuss some recent mathematical understanding of these methods that is based on a connection to Sigma-Delta quantizers, a popular class of algorithms for analog-to-digital conversion.

eess.IV↗

arXiv2023

Uncertainty quantification for learned ISTA

Model-based deep learning solutions to inverse problems have attracted increasing attention in recent years as they bridge state-of-the-art numerical performance with interpretability. In addition, the incorporated prior domain knowledge can make the training more efficient as the smaller number of parameters allows the training step to be executed with smaller datasets. Algorithm unrolling schemes stand out among these model-based learning techniques. Despite their rapid advancement and their close connection to traditional high-dimensional statistical methods, they lack certainty estimates and a theory for uncertainty quantification is still elusive. This work provides a step towards closing this gap proposing a rigorous way to obtain confidence intervals for the LISTA estimator.

stat.ML↗