arXiv Science⌕ Search

arXiv subjects

Michael Unser

Publications and source records attributed to Michael Unser.

At least 55 records · Page 3Linked to original sources

Iteratively Refined Image Reconstruction with Learned Attentive Regularizers

We propose a regularization scheme for image reconstruction that leverages the power of deep learning while hinging on classic sparsity-promoting models. Many deep-learning-based models are hard to interpret and cumbersome to analyze theoretically. In contrast, our scheme is interpretable because it corresponds to the minimization of a series of convex problems. For each problem in the series, a mask is generated based on the previous solution to refine the regularization strength spatially. In this way, the model becomes progressively attentive to the image structure. For the underlying update operator, we prove the existence of a fixed point. As a special case, we investigate a mask generator for which the fixed-point iterations converge to a critical point of an explicit energy functional. In our experiments, we match the performance of state-of-the-art learned variational models for the solution of inverse problems. Additionally, we offer a promising balance between interpretability, theoretical guarantees, reliability, and performance.

eess.IV↗

Distributional Extension and Invertibility of the $k$-Plane Transform and Its Dual

We investigate the distributional extension of the $k$-plane transform in $\mathbb{R}^d$ and of related operators. We parameterize the $k$-plane domain as the Cartesian product of the Stiefel manifold of orthonormal $k$-frames in $\mathbb{R}^d$ with $\mathbb{R}^{d-k}$. This parameterization imposes an isotropy condition on the range of the $k$-plane transform which is analogous to the even condition on the range of the Radon transform. We use our distributional formalism to investigate the invertibility of the dual $k$-plane transform (the "backprojection" operator). We provide a systematic construction (via a completion process) to identify Banach spaces in which the backprojection operator is invertible and present some prototypical examples. These include the space of isotropic finite Radon measures and isotropic $L^p$-functions for $1 < p < \infty$. Finally, we apply our results to study a new form of regularization for inverse problems.

math.FA↗

On the Number of Regions of Piecewise Linear Neural Networks

Many feedforward neural networks (NNs) generate continuous and piecewise-linear (CPWL) mappings. Specifically, they partition the input domain into regions on which the mapping is affine. The number of these so-called linear regions offers a natural metric to characterize the expressiveness of CPWL NNs. The precise determination of this quantity is often out of reach in practice, and bounds have been proposed for specific architectures, including for ReLU and Maxout NNs. In this work, we generalize these bounds to NNs with arbitrary and possibly multivariate CPWL activation functions. We first provide upper and lower bounds on the maximal number of linear regions of a CPWL NN given its depth, width, and the number of linear regions of its activation functions. Our results rely on the combinatorial structure of convex partitions and confirm the distinctive role of depth which, on its own, is able to exponentially increase the number of regions. We then introduce a complementary stochastic framework to estimate the average number of linear regions produced by a CPWL NN. Under reasonable assumptions, the expected density of linear regions along any 1D path is bounded by the product of depth, width, and a measure of activation complexity (up to a scaling factor). This yields an identical role to the three sources of expressiveness: no exponential growth with depth is observed anymore.

cs.LG↗

Learning Weakly Convex Regularizers for Convergent Image-Reconstruction Algorithms

We propose to learn non-convex regularizers with a prescribed upper bound on their weak-convexity modulus. Such regularizers give rise to variational denoisers that minimize a convex energy. They rely on few parameters (less than 15,000) and offer a signal-processing interpretation as they mimic handcrafted sparsity-promoting regularizers. Through numerical experiments, we show that such denoisers outperform convex-regularization methods as well as the popular BM3D denoiser. Additionally, the learned regularizer can be deployed to solve inverse problems with iterative schemes that provably converge. For both CT and MRI reconstruction, the regularizer generalizes well and offers an excellent tradeoff between performance, number of parameters, guarantees, and interpretability when compared to other data-driven approaches.

eess.IV↗

Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions

Lipschitz-constrained neural networks have several advantages over unconstrained ones and can be applied to a variety of problems, making them a topic of attention in the deep learning community. Unfortunately, it has been shown both theoretically and empirically that they perform poorly when equipped with ReLU activation functions. By contrast, neural networks with learnable 1-Lipschitz linear splines are known to be more expressive. In this paper, we show that such networks correspond to global optima of a constrained functional optimization problem that consists of the training of a neural network composed of 1-Lipschitz linear layers and 1-Lipschitz freeform activation functions with second-order total-variation regularization. Further, we propose an efficient method to train these neural networks. Our numerical experiments show that our trained networks compare favorably with existing 1-Lipschitz neural architectures.

cs.LG↗

A Neural-Network-Based Convex Regularizer for Inverse Problems

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in reconstruction quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these shortcomings while retaining the boost in performance. In this work, we tackle this issue by revisiting regularizers that are the sum of convex-ridge functions. The gradient of such regularizers is parameterized by a neural network that has a single hidden layer with increasing and learnable activation functions. This neural network is trained within a few minutes as a multistep Gaussian denoiser. The numerical experiments for denoising, CT, and MRI reconstruction show improvements over methods that offer similar reliability guarantees.

eess.IV↗

On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks

In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy for the optimization path of gradient descent initialized from zero. In this paper, we study a modification of the regularization path for infinite-width 2-layer ReLU neural networks with nonzero initial distribution of the weights at different scales. By exploiting a link with unbalanced optimal-transport theory, we show that, despite the non-convexity of the 2-layer network training, this problem admits an infinite-dimensional convex counterpart. We formulate the corresponding functional-optimization problem and investigate its main properties. In particular, we show that, as the scale of the initialization ranges between $0$ and $+\infty$, the associated path interpolates continuously between the so-called kernel and rich regimes. Numerical experiments confirm that, in our setting, the scaling path and the final states of the optimization path behave similarly, even beyond these extreme points.

cs.LG↗

Mechanical Artifacts in Optical Projection Tomography: Classification and Automatic Calibration

Optical projection tomography (OPT) is a powerful tool for biomedical studies. It achieves 3D visualization of mesoscopic biological samples with high spatial resolution using conventional tomographic-reconstruction algorithms. However, various artifacts degrade the quality of the reconstructed images due to experimental imperfections in the OPT instruments. While many efforts have been made to characterize and correct for these artifacts, they focus on one specific type of artifacts. This work has two contributions. First, we systematically document a catalog of mechanical artifacts based on a 3D description of the imaging system that uses a set of angular and translational parameters. Then, we introduce a calibration algorithm that recovers the unknown system parameters fed into the final 3D iterative reconstruction algorithm for a distortion-free volumetric image. Simulations with beads data and experimental results on a fluorescent textile fiber confirm that our algorithm successfully removes miscalibration artifacts in the reconstruction.

cs.CV↗

Optical Diffraction Tomography Meets Fluorescence Localization Microscopy

We show that structural information can be extracted from single molecule localization microscopy (SMLM) data. More precisely, we reinterpret SMLM data as the measures of a phaseless optical diffraction tomography system for which the illumination sources are fluorophores within the sample. Building upon this model, we propose a joint optimization framework to estimate both the refractive index map and the position of fluorescent molecules from the sole SMLM frames.

eess.SP↗

Explicit representations for Banach subspaces of Lizorkin distributions

The Lizorkin space is well-suited for studying various operators; e.g., fractional Laplacians and the Radon transform. In this paper, we show that the space is unfortunately not complemented in the Schwartz space. However, we can show that it is dense in $C_0(\mathbb R^d)$, a property that is shared by the larger Schwartz space and that turns out to be useful for applications. Based on this result, we investigate subspaces of Lizorkin distributions that are Banach spaces and for which a continuous representation operator exists. Then, we introduce a variational framework involving these spaces and that makes use of the constructed operator. By investigating two particular cases of this framework, we are able to strengthen existing results for fractional splines and 2-layer ReLU networks.

math.FA↗

Bayesian Inversion for Nonlinear Imaging Models using Deep Generative Priors

Most modern imaging systems incorporate a computational pipeline to infer the image of interest from acquired measurements. The Bayesian approach to solve such ill-posed inverse problems involves the characterization of the posterior distribution of the image. It depends on the model of the imaging system and on prior knowledge on the image of interest. In this work, we present a Bayesian reconstruction framework for nonlinear imaging models where we specify the prior knowledge on the image through a deep generative model. We develop a tractable posterior-sampling scheme based on the Metropolis-adjusted Langevin algorithm for the class of nonlinear inverse problems where the forward model has a neural-network-like structure. This class includes most practical imaging modalities. We introduce the notion of augmented deep generative priors in order to suitably handle the recovery of quantitative images.We illustrate the advantages of our framework by applying it to two nonlinear imaging modalities-phase retrieval and optical diffraction tomography.

cs.CV↗

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

We present a statistical framework to benchmark the performance of reconstruction algorithms for linear inverse problems, in particular, neural-network-based methods that require large quantities of training data. We generate synthetic signals as realizations of sparse stochastic processes, which makes them ideally matched to variational sparsity-promoting techniques. We derive Gibbs sampling schemes to compute the minimum mean-square error estimators for processes with Laplace, Student's t, and Bernoulli-Laplace innovations. These allow our framework to provide quantitative measures of the degree of optimality (in the mean-square-error sense) for any given reconstruction method. We showcase our framework by benchmarking the performance of some well-known variational methods and convolutional neural network architectures that perform direct nonlinear reconstructions in the context of deconvolution and Fourier sampling. Our experimental results support the understanding that, while these neural networks outperform the variational methods and achieve near-optimal results in many settings, their performance deteriorates severely for signals associated with heavy-tailed distributions.

eess.SP↗

From Nano to Macro: Overview of the IEEE Bio Image and Signal Processing Technical Committee

The Bio Image and Signal Processing (BISP) Technical Committee (TC) of the IEEE Signal Processing Society (SPS) promotes activities within the broad technical field of biomedical image and signal processing. Areas of interest include medical and biological imaging, digital pathology, molecular imaging, microscopy, and associated computational imaging, image analysis, and image-guided treatment, alongside physiological signal processing, computational biology, and bioinformatics. BISP has 40 members and covers a wide range of EDICS, including CIS-MI: Medical Imaging, BIO-MIA: Medical Image Analysis, BIO-BI: Biological Imaging, BIO: Biomedical Signal Processing, BIO-BCI: Brain/Human-Computer Interfaces, and BIO-INFR: Bioinformatics. BISP plays a central role in the organization of the IEEE International Symposium on Biomedical Imaging (ISBI) and contributes to the technical sessions at the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), and the IEEE International Conference on Image Processing (ICIP). In this paper, we provide a brief history of the TC, review the technological and methodological contributions its community delivered, and highlight promising new directions we anticipate.

eess.SP↗

Stability of Image-Reconstruction Algorithms

Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularization strategies ($\ell_2$ and $\ell_1$ regularization) and present novel stability results for $\ell_p$-regularized linear inverse problems for $p\in(1,\infty)$. Our results guarantee Lipschitz continuity for small $p$ and Hölder continuity for larger $p$. They generalize well to the $L_p(Ω)$ function spaces.

math.OC↗

Phase Retrieval: From Computational Imaging to Machine Learning

Phase retrieval consists in the recovery of a complex-valued signal from intensity-only measurements. As it pervades a broad variety of applications, many researchers have striven to develop phase-retrieval algorithms. Classical approaches involve techniques as varied as generic gradient-descent routines or specialized spectral methods, to name a few. Yet, the phase-recovery problem remains a challenge to this day. Recently, however, advances in machine learning have revitalized the study of phase retrieval in two ways: significant theoretical advances have emerged from the analogy between phase retrieval and single-layer neural networks; practical breakthroughs have been obtained thanks to deep-learning regularization. In this tutorial, we review phase retrieval under a unifying framework that encompasses classical and machine-learning methods. We focus on three key elements: applications, overview of recent reconstruction algorithms, and the latest theoretical results.

physics.optics↗

Self-Supervised Isotropic Superresolution Fetal Brain MRI

Superresolution T2-weighted fetal-brain magnetic-resonance imaging (FBMRI) traditionally relies on the availability of several orthogonal low-resolution series of 2-dimensional thick slices (volumes). In practice, only a few low-resolution volumes are acquired. Thus, optimization-based image-reconstruction methods require strong regularization using hand-crafted regularizers (e.g., TV). Yet, due to in utero fetal motion and the rapidly changing fetal brain anatomy, the acquisition of the high-resolution images that are required to train supervised learning methods is difficult. In this paper, we sidestep this difficulty by providing a proof of concept of a self-supervised single-volume superresolution framework for T2-weighted FBMRI (SAIR). We validate SAIR quantitatively in a motion-free simulated environment. Our results for different noise levels and resolution ratios suggest that SAIR is comparable to multiple-volume superresolution reconstruction methods. We also evaluate SAIR qualitatively on clinical FBMRI data. The results suggest SAIR could be incorporated into current reconstruction pipelines.

eess.IV↗

Linear Inverse Problems with Hessian-Schatten Total Variation

In this paper, we characterize the class of extremal points of the unit ball of the Hessian-Schatten total variation (HTV) functional. The underlying motivation for our work stems from a general representer theorem that characterizes the solution set of regularized linear inverse problems in terms of the extremal points of the regularization ball. Our analysis is mainly based on studying the class of continuous and piecewise linear (CPWL) functions. In particular, we show that in dimension $d=2$, CPWL functions are dense in the unit ball of the HTV functional. Moreover, we prove that a CPWL function is extremal if and only if its Hessian is minimally supported. For the converse, we prove that the density result (which we have only proven for dimension $d = 2$) implies that the closure of the CPWL extreme points contains all extremal points.

math.FA↗

Mechanical Artifacts in Optical Projection Tomography: Classification and Automatic Calibration

Optical projection tomography (OPT) is a powerful tool for biomedical studies. It achieves 3D visualization of mesoscopic biological samples with high spatial resolution using conventional tomographic-reconstruction algorithms. However, various artifacts degrade the quality of the reconstructed images due to experimental imperfections in the OPT instruments. While many efforts have been made to characterize and correct for these artifacts, they focus on one specific type of artifacts, whereas a comprehensive catalog of all sorts of mechanical artifacts does not currently exist. In this work, we systematically document many mechanical artifacts. We rely on a 3D description of the imaging system that uses a set of angular and translational parameters. We provide a catalog of artifacts. It lists their cause, resulting effects, and existing correction methods. Then, we introduce an automatic calibration algorithm that is able to recover the unknown system parameters fed into the final 3D iterative reconstruction algorithm for a distortion-free volumetric image. Simulations with beads data and experimental results on a fluorescent textile fiber confirm that our algorithm successfully removes miscalibration artifacts in the reconstruction.

physics.optics↗