arXiv Science⌕ Search

arXiv subjects

Martin Ehler

Publications and source records attributed to Martin Ehler.

At least 19 recordsLinked to original sources

Optimal bounds for design curves on the torus

A closed rectifiable curve on the torus $\mathbb T^d=\mathbb R^d/\mathbb Z^d$ is a $t$-design curve if normalized arclength integration is exact for trigonometric polynomials $e^{2πi \langle m,x\rangle}$ with frequencies $m\in\mathbb Z^d$ satisfying $\|m\|_\infty\le t$. For the arclength $L$ of such curves, we establish the sharp lower bound \begin{equation}\label{eq:abs} L\geq \Big(\frac{(t+1)^{2d}-1}{t(t+2)}\Big)^{1/2}. \end{equation} Equality is attained by the family of Korobov curves described in \cite{Kunis,Lindblad}. For even $t$, we also derive an explicit lower bound on the length of total-degree $t$-design curves, defined by the frequency condition $\|m\|_1\leq t$. This bound is sharp for $t=2$ in every dimension and for every even $t$ in dimension two. We determine the minimum length of hyperbolic cross $t^2$-design curves in dimension two. For $d\geq3$, we establish lower bounds of optimal order for the length of hyperbolic cross $t^d$-design curves. Finally, we obtain lower bounds of optimal order for the length of $t$-design curves associated with sparse frequency sets.

math.NA↗

Optimal asymptotic bounds for design curves on manifolds

A spherical $t$-design is a finite set of points on the $d$-dimensional unit sphere such that averaging any polynomial of degree at most $t$ over these points gives its exact spherical average. We study the corresponding notion for curves, where point averages are replaced by normalized arclength integration. For fixed $d\ge 2$, the length of such curves is known to be bounded below by a positive multiple of $t^{d-1}$ as $t\to\infty$. We show that this asymptotic order is attained: for every $d\ge 2$, there exists a constant $C_d>0$ such that, for every positive integer $t$ and every prescribed length $L\ge C_dt^{d-1}$, there is a simple spherical $t$-design curve of length $L$. This establishes the analogue for curves of the theorem of Bondarenko, Radchenko, and Viazovska, which resolved the Korevaar-Meyers conjecture for spherical design points. More generally, we prove the corresponding result on closed, connected Riemannian manifolds of dimension $d\ge 2$, where eigenfunctions of the Laplace-Beltrami operator substitute for polynomials. The proof combines a geometric construction of curves with continuously adjustable local lengths and an application of Brouwer's degree theorem.

math.MG↗

Quantitative Constraints for Stable Sampling on the Sphere

We derive quantitative volume constraints for sampling measures $μ_t$ on the unit sphere $\mathbb{S}^d$ that satisfy Marcinkiewicz-Zygmund inequalities of order $t$. Using precise localization estimates for Jacobi polynomials, we obtain explicit upper and lower bounds on the $μ_t$-mass of geodesic balls at the natural scale $t^{-1}$. Whereas constants are typically left implicit in the literature, we place special emphasis on fully explicit constants, and the results are genuinely quantitative. Moreover, these bounds yield quantitative constraints for the $s$-dimensional Hausdorff volume of Marcinkiewicz-Zygmund sampling sets and, in particular, optimal lower bounds for the length of Marcinkiewicz-Zygmund curves.

math.NA↗

Quantitative estimates: How well does the discrete Fourier transform approximate the Fourier transform on $\mathbb{R}$

In order to compute the Fourier transform of a function $f$ on the real line numerically, one samples $f$ on a grid and then takes the discrete Fourier transform. We derive exact error estimates for this procedure in terms of the decay and smoothness of $f$. The analysis provides a new recipe of how to relate the number of samples, the sampling interval, and the grid size.

math.NA↗

Aliasing in Convnets: A Frame-Theoretic Perspective

Using a stride in a convolutional layer inherently introduces aliasing, which has implications for numerical stability and statistical generalization. While techniques such as the parametrizations via paraunitary systems have been used to promote orthogonal convolution and thus ensure Parseval stability, a general analysis of aliasing and its effects on the stability has not been done in this context. In this article, we adapt a frame-theoretic approach to describe aliasing in convolutional layers with 1D kernels, leading to practical estimates for stability bounds and characterizations of Parseval stability, that are tailored to take short kernel sizes into account. From this, we derive two computationally very efficient optimization objectives that promote Parseval stability via systematically suppressing aliasing. Finally, for layers with random kernels, we derive closed-form expressions for the expected value and variance of the terms that describe the aliasing effects, revealing fundamental insights into the aliasing behavior at initialization.

cs.LG↗

Hybrid spherical designs

Spherical $t$-designs are finite point sets on the unit sphere that enable exact integration of polynomials of degree at most $t$ via equal-weight quadrature. This concept has recently been extended to spherical $t$-design curves by the use of normalized path integrals. However, explicit examples of such curves are rare. We construct new spherical $t$-design curves for small $t$ based on the edges of a distinct subclass of convex polytopes. We then introduce hybrid $t$-designs that combine points and curves for exact polynomial integration of higher degree. Our constructions are based on the vertices and edges of dual pairs of convex polytopes and polynomial invariants of their symmetry group. A notable result is a hybrid $t$-design for $t=19$.

math.CO↗

Injectivity of ReLU-layers: Tools from Frame Theory

Injectivity is the defining property of a mapping that ensures no information is lost and any input can be perfectly reconstructed from its output. By performing hard thresholding, the ReLU function naturally interferes with this property, making the injectivity analysis of ReLU layers in neural networks a challenging yet intriguing task that has not yet been fully solved. This article establishes a frame theoretic perspective to approach this problem. The main objective is to develop a comprehensive characterization of the injectivity behavior of ReLU layers in terms of all three involved ingredients: (i) the weights, (ii) the bias, and (iii) the domain where the data is drawn from. Maintaining a focus on practical applications, we limit our attention to bounded domains and present two methods for numerically approximating a maximal bias for given weights and data domains. These methods provide sufficient conditions for the injectivity of a ReLU layer on those domains and yield a novel practical methodology for studying the information loss in ReLU layers. Finally, we derive explicit reconstruction formulas based on the duality concept from frame theory.

cs.LG↗

Hold Me Tight: Stable Encoder-Decoder Design for Speech Enhancement

Convolutional layers with 1-D filters are often used as frontend to encode audio signals. Unlike fixed time-frequency representations, they can adapt to the local characteristics of input data. However, 1-D filters on raw audio are hard to train and often suffer from instabilities. In this paper, we address these problems with hybrid solutions, i.e., combining theory-driven and data-driven approaches. First, we preprocess the audio signals via a auditory filterbank, guaranteeing good frequency localization for the learned encoder. Second, we use results from frame theory to define an unsupervised learning objective that encourages energy conservation and perfect reconstruction. Third, we adapt mixed compressed spectral norms as learning objectives to the encoder coefficients. Using these solutions in a low-complexity encoder-mask-decoder model significantly improves the perceptual evaluation of speech quality (PESQ) in speech enhancement.

cs.SD↗

Instabilities in Convnets for Raw Audio

What makes waveform-based deep learning so hard? Despite numerous attempts at training convolutional neural networks (convnets) for filterbank design, they often fail to outperform hand-crafted baselines. These baselines are linear time-invariant systems: as such, they can be approximated by convnets with wide receptive fields. Yet, in practice, gradient-based optimization leads to suboptimal approximations. In our article, we approach this phenomenon from the perspective of initialization. We present a theory of large deviations for the energy response of FIR filterbanks with random Gaussian weights. We find that deviations worsen for large filters and locally periodic input signals, which are both typical for audio signal processing applications. Numerical simulations align with our theory and suggest that the condition number of a convolutional layer follows a logarithmic scaling law between the number and length of the filters, which is reminiscent of discrete wavelet bases.

cs.LG↗

visClust: A visual clustering algorithm based on orthogonal projections

We present a novel clustering algorithm, visClust, that is based on lower dimensional data representations and visual interpretation. Thereto, we design a transformation that allows the data to be represented by a binary integer array enabling the use of image processing methods to select a partition. Qualitative and quantitative analyses measured in accuracy and an adjusted Rand-Index show that the algorithm performs well while requiring low runtime and RAM. We compare the results to 6 state-of-the-art algorithms with available code, confirming the quality of visClust by superior performance in most experiments. Moreover, the algorithm asks for just one obligatory input parameter while allowing optimization via optional parameters. The code is made available on GitHub and straightforward to use.

cs.CV↗

Fitting Auditory Filterbanks with Multiresolution Neural Networks

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is guaranteed to yield Gabor filters, hence an optimal time-frequency localization; yet, this strong inductive bias comes at the detriment of representational capacity. In this paper, we aim to overcome this dilemma by introducing a neural audio model, named multiresolution neural network (MuReNN). The key idea behind MuReNN is to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT). Since the scale of DWT atoms grows exponentially between octaves, the receptive fields of the subsequent learnable convolutions in MuReNN are dilated accordingly. For a given real-world dataset, we fit the magnitude response of MuReNN to that of a well-established auditory filterbank: Gammatone for speech, CQT for music, and third-octave for urban sounds, respectively. This is a form of knowledge distillation (KD), in which the filterbank ''teacher'' is engineered by domain knowledge while the neural network ''student'' is optimized from data. We compare MuReNN to the state of the art in terms of goodness of fit after KD on a hold-out set and in terms of Heisenberg time-frequency localization. Compared to convnets and Gabor convolutions, we find that MuReNN reaches state-of-the-art performance on all three optimization problems.

cs.SD↗

Convex Geometry of ReLU-layers, Injectivity on the Ball and Local Reconstruction

The paper uses a frame-theoretic setting to study the injectivity of a ReLU-layer on the closed ball of $\mathbb{R}^n$ and its non-negative part. In particular, the interplay between the radius of the ball and the bias vector is emphasized. Together with a perspective from convex geometry, this leads to a computationally feasible method of verifying the injectivity of a ReLU-layer under reasonable restrictions in terms of an upper bound of the bias vector. Explicit reconstruction formulas are provided, inspired by the duality concept from frame theory. All this gives rise to the possibility of quantifying the invertibility of a ReLU-layer and a concrete reconstruction algorithm for any input vector on the ball.

cs.LG↗

t-design curves and mobile sampling on the sphere

In analogy to classical spherical t-design points, we introduce the concept of t-design curves on the sphere. This means that the line integral along a t-design curve integrates polynomials of degree t exactly. For low degrees we construct explicit examples. We also derive lower asymptotic bounds on the lengths of t-design curves. Our main results prove the existence of asymptotically optimal t-design curves in the Euclidean 2-sphere and the existence of t-design curves in the d-sphere.

math.MG↗

Gauss Quadrature for Freud Weights, Modulation Spaces, and Marcinkiewicz-Zygmund Inequalities

We study Gauss quadrature for Freud weights and derive worst case error estimates for functions in a family of associated Sobolev spaces. For the Gaussian weight $e^{-πx^2}$ these spaces coincide with a class of modulation spaces which are well-known in (time-frequency) analysis and also appear under the name of Hermite spaces. Extensions are given to more general sets of nodes that are derived from Marcinkiewicz-Zygmund inequalities. This generalization can be interpreted as a stability result for Gauss quadrature.

math.NA↗

Blood vessel segmentation in en-face OCTA images: a frequency based method

Optical coherence tomography angiography (OCTA) is a novel noninvasive imaging modality for visualization of retinal blood flow in the human retina. Using specific OCTA imaging biomarkers for the identification of pathologies, automated image segmentations of the blood vessels can improve subsequent analysis and diagnosis. We present a novel segmentation method for vessel density identification based on frequency representations of the image, in particular, using so-called Gabor filter banks. The algorithm is evaluated qualitatively and quantitatively on an OCTA image in-house data set from $10$ eyes acquired by a Cirrus HD-OCT device. Qualitatively, the segmentation outcomes received very good visual evaluation feedback by experts. Quantitatively, we compared the resulting vessel density values with automated in-built values provided by the device. The results underline the visual evaluation. For the evaluation of the FAZ identification substep, manual annotations of $2$ expert graders were used, showing that our results coincide well in visual and quantitative manners. Lastly, we suggest the computation of adaptive local vessel density maps that allow straightforward analysis of retinal blood flow in a local manner.

eess.IV↗

Curve Based Approximation of Measures on Manifolds by Discrepancy Minimization

The approximation of probability measures on compact metric spaces and in particular on Riemannian manifoldsby atomic or empirical ones is a classical task in approximation and complexity theory with a wide range of applications. Instead of point measures we are concerned with the approximation by measures supported on Lipschitz curves. Special attention is paid to push-forward measures of Lebesgue measures on the interval by such curves. Using the discrepancy as distance between measures, we prove optimal approximation rates in terms of Lipschitz constants of curves. Having established the theoretical convergence rates, we are interested in the numerical minimization of the discrepancy between a given probability measure and the set of push-forward measures of Lebesgue measures on the interval by Lipschitz curves. We present numerical examples for measures on the 2- and 3-dimensional torus, the 2-sphere, the rotation group on $\mathbb R^3$ and the Grassmannian of all 2-dimensional linear subspaces of $\mathbb{R}^4$. Our algorithm of choice is a conjugate gradient method on these manifolds which incorporates second-oder information. For efficiently computing the gradients and the Hessians within the algorithm, we approximate the given measures by truncated Fourier series and use fast Fourier transform techniques on these manifolds.

math.OC↗