arXiv subjects

Mark Kon

Publications and source records attributed to Mark Kon.

At least 19 recordsLinked to original sources

arXiv2025

FINDER: Feature Inference on Noisy Datasets using Eigenspace Residuals

''Noisy'' datasets (regimes with low signal to noise ratios, small sample sizes, faulty data collection, etc) remain a key research frontier for classification methods with both theoretical and practical implications. We introduce FINDER, a rigorous framework for analyzing generic classification problems, with tailored algorithms for noisy datasets. FINDER incorporates fundamental stochastic analysis ideas into the feature learning and inference stages to optimally account for the randomness inherent to all empirical datasets. We construct ''stochastic features'' by first viewing empirical datasets as realizations from an underlying random field (without assumptions on its exact distribution) and then mapping them to appropriate Hilbert spaces. The Kosambi-Karhunen-Lo\'eve expansion (KLE) breaks these stochastic features into computable irreducible components, which allow classification over noisy datasets via an eigen-decomposition: data from different classes resides in distinct regions, identified by analyzing the spectrum of the associated operators. We validate FINDER on several challenging, data-deficient scientific domains, producing state of the art breakthroughs in: (i) Alzheimer's Disease stage classification, (ii) Remote sensing detection of deforestation. We end with a discussion on when FINDER is expected to outperform existing methods, its failure modes, and other limitations.

cs.LG↗

arXiv2025

deFOREST: Fusing Optical and Radar satellite data for Enhanced Sensing of Tree-loss

In this paper we develop a deforestation detection pipeline that incorporates optical and Synthetic Aperture Radar (SAR) data. A crucial component of the pipeline is the construction of anomaly maps of the optical data, which is done using the residual space of a discrete Karhunen-Lo\'{e}ve (KL) expansion. Anomalies are quantified using a concentration bound on the distribution of the residual components for the nominal state of the forest. This bound does not require prior knowledge on the distribution of the data. This is in contrast to statistical parametric methods that assume knowledge of the data distribution, an impractical assumption that is especially infeasible for high dimensional data such as ours. Once the optical anomaly maps are computed they are combined with SAR data, and the state of the forest is classified by using a Hidden Markov Model (HMM). We test our approach with Sentinel-1 (SAR) and Sentinel-2 (Optical) data on a $92\,km \times 92\,km$ region in the Amazon forest. The results show that both the hybrid optical-radar and optical only methods achieve high accuracy that is superior to the recent state-of-the-art hybrid method. Moreover, the hybrid method is significantly more robust in the case of sparse optical data that are common in highly cloudy regions.

stat.ML↗

arXiv2024

Asymptotic Lower Bounds for the Feedback Arc Set Problem in Random Graphs

Given a directed graph, the Minimum Feedback Arc Set (FAS) problem asks for a minimum (size) set of arcs in a directed graph, which, when removed, results in an acyclic graph. In a seminal paper, Berger and Shor [1], in 1990, developed initial upper bounds for the FAS problem in general directed graphs. Here we find asymptotic \textit{lower bounds} for the FAS problem in a class of random, oriented, directed graphs derived from the Erd\H{o}s-R\'{e}nyi model $G(n,M)$, with n vertices and M (undirected) edges, the latter randomly chosen. Each edge is then randomly given a direction to form our directed graph. We show that $$Pr\left(\textbf{Y}^* \le M \left( \frac{1}{2} -\sqrt{\frac{\log n}{\Delta_{av}}}\right)\right)$$ approaches zero exponentially in $n$, with $\textbf{Y}^*$ the (random) size of the minimum feedback arc set and $\Delta_{av}=2M/n$ the average vertex degree. Lower bounds for random tournaments, a special case, were obtained by Spencer [12] and de la Vega [13] and these are discussed. In comparing the bound above to averaged experimental FAS data on related random graphs developed by K. Hanauer [7] we find that the approximation $\textbf{Y}^*_{av} \approx M\left( \frac{1}{2} -\frac{1}{2}\sqrt{\frac{\log n}{\Delta_{av}}}\right)$ lies remarkably close graphically to the algorithmically computed average size $\textbf{Y}^*_{av}$ of minimum feedback arc sets.

math.CO↗

arXiv2024

Classifying the Concentration of the Boolean Cube for Dependent Distributions

A metric probability space $(\Omega,d)$ obeys the ${\it concentration\; of\; measure\; phenomenon}$ if subsets of measure $1/2$ enlarge to subsets of measure close to 1 as a transition parameter $\epsilon$ approaches a limit. In this paper we consider the concentration of the space itself, namely the concentration of the metric $d(x,y)$ for a fixed $y\in \Omega$. For any $y\in \Omega$, the concentration of $d(x,y)$ is guaranteed for product distributions in high dimensions $n$, as $d(x,y)$ is a Lipschitz function in $x$. In fact, in the product setting, the rate at which the metric concentrates is of the same order in $n$ for any fixed $y\in \Omega$. The same thing, however, cannot be said for certain dependent (non-product) distributions. For the Boolean cube $I_n$ (a widely analyzed simple model), we show that, for any dependent distribution, the rate of concentration of the Hamming distance $d_H(x,y)$, for a fixed $y$, depends on the choice of $y\in I_n$, and on the variance of the conditional distributions $\mu(x_k \mid x_1,\dots, x_{k-1})$, $2\leq k\leq n$. We give an inductive bound which holds for all probability distributions on the Boolean cube, and characterize the quality of concentration by a certain positive (negative) correlation condition. Our method of proof is advantageous in that it is both simple and comprehensive. We consider uniform bounding techniques when the variance of the conditional distributions is negligible, and show how this basic technique applies to the concentration of the entire class of Lipschitz functions on the Boolean cube.

math.PR↗

arXiv2024

Uncertainty quantification of receptor ligand binding sites prediction

Recent advancements in protein docking site prediction have highlighted the limitations of traditional rigid docking algorithms, like PIPER, which often neglect critical stochastic elements such as solvent-induced fluctuations. These oversights can lead to inaccuracies in identifying viable docking sites due to the complexity of high-dimensional, stochastic energy manifolds with low regularity. To address this issue, our research introduces a novel model where the molecular shapes of ligands and receptors are represented using multi-variate Karhunen-Lo `eve (KL) expansions. This method effectively captures the stochastic nature of energy manifolds, allowing for a more accurate representation of molecular interactions.Developed as a plugin for PIPER, our scientific computing software enhances the platform, delivering robust uncertainty measures for the energy manifolds of ranked binding sites. Our results demonstrate that top-ranked binding sites, characterized by lower uncertainty in the stochastic energy manifold, align closely with actual docking sites. Conversely, sites with higher uncertainty correlate with less optimal docking positions. This distinction not only validates our approach but also sets a new standard in protein docking predictions, offering substantial implications for future molecular interaction research and drug development.

q-bio.QM↗

arXiv2024

Feature Network Methods in Machine Learning and Applications

A machine learning (ML) feature network is a graph that connects ML features in learning tasks based on their similarity. This network representation allows us to view feature vectors as functions on the network. By leveraging function operations from Fourier analysis and from functional analysis, one can easily generate new and novel features, making use of the graph structure imposed on the feature vectors. Such network structures have previously been studied implicitly in image processing and computational biology. We thus describe feature networks as graph structures imposed on feature vectors, and provide applications in machine learning. One application involves graph-based generalizations of convolutional neural networks, involving structured deep learning with hierarchical representations of features that have varying depth or complexity. This extends also to learning algorithms that are able to generate useful new multilevel features. Additionally, we discuss the use of feature networks to engineer new features, which can enhance the expressiveness of the model. We give a specific example of a deep tree-structured feature network, where hierarchical connections are formed through feature clustering and feed-forward learning. This results in low learning complexity and computational efficiency. Unlike "standard" neural features which are limited to modulated (thresholded) linear combinations of adjacent ones, feature networks offer more general feedforward dependencies among features. For example, radial basis functions or graph structure-based dependencies between features can be utilized.

stat.ML↗

arXiv2024

Asymptotics of the Minimal Feedback Arc Set in Erd\H{o}s-R\'{e}nyi Graphs

Given a directed graph, the Minimal Feedback Arc Set (FAS) problem asks for a minimal set of arcs which, when removed, results in an acyclic graph. Equivalently, the FAS problem asks to find an ordering of the vertices that minimizes the number of feedback arcs. The FAS problem is considered an algorithmic problem of central importance in discrete mathematics. Our purpose in this paper is to consider the problem in the context of Erd\H{o}s-R\'{e}nyi random directed graphs, denoted $D(n,p)$, in which each possible directed arc is included with a fixed probability $p>0$. Our interest is the typical ratio of the number of feedforward arcs to the number of feedback arcs that are removed in the FAS problem. We show that as the number $n$ of vertices goes to infinity the probability that this ratio is greater than $1+\epsilon$ for any fixed $\epsilon > 0$ approaches zero. Similarly, letting $p$ go to zero as $n\rightarrow \infty$ this result remains true if $p>C\log{n}/n$ where $C$ depends on $\epsilon$.

math.CO↗

arXiv2023

Uncertainty quantification and complex analyticity of the nonlinear Poisson-Boltzmann equation for the interface problem with random domains

The nonlinear Poisson-Boltzmann equation (NPBE) is an elliptic partial differential equation used in applications such as protein interactions and biophysical chemistry (among many others). It describes the nonlinear electrostatic potential of charged bodies submerged in an ionic solution. The kinetic presence of the solvent molecules introduces randomness to the shape of a protein, and thus a more accurate model that incorporates these random perturbations of the domain is analyzed to compute the statistics of quantities of interest of the solution. When the parameterization of the random perturbations is high-dimensional, this calculation is intractable as it is subject to the curse of dimensionality. However, if the solution of the NPBE varies analytically with respect to the random parameters, the problem becomes amenable to techniques such as sparse grids and deep neural networks. In this paper, we show analyticity of the solution of the NPBE with respect to analytic perturbations of the domain by using the analytic implicit function theorem and the domain mapping method. Previous works have shown analyticity of solutions to linear elliptic equations but not for nonlinear problems. We further show how to derive \emph{a priori} bounds on the size of the region of analyticity. This method is applied to the trypsin molecule to demonstrate that the convergence rates of the quantity of interest are consistent with the analyticity result. Furthermore, the approach developed here is sufficiently general enough to be applied to other nonlinear problems in uncertainty quantification.

math.NA↗

arXiv2023

Analytic regularity of strong solutions for the complexified stochastic non-linear Poisson Boltzmann Equation

Semi-linear elliptic Partial Differential Equations (PDEs) such as the non-linear Poisson Boltzmann Equation (nPBE) is highly relevant for non-linear electrostatics in computational biology and chemistry. It is of particular importance for modeling potential fields from molecules in solvents or plasmas with stochastic fluctuations. The extensive applications include ones in condensed matter and solid state physics, chemical physics, electrochemistry, biochemistry, thermodynamics, statistical mechanics, and materials science, among others. In this paper we study the complex analytic properties of semi-linear elliptic Partial Differential Equations with respect to random fluctuations on the domain. We first prove the existence and uniqueness of the nPBE on a bounded domain in $\mathbb{R}^3$. This proof relies on the application of a contraction mapping reasoning, as the standard convex optimization argument for the deterministic nPBE no longer applies. Using the existence and uniqueness result we subsequently show that solution to the nPBE admits an analytic extension onto a well defined region in the complex hyperplane with respect to the number of stochastic variables. Due to the analytic extension, stochastic collocation theory for sparse grids predict algebraic to sub-exponential convergence rates with respect to the number of knots. A series of numerical experiments with sparse grids is consistent with this prediction and the analyticity result. Finally, this approach readily extends to a wide class of semi-linear elliptic PDEs.

math.NA↗

arXiv2023

The Topology of Negatively Associated Distributions

We consider the sets of negatively associated (NA) and negatively correlated (NC) distributions as subsets of the space $\mathcal{M}$ of all probability distributions on $\mathbb{R}^n$, in terms of their relative topological structures within the topological space of all measures on a given measurable space. We prove that the class of NA distributions has a non-empty interior with respect to the topology of the total variation metric on $\mathcal{M}$. We show however that this is not the case in the weak topology (i.e. the topology of convergence in distribution), unless the underlying probability space is finite. We consider both the convexity and the connectedness of these classes of probability measures, and also consider the two classes on their (widely studied) restrictions to the Boolean cube in $\mathbb{R}^n$.

math.PR↗

arXiv2022

Distribution-Free Stochastic Analysis and Robust Multilevel Vector Field Anomaly Detection

Massive vector field datasets are common in multi-spectral optical and radar sensors, among many other emerging areas of application. We develop a novel stochastic functional (data) analysis approach for detecting anomalies based on the covariance structure of nominal stochastic behavior across a domain. An optimal vector field Karhunen-Loeve expansion is applied to such random field data. A series of multilevel orthogonal functional subspaces is constructed from the geometry of the domain, adapted from the KL expansion. Detection is achieved by examining the projection of the random field on the multilevel basis. A critical feature of this approach is that reliable hypothesis tests are formed, which do not require prior assumptions on probability distributions of the data. The method is applied to the important problem of degradation in the Amazon forest. Due to the complexity and high dimensionality of satellite imagery, it is not feasible to assume known distributions, nor to estimate them. In addition to providing reliable hypothesis tests, our approach shows the advantage of using multiple bands of data in a vectorized complex, leading to better anomaly detection. Furthermore, using simulated data, our approach is capable of detecting subtle anomalies that are impossible to detect with PCA-based methods.

stat.ML↗

arXiv2021

Multilevel Stochastic Optimization for Imputation in Massive Medical Data Records

It has long been a recognized problem that many datasets contain significant levels of missing numerical data. A potentially critical predicate for application of machine learning methods to datasets involves addressing this problem. However, this is a challenging task. In this paper, we apply a recently developed multi-level stochastic optimization approach to the problem of imputation in massive medical records. The approach is based on computational applied mathematics techniques and is highly accurate. In particular, for the Best Linear Unbiased Predictor (BLUP) this multi-level formulation is exact, and is significantly faster and more numerically stable. This permits practical application of Kriging methods to data imputation problems for massive datasets. We test this approach on data from the National Inpatient Sample (NIS) data records, Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality. Numerical results show that the multi-level method significantly outperforms current approaches and is numerically robust. It has superior accuracy as compared with methods recommended in the recent report from HCUP. Benchmark tests show up to 75% reductions in error. Furthermore, the results are also superior to recent state of the art methods such as discriminative deep learning.

stat.ML↗

arXiv2021

Stochastic tensor space feature theory with applications to robust machine learning

In this paper we develop a Multilevel Orthogonal Subspace (MOS) Karhunen-Loeve feature theory based on stochastic tensor spaces, for the construction of robust machine learning features. Training data are treated as instances of a random field within a relevant Bochner space. Our key observation is that separate machine learning classes can reside predominantly in mostly distinct subspaces. Using the Karhunen-Loeve expansion and a hierarchical expansion of the first (nominal) class, a MOS is constructed to detect anomalous signal components, treating the second class as an outlier of the first. The projection coefficients of the input data into these subspaces are then used to train a Machine Learning (ML) classifier. These coefficients become new features from which much clearer separation surfaces can arise for the underlying classes. Tests in the blood plasma dataset (Alzheimer's Disease Neuroimaging Initiative) show dramatic increases in accuracy. This contrast to popular ML methods such as Gradient Boosting, RUS Boost, Random Forest and Neural Networks. We show that with a non-invasive blood test, high-accuracy results can be obtained for predicting AD stages such as cognitive normal, mild cognitive impairment and dementia.

stat.ML↗

arXiv2021

Existence of Strong Solution for the Complexified Non-linear Poisson Boltzmann Equation

We prove the existence and uniqueness of the complexified Nonlinear Poisson-Boltzmann Equation (nPBE) in a bounded domain in $\mathbb{R}^3$. The nPBE is a model equation in nonlinear electrostatics. The standard convex optimization argument to the complexified nPBE no longer applies, but instead, a contraction mapping argument is developed. Furthermore, we show that uniqueness can be lost if the hypotheses given are not satisfied. The complixified nPBE is highly relevant to regularity analysis of the solution of the real nPBE with respect to the dielectric (diffusion) and Debye-H\"uckel coefficients. This approach is also well-suited to investigate the existence and uniqueness problem for a wide class of semi-linear elliptic Partial Differential Equations (PDEs).

math.AP↗

arXiv2020

Change Detection: A functional analysis perspective

We develop a new approach for detecting changes in the behavior of stochastic processes and random fields based on tensor product representations such as the Karhunen-Lo\`{e}ve expansion. From the associated eigenspaces of the covariance operator a series of nested function spaces are constructed, allowing detection of signals lying in orthogonal subspaces. In particular this can succeed even if the stochastic behavior of the signal changes either in a global or local sense. A mathematical approach is developed to locate and measure sizes of extraneous components based on construction of multilevel nested subspaces. We show examples in $\mathbb{R}$ and on a spherical domain $\mathbb{S}^{2}$. However, the method is flexible, allowing the detection of orthogonal signals on general topologies, including spatio-temporal domains.

math.PR↗

arXiv2019

Continuum Limits of the 1D Discrete Time Quantum Walk

The discrete time quantum walk (DTQW) is a universal quantum computational model. Significant relationships between discrete and corresponding continuous quantum systems have been studied since the work of Pauli and Feynman. This work continues the study of relationships between discrete quantum models and their ostensive continuum counterparts by developing a formal transition between discrete and continuous quantum systems through a formal framework for continuum limits of the DTQW. Under this framework, we prove two constructive theorems concerning which internal discrete transitions ("coins") admit nontrivial continuum limits. We additionally prove that the continuous space limit of the continuous time limit of the DTQW can only yield massless states which obey the Dirac equation. Finally, we demonstrate that the continuous time limit of the DTQW can be identified with the canonical continuous time quantum walk (CTQW) when the coin is allowed to transition through the continuous limit process.

quant-ph↗

arXiv2019

Analytic regularity and stochastic collocation of high dimensional Newton iterates

In this paper we introduce concepts from uncertainty quantification (UQ) and numerical analysis for the efficient evaluation of stochastic high dimensional Newton iterates. In particular, we develop complex analytic regularity theory of the solution with respect to the random variables. This justifies the application of sparse grids for the computation of stochastic moments. Convergence rates are derived and are shown to be subexponential or algebraic with respect to the number of realizations of random perturbations. Due the accuracy of the method, sparse grids are well suited for computing low probability events with high confidence. We apply our method to the power flow problem. Numerical experiments on the 39 bus New England power system model with large stochastic loads are consistent with the theoretical convergence rates.

math.NA↗

arXiv2019

Absorption Probabilities of Quantum Walks

Quantum walks are known to have nontrivial interaction with absorbing boundaries. In particular, Ambainis et.\ al.\ \cite{ambainis01} showed that in the $(\Z ,C_1,H)$ quantum walk (one-dimensional Hadamard walk) an absorbing boundary partially reflects information. These authors also conjectured that the left absorption probabilities $P_n^{(1)}(1,0)$ related to the finite absorbing Hadamard walks $(\Z ,C_1,H,\{ 0,n\} )$ satisfy a linear fractional recurrence in $n$ (here $P_n(1,0)$ is the probability that a Hadamard walk particle initialized in $|1\rangle |R\rangle$ is eventually absorbed at $|0\rangle$ and not at $|n\rangle$). This result, as well as a third order linear recurrence in initial position $m$ of $P_n^{(m)}(1,0)$, was later proved by Bach and Borisov \cite{bach09} using techniques from complex analysis. In this paper we extend these results to general two state quantum walks and three-state Grover walks, while providing a partial calculation for absorption in $d$-dimensional Grover walks by a $d-1$-dimensional wall. In the one-dimensional cases, we prove partial reflection of information, a linear fractional recurrence in lattice size, and a linear recurrence in initial position.

quant-ph↗