arXiv subjects

Xing Liu

Publications and source records attributed to Xing Liu.

At least 127 records · Page 7Linked to original sources

arXiv2022

Machine Learning in Heterogeneous Porous Materials

The "Workshop on Machine learning in heterogeneous porous materials" brought together international scientific communities of applied mathematics, porous media, and material sciences with experts in the areas of heterogeneous materials, machine learning (ML) and applied mathematics to identify how ML can advance materials research. Within the scope of ML and materials research, the goal of the workshop was to discuss the state-of-the-art in each community, promote crosstalk and accelerate multi-disciplinary collaborative research, and identify challenges and opportunities. As the end result, four topic areas were identified: ML in predicting materials properties, and discovery and design of novel materials, ML in porous and fractured media and time-dependent phenomena, Multi-scale modeling in heterogeneous porous materials via ML, and Discovery of materials constitutive laws and new governing equations. This workshop was part of the AmeriMech Symposium series sponsored by the National Academies of Sciences, Engineering and Medicine and the U.S. National Committee on Theoretical and Applied Mechanics.

cs.LG↗

arXiv2021

Bayesian Probabilistic Numerical Integration with Tree-Based Models

Bayesian quadrature (BQ) is a method for solving numerical integration problems in a Bayesian manner, which allows users to quantify their uncertainty about the solution. The standard approach to BQ is based on a Gaussian process (GP) approximation of the integrand. As a result, BQ is inherently limited to cases where GP approximations can be done in an efficient manner, thus often prohibiting very high-dimensional or non-smooth target functions. This paper proposes to tackle this issue with a new Bayesian numerical integration algorithm based on Bayesian Additive Regression Trees (BART) priors, which we call BART-Int. BART priors are easy to tune and well-suited for discontinuous functions. We demonstrate that they also lend themselves naturally to a sequential design setting and that explicit convergence rates can be obtained in a variety of settings. The advantages and disadvantages of this new methodology are highlighted on a set of benchmark tests including the Genz functions, and on a Bayesian survey design problem.

stat.ME↗

arXiv2021

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

While post-training model compression can greatly reduce the inference cost of a deep neural network, uncompressed training still consumes a huge amount of hardware resources, run-time and energy. It is highly desirable to directly train a compact neural network from scratch with low memory and low computational cost. Low-rank tensor decomposition is one of the most effective approaches to reduce the memory and computing requirements of large-size neural networks. However, directly training a low-rank tensorized neural network is a very challenging task because it is hard to determine a proper tensor rank {\it a priori}, which controls the model complexity and compression ratio in the training process. This paper presents a novel end-to-end framework for low-rank tensorized training of neural networks. We first develop a flexible Bayesian model that can handle various low-rank tensor formats (e.g., CP, Tucker, tensor train and tensor-train matrix) that compress neural network parameters in training. This model can automatically determine the tensor ranks inside a nonlinear forward model, which is beyond the capability of existing Bayesian tensor methods. We further develop a scalable stochastic variational inference solver to estimate the posterior density of large-scale problems in training. Our work provides the first general-purpose rank-adaptive framework for end-to-end tensorized training. Our numerical results on various neural network architectures show orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the training process. Specifically, on a very large deep learning recommendation system with over $4.2\times 10^9$ model parameters, our method can reduce the variables to only $1.6\times 10^5$ automatically in the training process (i.e., by $2.6\times 10^4$ times) while achieving almost the same accuracy.

cs.LG↗

arXiv2021

Matching in the Dark: A Dataset for Matching Image Pairs of Low-light Scenes

This paper considers matching images of low-light scenes, aiming to widen the frontier of SfM and visual SLAM applications. Recent image sensors can record the brightness of scenes with more than eight-bit precision, available in their RAW-format image. We are interested in making full use of such high-precision information to match extremely low-light scene images that conventional methods cannot handle. For extreme low-light scenes, even if some of their brightness information exists in the RAW format images' low bits, the standard raw image processing on cameras fails to utilize them properly. As was recently shown by Chen et al., CNNs can learn to produce images with a natural appearance from such RAW-format images. To consider if and how well we can utilize such information stored in RAW-format images for image matching, we have created a new dataset named MID (matching in the dark). Using it, we experimentally evaluated combinations of eight image-enhancing methods and eleven image matching methods consisting of classical/neural local descriptors and classical/neural initial point-matching methods. The results show the advantage of using the RAW-format images and the strengths and weaknesses of the above component methods. They also imply there is room for further research.

cs.CV↗

arXiv2021

3D High-Fidelity Mask Face Presentation Attack Detection Challenge

The threat of 3D masks to face recognition systems is increasingly serious and has been widely concerned by researchers. To facilitate the study of the algorithms, a large-scale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask) has been collected. Specifically, it consists of a total amount of 54, 600 videos which are recorded from 75 subjects with 225 realistic masks under 7 new kinds of sensors. Based on this dataset and Protocol 3 which evaluates both the discrimination and generalization ability of the algorithm under the open set scenarios, we organized a 3D High-Fidelity Mask Face Presentation Attack Detection Challenge to boost the research of 3D mask-based attack detection. It attracted 195 teams for the development phase with a total of 18 teams qualifying for the final round. All the results were verified and re-run by the organizing team, and the results were used for the final ranking. This paper presents an overview of the challenge, including the introduction of the dataset used, the definition of the protocol, the calculation of the evaluation criteria, and the summary and publication of the competition results. Finally, we focus on introducing and analyzing the top ranking algorithms, the conclusion summary, and the research ideas for mask attack detection provided by this competition.

cs.CV↗

arXiv2021

A New Dataset, Poisson GAN and AquaNet for Underwater Object Grabbing

To boost the object grabbing capability of underwater robots for open-sea farming, we propose a new dataset (UDD) consisting of three categories (seacucumber, seaurchin, and scallop) with 2,227 images. To the best of our knowledge, it is the first 4K HD dataset collected in a real open-sea farm. We also propose a novel Poisson-blending Generative Adversarial Network (Poisson GAN) and an efficient object detection network (AquaNet) to address two common issues within related datasets: the class-imbalance problem and the problem of mass small object, respectively. Specifically, Poisson GAN combines Poisson blending into its generator and employs a new loss called Dual Restriction loss (DR loss), which supervises both implicit space features and image-level features during training to generate more realistic images. By utilizing Poisson GAN, objects of minority class like seacucumber or scallop could be added into an image naturally and annotated automatically, which could increase the loss of minority classes during training detectors to eliminate the class-imbalance problem; AquaNet is a high-efficiency detector to address the problem of detecting mass small objects from cloudy underwater pictures. Within it, we design two efficient components: a depth-wise-convolution-based Multi-scale Contextual Features Fusion (MFF) block and a Multi-scale Blursampling (MBP) module to reduce the parameters of the network to 1.3 million. Both two components could provide multi-scale features of small objects under a short backbone configuration without any loss of accuracy. In addition, we construct a large-scale augmented dataset (AUDD) and a pre-training dataset via Poisson GAN from UDD. Extensive experiments show the effectiveness of the proposed Poisson GAN, AquaNet, UDD, AUDD, and pre-training dataset.

cs.CV↗

arXiv2021

Difference methods for time discretization of stochastic wave equation

The time discretization of stochastic spectral fractional wave equation is studied by using the difference methods. Firstly, we exploit rectangle formula to get a low order time discretization, whose the strong convergence order is smaller than $1$ in the sense of mean-squared $L^2$-norm. Meanwhile, by modifying the low order method with trapezoidal rule, the convergence rate is improved at expenses of requiring some extra temporal regularity to the solution. The modified scheme has superlinear convergence rate under the mean-squared $L^2$-norm. Several numerical experiments are provided to confirm the theoretical error estimates.

math.NA↗

arXiv2021

High-accuracy time discretization of stochastic fractional diffusion equation

A high-accuracy time discretization is discussed to numerically solve the nonlinear fractional diffusion equation forced by a space-time white noise. The main purpose of this paper is to improve the temporal convergence rate by modifying the semi-implicit Euler scheme. The solution of the equation is only Hölder continuous in time, which is disadvantageous to improve the temporal convergence rate. Firstly, the system is transformed into an equivalent form having better regularity than the original one in time. But the regularity of nonlinear term remains unchanged. Then, combining Lagrange mean value theorem and independent increments of Brownian motion leads to a higher accuracy discretization of nonlinear term which ensures the implementation of the proposed time discretization scheme without loss of convergence rate. Our scheme can improve the convergence rate from ${\min\{\fracγ{2α},\frac{1}{2}\}}$ to ${\min\{\fracγα,1\}}$ in the sense of mean-squared $L^2$-norm. The theoretical error estimates are confirmed by extensive numerical experiments.

math.NA↗

arXiv2021

Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection

Face presentation attack detection (PAD) is essential to secure face recognition systems primarily from high-fidelity mask attacks. Most existing 3D mask PAD benchmarks suffer from several drawbacks: 1) a limited number of mask identities, types of sensors, and a total number of videos; 2) low-fidelity quality of facial masks. Basic deep models and remote photoplethysmography (rPPG) methods achieved acceptable performance on these benchmarks but still far from the needs of practical scenarios. To bridge the gap to real-world applications, we introduce a largescale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask). Specifically, a total amount of 54,600 videos are recorded from 75 subjects with 225 realistic masks by 7 new kinds of sensors. Together with the dataset, we propose a novel Contrastive Context-aware Learning framework, namely CCL. CCL is a new training methodology for supervised PAD tasks, which is able to learn by leveraging rich contexts accurately (e.g., subjects, mask material and lighting) among pairs of live faces and high-fidelity mask attacks. Extensive experimental evaluations on HiFiMask and three additional 3D mask datasets demonstrate the effectiveness of our method.

cs.CV↗

arXiv2021

TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models

The memory capacity of embedding tables in deep learning recommendation models (DLRMs) is increasing dramatically from tens of GBs to TBs across the industry. Given the fast growth in DLRMs, novel solutions are urgently needed, in order to enable fast and efficient DLRM innovations. At the same time, this must be done without having to exponentially increase infrastructure capacity demands. In this paper, we demonstrate the promising potential of Tensor Train decomposition for DLRMs (TT-Rec), an important yet under-investigated context. We design and implement optimized kernels (TT-EmbeddingBag) to evaluate the proposed TT-Rec design. TT-EmbeddingBag is 3 times faster than the SOTA TT implementation. The performance of TT-Rec is further optimized with the batched matrix multiplication and caching strategies for embedding vector lookup operations. In addition, we present mathematically and empirically the effect of weight initialization distribution on DLRM accuracy and propose to initialize the tensor cores of TT-Rec following the sampled Gaussian distribution. We evaluate TT-Rec across three important design space dimensions -- memory capacity, accuracy, and timing performance -- by training MLPerf-DLRM with Criteo's Kaggle and Terabyte data sets. TT-Rec achieves 117 times and 112 times model size compression, for Kaggle and Terabyte, respectively. This impressive model size reduction can come with no accuracy nor training time overhead as compared to the uncompressed baseline.

cs.LG↗

arXiv2021

Pushing the Envelope of Thin Crack Detection

In this study, we consider the problem of detecting cracks from the image of a concrete surface for automated inspection of infrastructure, such as bridges. Its overall accuracy is determined by how accurately thin cracks with sub-pixel widths can be detected. Our interest is in making it possible to detect cracks close to the limit of thinness if it can be defined. Toward this end, we first propose a method for training a CNN to make it detect cracks more accurately than humans while training them on human-annotated labels. To achieve this seemingly impossible goal, we intentionally lower the spatial resolution of input images while maintaining that of their labels when training a CNN. This makes it possible to annotate cracks that are too thin for humans to detect, which we call super-human labels. We experimentally show that this makes it possible to detect cracks from an image of one-third the resolution of images used for annotation with about the same accuracy. We additionally propose three methods for further improving the detection accuracy of thin cracks: i) P-pooling to maintain small image structures during downsampling operations; ii) Removal of short-segment cracks in a post-processing step utilizing a prior of crack shapes learned using the VAE-GAN framework; iii) Modeling uncertainty of the prediction to better handle hard labels beyond the limit of CNNs' detection ability, which technically work as noisy labels. We experimentally examine the effectiveness of these methods.

cs.CV↗

arXiv2020

Numerical methods for the two-dimensional Fokker-Planck equation governing the probability density function of the tempered fractional Brownian motion

In this paper, we study the numerical schemes for the two-dimensional Fokker-Planck equation governing the probability density function of the tempered fractional Brownian motion. The main challenges of the numerical schemes come from the singularity in the time direction. When $0<H<0.5$, a change of variables $\partial \left(t^{2H}\right)=2Ht^{2H-1}\partial t$ avoids the singularity of numerical computation at $t=0$, which naturally results in nonuniform time discretization and greatly improves the computational efficiency. For $0.5<H<1$, the time span dependent numerical scheme and nonuniform time discretization are introduced to ensure the effectiveness of the calculation and the computational efficiency. By numerically solving the corresponding Fokker-Planck equation, we obtain the mean squared displacement of stochastic processes, which conforms to the characteristics of the tempered fractional Brownian motion.

math.NA↗

arXiv2020

Higher order approximation for stochastic wave equation

The infinitesimal generator (fractional Laplacian) of a process obtained by subordinating a killed Brownian motion catches the power-law attenuation of wave propagation. This paper studies the numerical schemes for the stochastic wave equation with fractional Laplacian as the space operator, the noise term of which is an infinite dimensional Brownian motion or fractional Brownian motion (fBm). Firstly, we establish the regularity of the mild solution of the stochastic fractional wave equation. Then a spectral Galerkin method is used for the approximation in space, and the space convergence rate is improved by postprocessing the infinite dimensional Gaussian noise. In the temporal direction, when the time derivative of the mild solution is bounded in the sense of mean-squared $L^p$-norm, we propose a modified stochastic trigonometric method, getting a higher strong convergence rate than the existing results, i.e., the time convergence rate is bigger than $1$. Particularly, for time discretization, the provided method can achieve an order of $2$ at the expenses of requiring some extra regularity to the mild solution. The theoretical error estimates are confirmed by numerical experiments.

math.NA↗

arXiv2020

Gyrokinetics investigations of an I-mode pedestal on Alcator C-Mod

Naturally stable to ELMs, and with widths larger than EPED predictions, the I-modes are an excellent laboratory for investigating the role of drift micro-instabilities in pedestals since I-mode pedestal are not "limited" by MHD instabilities. We present here a study based on gyrokinetic simulations (using GENE) to model fluctuations and heat transport in the I-mode pedestals in C-Mod. We find the Weakly Coherent Mode observed on C-Mod I-mode to be an electrostatic Ion Temperature Gradient/Impurity density gradient (ITG/Impurity) driven mode. The ITG/Impurity mode match frequency and the impurity confinement time observed on the I-mode. Nonlinear ETG simulations, can match experimental heat flux with profile adjustment well within experimental error bars. Simulations, varying impurity level (Zeff) and temperature and density profiles (within experimental error bars), are used to probe the sensitivity of fluctuations and transport.

physics.plasm-ph↗

arXiv2020

Restoring Images with Unknown Degradation Factors by Recurrent Use of a Multi-branch Network

The employment of convolutional neural networks has achieved unprecedented performance in the task of image restoration for a variety of degradation factors. However, high-performance networks have been specifically designed for a single degradation factor. In this paper, we tackle a harder problem, restoring a clean image from its degraded version with an unknown degradation factor, subject to the condition that it is one of the known factors. Toward this end, we design a network having multiple pairs of input and output branches and use it in a recurrent fashion such that a different branch pair is used at each of the recurrent paths. We reinforce the shared part of the network with improved components so that it can handle different degradation factors. We also propose a two-step training method for the network, which consists of multi-task learning and finetuning. The experimental results show that the proposed network yields at least comparable or sometimes even better performance on four degradation factors as compared with the best dedicated network for each of the four. We also test it on a further harder task where the input image contains multiple degradation factors that are mixed with unknown mixture ratios, showing that it achieves better performance than the previous state-of-the-art method designed for the task.

cs.CV↗

arXiv2019

A Compared Study Between Some Subspace Based Algorithms

The technology of face recognition has made some progress in recent years. After studying the PCA, 2DPCA, R1-PCA, L1-PCA, KPCA and KECA algorithms, in this paper ECA (2DECA) is proposed by extracting features in PCA (2DPCA) based on Renyi entropy contribution. And then we conduct a study on the 2DL1-PCA and 2DR1-PCA algorithms. On the basis of the experiments, this paper compares the difference of the recognition accuracy and operational efficiency between the above algorithms.

cs.CV↗

arXiv2019

2DR1-PCA and 2DL1-PCA: two variant 2DPCA algorithms based on none L2 norm

In this paper, two novel methods: 2DR1-PCA and 2DL1-PCA are proposed for face recognition. Compared to the traditional 2DPCA algorithm, 2DR1-PCA and 2DL1-PCA are based on the R1 norm and L1 norm, respectively. The advantage of these proposed methods is they are less sensitive to outliers. These proposed methods are tested on the ORL, YALE and XM2VTS databases and the performance of the related methods is compared experimentally.

cs.CV↗

arXiv2019

Numerical approximation for fractional diffusion equation forced by a tempered fractional Gaussian noise

This paper discusses the fractional diffusion equation forced by a tempered fractional Gaussian noise. The fractional diffusion equation governs the probability density function of the subordinated killed Brownian motion. The tempered fractional Gaussian noise plays the role of fluctuating external source with the property of localization. We first establish the regularity of the infinite dimensional stochastic integration of the tempered fractional Brownian motion and then build the regularity of the mild solution of the fractional stochastic diffusion equation. The spectral Galerkin method is used for space approximation; after that the system is transformed into an equivalent form having better regularity than the original one in time. Then we use the semi-implicit Euler scheme to discretize the time derivative. In terms of the temporal-spatial error splitting technique, we obtain the error estimates of the fully discrete scheme in the sense of mean-squared $L^2$-norm. Extensive numerical experiments confirm the theoretical estimates.

math.NA↗