arXiv subjects

Min Zhang

Publications and source records attributed to Min Zhang.

At least 793 records · Page 44Linked to original sources

arXiv2020

Accelerating Robustness Verification of Deep Neural Networks Guided by Target Labels

Deep Neural Networks (DNNs) have become key components of many safety-critical applications such as autonomous driving and medical diagnosis. However, DNNs have been shown suffering from poor robustness because of their susceptibility to adversarial examples such that small perturbations to an input result in misprediction. Addressing to this concern, various approaches have been proposed to formally verify the robustness of DNNs. Most of these approaches reduce the verification problem to optimization problems of searching an adversarial example for a given input so that it is not correctly classified to the original label. However, they are limited in accuracy and scalability. In this paper, we propose a novel approach that can accelerate the robustness verification techniques by guiding the verification with target labels. The key insight of our approach is that the robustness verification problem of DNNs can be solved by verifying sub-problems of DNNs, one per target label. Fixing the target label during verification can drastically reduce the search space and thus improve the efficiency. We also propose an approach by leveraging symbolic interval propagation and linear relaxation techniques to sort the target labels in terms of chances that adversarial examples exist. This often allows us to quickly falsify the robustness of DNNs and the verification for remaining target labels could be avoided. Our approach is orthogonal to, and can be integrated with, many existing verification techniques. For evaluation purposes, we integrate it with three recent promising DNN verification tools, i.e., MipVerify, DeepZ, and Neurify. Experimental results show that our approach can significantly improve these tools by 36X speedup when the perturbation distance is set in a reasonable range.

cs.LG↗

arXiv2020

High-order conservative positivity-preserving DG-interpolation for deforming meshes and application to moving mesh DG simulation of radiative transfer

Solution interpolation between deforming meshes is an important component for several applications in scientific computing, including indirect arbitrary-Lagrangian-Eulerian and rezoning moving mesh methods in numerical solution of partial differential equations. In this paper, a high-order, conservative, and positivity-preserving interpolation scheme is developed based on the discontinuous Galerkin solution of a linear time-dependent equation on deforming meshes. The scheme works for bounded but otherwise arbitrary mesh deformation from the old mesh to the new one. The cost and positivity preservation (with a linear scaling limiter) of the DG-interpolation are investigated. Numerical examples are presented to demonstrate the properties of the interpolation scheme. The DG-interpolation is applied to the rezoning moving mesh DG solution of the radiative transfer equation, an integro-differential equation modeling the conservation of photons and involving time, space, and angular variables. Numerical results obtained for examples in one and two spatial dimensions with various settings show that the resulting rezoning moving mesh DG method maintains the same convergence order as the standard DG method, is more efficient than the method with a fixed uniform mesh, and is able to preserve the positivity of the radiative intensity.

math.NA↗

arXiv2020

On two Diophantine inequalities over primes (II)

Let $1<c<\frac{26088036}{12301745},c\not=2$ and $N$ be a sufficiently large real number. In this paper, it is proved that, for almost all $R\in (N,2N]$, the Diophantine inequality \begin{equation*} \big|p_1^c+p_2^c+p_3^c-R\big|<\log^{-1}N \end{equation*} is solvable in primes $p_1,p_2,p_3$. Moreover, we also prove that the following Diophantine inequality \begin{equation*} \big|p_1^c+p_2^c+p_3^c+p_4^c+p_5^c+p_6^c-N\big|<\log^{-1}N \end{equation*} is solvable in prime variables $p_1,p_2,p_3,p_4,p_5,p_6$, which improves the previous result $1<c<\frac{37}{18},c\neq2$.

math.NT↗

arXiv2020

RepBERT: Contextualized Text Embeddings for First-Stage Retrieval

Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings. The inner products of query and document embeddings are regarded as relevance scores. On MS MARCO Passage Ranking task, RepBERT achieves state-of-the-art results among all initial retrieval techniques. And its efficiency is comparable to bag-of-words methods.

cs.IR↗

arXiv2020

Train Your Data Processor: Distribution-Aware and Error-Compensation Coordinate Decoding for Human Pose Estimation

Recently, the leading performance of human pose estimation is dominated by heatmap based methods. While being a fundamental component of heatmap processing, heatmap decoding (i.e. transforming heatmaps to coordinates) receives only limited investigations, to our best knowledge. This work fills the gap by studying the heatmap decoding processing with a particular focus on the errors introduced throughout the prediction process. We found that the errors of heatmap based methods are surprisingly significant, which nevertheless was universally ignored before. In view of the discovered importance, we further reveal the intrinsic limitations of the previous widely used heatmap decoding methods and thereout propose a Distribution-Aware and Error-Compensation Coordinate Decoding (DAEC). Serving as a model-agnostic plug-in, DAEC learns its decoding strategy from training data and remarkably improves the performance of a variety of state-of-the-art human pose estimation models with negligible extra computation. Specifically, equipped with DAEC, the SimpleBaseline-ResNet152-256x192 and HRNet-W48-256x192 are significantly improved by 2.6 AP and 2.9 AP achieving 72.6 AP and 75.7 AP on COCO, respectively. Moreover, the HRNet-W32-256x256 and ResNet-152-256x256 frameworks enjoy even more dramatic promotions of 8.4% and 7.8% on MPII with PCKh0.1 metric. Extensive experiments performed on these two common benchmarks, demonstrates that DAEC exceeds its competitors by considerable margins, backing up the rationality and generality of our novel heatmap decoding idea. The project is available at https://github.com/fyang235/DAEC.

cs.CV↗

arXiv2020

Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences

In this paper, we propose a new task of machine translation (MT), which is based on no parallel sentences but can refer to a ground-truth bilingual dictionary. Motivated by the ability of a monolingual speaker learning to translate via looking up the bilingual dictionary, we propose the task to see how much potential an MT system can attain using the bilingual dictionary and large scale monolingual corpora, while is independent on parallel sentences. We propose anchored training (AT) to tackle the task. AT uses the bilingual dictionary to establish anchoring points for closing the gap between source language and target language. Experiments on various language pairs show that our approaches are significantly better than various baselines, including dictionary-based word-by-word translation, dictionary-supervised cross-lingual word embedding transformation, and unsupervised MT. On distant language pairs that are hard for unsupervised MT to perform well, AT performs remarkably better, achieving performances comparable to supervised SMT trained on more than 4M parallel sentences.

cs.CL↗

arXiv2020

Efficient Second-Order TreeCRF for Neural Dependency Parsing

In the deep learning (DL) era, parsing models are extremely simplified with little hurt on performance, thanks to the remarkable capability of multi-layer BiLSTMs in context representation. As the most popular graph-based dependency parser due to its high efficiency and performance, the biaffine parser directly scores single dependencies under the arc-factorization assumption, and adopts a very simple local token-wise cross-entropy training loss. This paper for the first time presents a second-order TreeCRF extension to the biaffine parser. For a long time, the complexity and inefficiency of the inside-outside algorithm hinder the popularity of TreeCRF. To address this issue, we propose an effective way to batchify the inside and Viterbi algorithms for direct large matrix operation on GPUs, and to avoid the complex outside algorithm via efficient back-propagation. Experiments and analysis on 27 datasets from 13 languages clearly show that techniques developed before the DL era, such as structural learning (global TreeCRF loss) and high-order modeling are still useful, and can further boost parsing performance over the state-of-the-art biaffine parser, especially for partially annotated training data. We release our code at https://github.com/yzhangcs/crfpar.

cs.CL↗

arXiv2020

A high-order well-balanced positivity-preserving moving mesh DG method for the shallow water equations with non-flat bottom topography

A rezoning-type adaptive moving mesh discontinuous Galerkin method is proposed for the numerical solution of the shallow water equations with non-flat bottom topography. The well-balance property is crucial to the simulation of perturbation waves over the lake-at-rest steady state such as waves on a lake or tsunami waves in the deep ocean. To ensure the well-balance and positivity-preserving properties, strategies are discussed in the use of slope limiting, positivity-preservation limiting, and data transferring between meshes. Particularly, it is suggested that a DG-interpolation scheme be used for the interpolation of both the flow variables and bottom topography from the old mesh to the new one and after each application of the positivity-preservation limiting on the water depth, a high-order correction be made to the approximation of the bottom topography according to the modifications in the water depth. Moreover, mesh adaptation based on the equilibrium variable and water depth is shown to give more desirable results than that based on the commonly used entropy function. Numerical examples in one and two spatial dimensions are presented to demonstrate the well-balance and positivity-preserving properties of the method and its ability to capture small perturbations of the lake-at-rest steady state.

math.NA↗

arXiv2020

Risk Minimization, Regret Minimization and Progressive Hedging Algorithms

This paper begins with a study on the dual representations of risk and regret measures and their impact on modeling multistage decision making under uncertainty. A relationship between risk envelopes and regret envelopes is established by using the Lagrangian duality theory. Such a relationship opens a door to a decomposition scheme, called progressive hedging, for solving multistage risk minimization and regret minimization problems. In particular, the classical progressive hedging algorithm is modified in order to handle a new class of linkage constraints that arises from reformulations and other applications of risk and regret minimization problems. Numerical results are provided to show the efficiency of the progressive hedging algorithms.

q-fin.MF↗

arXiv2020

Altruism and anxiety: Engagement with online community support initiatives (OCSIs) during Covid-19 lockdown in the UK and Ireland

Given concerns about mental health during periods of Covid-19 lockdown, it important to understand how engagement with online Covid-19 related material can affect mood. In the UK and Ireland, online community support initiatives (OCSIs) have emerged to help people manage their lives. Yet, little is known about how people engaged with these or whether they influenced subsequent mood. We conducted surveys to explore how people in the UK and Ireland engaged with OCSIs, and found that 70% did so to offer support (e.g. to provide company). Those who did so reported feeling significantly calmer afterwards, those who engaged for general concerns (e.g. in response to anti-social behaviour) reported feeling significantly more anxious afterwards, but there was no difference in reported mood for those who engaged for other reasons (e.g. to share experiences or views). Thus, engaging with an OCSI for altruistic purposes might help to make people feel calmer.

cs.CY↗

arXiv2020

PointHop++: A Lightweight Learning Model on Point Sets for 3D Classification

The PointHop method was recently proposed by Zhang et al. for 3D point cloud classification with unsupervised feature extraction. It has an extremely low training complexity while achieving state-of-the-art classification performance. In this work, we improve the PointHop method furthermore in two aspects: 1) reducing its model complexity in terms of the model parameter number and 2) ordering discriminant features automatically based on the cross-entropy criterion. The resulting method is called PointHop++. The first improvement is essential for wearable and mobile computing while the second improvement bridges statistics-based and optimization-based machine learning methodologies. With experiments conducted on the ModelNet40 benchmark dataset, we show that the PointHop++ method performs on par with deep neural network (DNN) solutions and surpasses other unsupervised feature extraction methods.

cs.CV↗

arXiv2020

Fully-Corrective Gradient Boosting with Squared Hinge: Fast Learning Rates and Early Stopping

Boosting is a well-known method for improving the accuracy of weak learners in machine learning. However, its theoretical generalization guarantee is missing in literature. In this paper, we propose an efficient boosting method with theoretical generalization guarantees for binary classification. Three key ingredients of the proposed boosting method are: a) the \textit{fully-corrective greedy} (FCG) update in the boosting procedure, b) a differentiable \textit{squared hinge} (also called \textit{truncated quadratic}) function as the loss function, and c) an efficient alternating direction method of multipliers (ADMM) algorithm for the associated FCG optimization. The used squared hinge loss not only inherits the robustness of the well-known hinge loss for classification with outliers, but also brings some benefits for computational implementation and theoretical justification. Under some sparseness assumption, we derive a fast learning rate of the order ${\cal O}((m/\log m)^{-1/4})$ for the proposed boosting method, which can be further improved to ${\cal O}((m/\log m)^{-1/2})$ if certain additional noise assumption is imposed, where $m$ is the size of sample set. Both derived learning rates are the best ones among the existing generalization results of boosting-type methods for classification. Moreover, an efficient early stopping scheme is provided for the proposed method. A series of toy simulations and real data experiments are conducted to verify the developed theories and demonstrate the effectiveness of the proposed method.

cs.LG↗

arXiv2020

On Waring-Goldbach Problem for Squares, Cubes and Higher Powers

Let $\mathcal{P}_r$ denote an almost-prime with at most $r$ prime factors, counted according to multiplicity. In this paper, we generalize the result of Vaughan for ternary admissible exponent. Moreover, we use the refined admissible exponent to prove that, for $3\leqslant k\leqslant 14$ and for every sufficiently large even integer $n$, the following equation \begin{equation*} n=x^2+p_1^2+p_2^3+p_3^3+p_4^3+p_5^k \end{equation*} is solvable with $x$ being an almost-prime $\mathcal{P}_{r(k)}$ and the other variables primes, where $r(k)$ is defined in Theorem. This result constitutes a deepening upon that of previous results.

math.NT↗

arXiv2020

An Additive Problem over Piatetski-Shapiro Primes and Almost-primes

Let $\mathcal{P}_r$ denote an almost-prime with at most $r$ prime factors, counted according to multiplicity. In this paper, we establish a theorem of Bombieri-Vinogradov type for the Piatetski-Shapiro primes $p=[n^{1/γ}]$ with $\frac{85}{86}<γ<1$. Moreover, we use this result to prove that, for $0.9989445<γ<1$, there exist infinitely many Piatetski-Shapiro primes such that $p+2=\mathcal{P}_3$, which improves the previous results of Lu, Wang and Cai, and Peneva.

math.NT↗

arXiv2020

AE-OT-GAN: Training GANs from data specific latent distribution

Though generative adversarial networks (GANs) areprominent models to generate realistic and crisp images,they often encounter the mode collapse problems and arehard to train, which comes from approximating the intrinsicdiscontinuous distribution transform map with continuousDNNs. The recently proposed AE-OT model addresses thisproblem by explicitly computing the discontinuous distribu-tion transform map through solving a semi-discrete optimaltransport (OT) map in the latent space of the autoencoder.However the generated images are blurry. In this paper, wepropose the AE-OT-GAN model to utilize the advantages ofthe both models: generate high quality images and at thesame time overcome the mode collapse/mixture problems.Specifically, we first faithfully embed the low dimensionalimage manifold into the latent space by training an autoen-coder (AE). Then we compute the optimal transport (OT)map that pushes forward the uniform distribution to the la-tent distribution supported on the latent manifold. Finally,our GAN model is trained to generate high quality imagesfrom the latent distribution, the distribution transform mapfrom which to the empirical data distribution will be con-tinuous. The paired data between the latent code and thereal images gives us further constriction about the generator.Experiments on simple MNIST dataset and complex datasetslike Cifar-10 and CelebA show the efficacy and efficiency ofour proposed method.

cs.CV↗

arXiv2020

Stochastic Item Descent Method for Large Scale Equal Circle Packing Problem

Stochastic gradient descent (SGD) is a powerful method for large-scale optimization problems in the area of machine learning, especially for a finite-sum formulation with numerous variables. In recent years, mini-batch SGD gains great success and has become a standard technique for training deep neural networks fed with big amount of data. Inspired by its success in deep learning, we apply the idea of SGD with batch selection of samples to a classic optimization problem in decision version. Given $n$ unit circles, the equal circle packing problem (ECPP) asks whether there exist a feasible packing that could put all the circles inside a circular container without overlapping. Specifically, we propose a stochastic item descent method (SIDM) for ECPP in large scale, which randomly divides the unit circles into batches and runs Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm on the corresponding batch function iteratively to speedup the calculation. We also increase the batch size during the batch iterations to gain higher quality solution. Comparing to the current best packing algorithms, SIDM greatly speeds up the calculation of optimization process and guarantees the solution quality for large scale instances with up to 1500 circle items, while the baseline algorithms usually handle about 300 circle items. The results indicate the highly efficiency of SIDM for this classic optimization problem in large scale, and show potential for other large scale classic optimization problems in which gradient descent is used for optimization.

math.OC↗

arXiv2019

PointHop: An Explainable Machine Learning Method for Point Cloud Classification

An explainable machine learning method for point cloud classification, called the PointHop method, is proposed in this work. The PointHop method consists of two stages: 1) local-to-global attribute building through iterative one-hop information exchange, and 2) classification and ensembles. In the attribute building stage, we address the problem of unordered point cloud data using a space partitioning procedure and developing a robust descriptor that characterizes the relationship between a point and its one-hop neighbor in a PointHop unit. When we put multiple PointHop units in cascade, the attributes of a point will grow by taking its relationship with one-hop neighbor points into account iteratively. Furthermore, to control the rapid dimension growth of the attribute vector associated with a point, we use the Saab transform to reduce the attribute dimension in each PointHop unit. In the classification and ensemble stage, we feed the feature vector obtained from multiple PointHop units to a classifier. We explore ensemble methods to improve the classification performance furthermore. It is shown by experimental results that the PointHop method offers classification performance that is comparable with state-of-the-art methods while demanding much lower training complexity.

cs.CV↗

arXiv2019

Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation

Transfer learning between different language pairs has shown its effectiveness for Neural Machine Translation (NMT) in low-resource scenario. However, existing transfer methods involving a common target language are far from success in the extreme scenario of zero-shot translation, due to the language space mismatch problem between transferor (the parent model) and transferee (the child model) on the source side. To address this challenge, we propose an effective transfer learning approach based on cross-lingual pre-training. Our key idea is to make all source languages share the same feature space and thus enable a smooth transition for zero-shot translation. To this end, we introduce one monolingual pre-training method and two bilingual pre-training methods to obtain a universal encoder for different languages. Once the universal encoder is constructed, the parent model built on such encoder is trained with large-scale annotated data and then directly applied in zero-shot translation scenario. Experiments on two public datasets show that our approach significantly outperforms strong pivot-based baseline and various multilingual NMT approaches.

cs.CL↗