arXiv Science⌕ Search

arXiv subjects

Jian Li

Publications and source records attributed to Jian Li.

At least 451 records · Page 25Linked to original sources
arXiv2022

FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning

Most previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines (i.e., pretrained models with over one billion parameters). Under this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to generate label-flipped data. Central to the idea of FlipDA is the discovery that generating label-flipped data is more crucial to the performance than generating label-preserved data. Experiments show that FlipDA achieves a good tradeoff between effectiveness and robustness -- it substantially improves many tasks while not negatively affecting the others.

cs.CL↗
arXiv2022

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted much recent attention. However, prior methods have been evaluated under a disparate set of protocols, which hinders fair comparison and measuring progress of the field. To address this issue, we introduce an evaluation framework that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability. Under this new evaluation framework, we re-evaluate several state-of-the-art few-shot methods for NLU tasks. Our framework reveals new insights: (1) both the absolute performance and relative gap of the methods were not accurately estimated in prior literature; (2) no single method dominates most tasks with consistent performance; (3) improvements of some methods diminish with a larger pretrained model; and (4) gains from different methods are often complementary and the best combined model performs close to a strong fully-supervised baseline. We open-source our toolkit, FewNLU, that implements our evaluation framework along with a number of state-of-the-art methods.

cs.CL↗
arXiv2022

Accelerating Serverless Computing by Harvesting Idle Resources

Serverless computing automates fine-grained resource scaling and simplifies the development and deployment of online services with stateless functions. However, it is still non-trivial for users to allocate appropriate resources due to various function types, dependencies, and input sizes. Misconfiguration of resource allocations leaves functions either under-provisioned or over-provisioned and leads to continuous low resource utilization. This paper presents Freyr, a new resource manager (RM) for serverless platforms that maximizes resource efficiency by dynamically harvesting idle resources from over-provisioned functions to under-provisioned functions. Freyr monitors each function's resource utilization in real-time, detects over-provisioning and under-provisioning, and learns to harvest idle resources safely and accelerates functions efficiently by applying deep reinforcement learning algorithms along with a safeguard mechanism. We have implemented and deployed a Freyr prototype in a 13-node Apache OpenWhisk cluster. Experimental results show that 38.8% of function invocations have idle resources harvested by Freyr, and 39.2% of invocations are accelerated by the harvested resources. Freyr reduces the 99th-percentile function response latency by 32.1% compared to the baseline RMs.

cs.DC↗
arXiv2022

Generalized Unrelated Machine Scheduling Problem

We study the generalized load-balancing (GLB) problem, where we are given $n$ jobs, each of which needs to be assigned to one of $m$ unrelated machines with processing times $\{p_{ij}\}$. Under a job assignment $σ$, the load of each machine $i$ is $ψ_i(\mathbf{p}_{i}[σ])$ where $ψ_i:\mathbb{R}^n\rightarrow\mathbb{R}_{\geq0}$ is a symmetric monotone norm and $\mathbf{p}_{i}[σ]$ is the $n$-dimensional vector $\{p_{ij}\cdot \mathbf{1}[σ(j)=i]\}_{j\in [n]}$. Our goal is to minimize the generalized makespan $ϕ(\mathsf{load}(σ))$, where $ϕ:\mathbb{R}^m\rightarrow\mathbb{R}_{\geq0}$ is another symmetric monotone norm and $\mathsf{load}(σ)$ is the $m$-dimensional machine load vector. This problem significantly generalizes many classic optimization problems, e.g., makespan minimization, set cover, minimum-norm load-balancing, etc. We obtain a polynomial time randomized algorithm that achieves an approximation factor of $O(\log n)$, matching the lower bound of set cover up to constant factor. We achieve this by rounding a novel configuration LP relaxation with exponential number of variables. To approximately solve the configuration LP, we design an approximate separation oracle for its dual program. In particular, the separation oracle can be reduced to the norm minimization with a linear constraint (NormLin) problem and we devise a polynomial time approximation scheme (PTAS) for it, which may be of independent interest.

cs.DS↗
arXiv2022

ASFD: Automatic and Scalable Face Detector

Along with current multi-scale based detectors, Feature Aggregation and Enhancement (FAE) modules have shown superior performance gains for cutting-edge object detection. However, these hand-crafted FAE modules show inconsistent improvements on face detection, which is mainly due to the significant distribution difference between its training and applying corpus, COCO vs. WIDER Face. To tackle this problem, we essentially analyse the effect of data distribution, and consequently propose to search an effective FAE architecture, termed AutoFAE by a differentiable architecture search, which outperforms all existing FAE modules in face detection with a considerable margin. Upon the found AutoFAE and existing backbones, a supernet is further built and trained, which automatically obtains a family of detectors under the different complexity constraints. Extensive experiments conducted on popular benchmarks, WIDER Face and FDDB, demonstrate the state-of-the-art performance-efficiency trade-off for the proposed automatic and scalable face detector (ASFD) family. In particular, our strong ASFD-D6 outperforms the best competitor with AP 96.7/96.2/92.1 on WIDER Face test, and the lightweight ASFD-D0 costs about 3.1 ms, more than 320 FPS, on the V100 GPU with VGA-resolution images.

cs.CV↗
arXiv2022

On $n$-tuplewise IP-sensitivity and thick sensitivity

Let $(X,T)$ be a topological dynamical system and $n\geq 2$. We say that $(X,T)$ is $n$-tuplewise IP-sensitive (resp. $n$-tuplewise thickly sensitive) if there exists a constant $δ>0$ with the property that for each non-empty open subset $U$ of $X$, there exist $x_1,x_2,\dotsc,x_n\in U$ such that \[ \Bigl\{k\in\mathbb{N}\colon \min_{1\le i δ\Bigr\} \] is an IP-set (resp. a thick set). We obtain several sufficient and necessary conditions of a dynamical system to be $n$-tuplewise IP-sensitive or $n$-tuplewise thickly sensitive and show that any non-trivial weakly mixing system is $n$-tuplewise IP-sensitive for all $n\geq 2$, while it is $n$-tuplewise thickly sensitive if and only if it has at least $n$ minimal points. We characterize two kinds of sensitivity by considering some kind of factor maps. We introduce the opposite side of pairwise IP-sensitivity and pairwise thick sensitivity, named (almost) pairwise IP$^*$-equicontinuity and (almost) pairwise syndetic equicontinuity, and obtain dichotomies results for them. In particular, we show that a minimal system is distal if and only if it is pairwise IP$^*$-equicontinuous. We show that every minimal system admits a maximal almost pairwise IP$^*$-equicontinuous factor and admits a maximal pairwise syndetic equicontinuous factor, and characterize them by the factor maps to their maximal distal factors.

math.DS↗
arXiv2022

Machine learning prediction for mean motion resonance behaviour -- The planar case

Most recently, machine learning has been used to study the dynamics of integrable Hamiltonian systems and the chaotic 3-body problem. In this work, we consider an intermediate case of regular motion in a non-integrable system: the behaviour of objects in the 2:3 mean motion resonance with Neptune. We show that, given initial data from a short 6250 yr numerical integration, the best-trained artificial neural network (ANN) can predict the trajectories of the 2:3 resonators over the subsequent 18750 yr evolution, covering a full libration cycle over the combined time period. By comparing our ANN's prediction of the resonant angle to the outcome of numerical integrations, the former can predict the resonant angle with an accuracy as small as of a few degrees only, while it has the advantage of considerably saving computational time. More specifically, the trained ANN can effectively measure the resonant amplitudes of the 2:3 resonators, and thus provides a fast approach that can identify the resonant candidates. This may be helpful in classifying a huge population of KBOs to be discovered in future surveys.

astro-ph.EP↗
arXiv2022

SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution

In the practical application of restoring low-resolution gray-scale images, we generally need to run three separate processes of image colorization, super-resolution, and dows-sampling operation for the target device. However, this pipeline is redundant and inefficient for the independent processes, and some inner features could have been shared. Therefore, we present an efficient paradigm to perform {S}imultaneously Image {C}olorization and {S}uper-resolution (SCS) and propose an end-to-end SCSNet to achieve this goal. The proposed method consists of two parts: colorization branch for learning color information that employs the proposed plug-and-play \emph{Pyramid Valve Cross Attention} (PVCAttn) module to aggregate feature maps between source and reference images; and super-resolution branch for integrating color and texture information to predict target images, which uses the designed \emph{Continuous Pixel Mapping} (CPM) module to predict high-resolution images at continuous magnification. Furthermore, our SCSNet supports both automatic and referential modes that is more flexible for practical application. Abundant experiments demonstrate the superiority of our method for generating authentic images over state-of-the-art methods, e.g., averagely decreasing FID by 1.8$\downarrow$ and 5.1 $\downarrow$ compared with current best scores for automatic and referential modes, respectively, while owning fewer parameters (more than $\times$2$\downarrow$) and faster running speed (more than $\times$3$\uparrow$).

cs.CV↗
arXiv2022

CDNNs: The coupled deep neural networks for coupling of the Stokes and Darcy-Forchheimer problems

In this article, we present an efficient deep learning method called coupled deep neural networks (CDNNs) for coupled physical problems. Our method compiles the interface conditions of the coupled PDEs into the networks properly and can be served as an efficient alternative to the complex coupled problems. To impose energy conservation constraints, the CDNNs utilize simple fully connected layers and a custom loss function to perform the model training process as well as the physical property of the exact solution. The approach can be beneficial for the following reasons: Firstly, we sampled randomly and only input spatial coordinates without being restricted by the nature of samples. Secondly, our method is meshfree which makes it more efficient than the traditional methods. Finally, our method is parallel and can solve multiple variables independently at the same time. We give the theory to guarantee the convergence of the loss function and the convergence of the neural networks to the exact solution. Some numerical experiments are performed and discussed to demonstrate the performance of the proposed method.

math.NA↗
arXiv2021

Scalable algorithm simplification using quantum AND logic

Implementing quantum algorithms on realistic hardware requires translating high-level global operations into sequences of native elementary gates, a process known as quantum compiling. Physical limitations, such as constraints in connectivity and gate alphabets, often result in unacceptable implementation costs. To enable successful near-term applications, it is crucial to optimize compilation by exploiting the potential capabilities of existing hardware. Here, we implement a resource-efficient construction for a quantum version of AND logic that can reduce the cost, enabling the execution of key quantum circuits. On a high-scalability superconducting quantum processor, we demonstrate low-depth synthesis of high-fidelity generalized Toffoli gates with up to 8 qubits and Grover's search algorithm in a search space of up to 64 entries; both are the largest such implementations in scale to date. Our experimental demonstration illustrates a scalable implementation of simplifying quantum algorithms, paving the way for larger, more meaningful quantum applications on noisy devices.

quant-ph↗
arXiv2021

Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model

Inspired by biological evolution, we explain the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derive that both of them have consistent mathematical representation. Analogous to the dynamic local population in EA, we improve the existing transformer structure and propose a more efficient EAT model, and design task-related heads to deal with different tasks more flexibly. Moreover, we introduce the spatial-filling curve into the current vision transformer to sequence image data into a uniform sequential format. Thus we can design a unified EAT framework to address multi-modal tasks, separating the network architecture from the data format adaptation. Our approach achieves state-of-the-art results on the ImageNet classification task compared with recent vision transformer works while having smaller parameters and greater throughput. We further conduct multi-modal tasks to demonstrate the superiority of the unified EAT, e.g., Text-Based Image Retrieval, and our approach improves the rank-1 by +3.7 points over the baseline on the CSS dataset.

cs.CV↗
arXiv2021

Simultaneous Monitoring of a Large Number of Heterogeneous Categorical Data Streams

This article proposes a powerful scheme to monitor a large number of categorical data streams with heterogeneous parameters or nature. The data streams considered may be either nominal with a number of attribute levels or ordinal with some natural order among their attribute levels, such as good, marginal, and bad. For an ordinal data stream, it is assumed that there is a corresponding latent continuous data stream determining it. Furthermore, different data streams may have different number of attribute levels and different values of level probabilities. Due to high dimensionality, traditional multivariate categorical control charts cannot be applied. Here we integrate the local exponentially weighted likelihood ratio test statistics from each single stream, regardless of nominal or ordinal, into a powerful goodness-of-fit test by some normalization procedure. A global monitoring statistic is proposed ultimately. Simulation results have demonstrated the robustness and efficiency of our method.

stat.ME↗
arXiv2021

Superrobust Geometric Control of a Superconducting Circuit

Geometric phases accompanying adiabatic quantum evolutions can be used to construct robust quantum control for quantum information processing due to their noise-resilient feature. A significant development along this line is to construct geometric gates using nonadiabatic quantum evolutions to reduce errors due to decoherence. However, it has been shown that nonadiabatic geometric gates are not necessarily more robust than dynamical ones, in contrast to an intuitive expectation. Here we experimentally investigate this issue for the case of nonadiabatic holonomic quantum computation~(NHQC) and show that conventional NHQC schemes cannot guarantee the expected robustness due to a cross coupling to the states outside the computational space. We implement a different set of constraints for gate construction in order to suppress such cross coupling to achieve an enhanced robustness. Using a superconducting quantum circuit, we demonstrate high-fidelity holonomic gates whose infidelity against quasi-static transverse errors can be suppressed up to the fourth order, instead of the second order in conventional NHQC and dynamical gates. In addition, we explicitly measure the accumulated dynamical phase due to the above mentioned cross coupling and verify that it is indeed much reduced in our NHQC scheme. We further demonstrate a protocol for constructing two-qubit NHQC gates also with an enhanced robustness.

quant-ph↗
arXiv2021

Establishing a non-hydrostatic global atmospheric modeling system (iAMAS) at 3-km horizontal resolution with online integrated aerosol feedbacks on the Sunway supercomputer of China

During the era of global warming and highly urbanized development, extreme and high impact weather as well as air pollution incidents influence everyday life and might even cause the incalculable loss of life and property. Although with the vast development of numerical simulation of atmosphere, there still exists substantial forecast biases objectively. To predict extreme weather, severe air pollution, and abrupt climate change accurately, the numerical atmospheric model requires not only to simulate meteorology and atmospheric compositions and their impacts simultaneously involving many sophisticated physical and chemical processes but also at high spatiotemporal resolution. Global atmospheric simulation of meteorology and atmospheric compositions simultaneously at spatial resolutions of a few kilometers remains challenging due to its intensive computational and input/output (I/O) requirement. Through multi-dimension-parallelism structuring, aggressive and finer-grained optimizing, manual vectorizing, and parallelized I/O fragmenting, an integrated Atmospheric Model Across Scales (iAMAS) was established on the new Sunway supercomputer platform to significantly increase the computational efficiency and reduce the I/O cost. The global 3-km atmospheric simulation for meteorology with online integrated aerosol feedbacks with iAMAS was scaled to 39,000,000 processor cores and achieved the speed of 0.82 simulation day per hour (SDPH) with routine I/O, which enables us to perform 5-day global weather forecast at 3-km horizontal resolution with online natural aerosol impacts. The results demonstrate the promising future that the increasing of spatial resolution to a few kilometers with online integrated aerosol impacts may significantly improve the global weather forecast.

physics.ao-ph↗
arXiv2021

AutoHEnsGNN: Winning Solution to AutoGraph Challenge for KDD Cup 2020

Graph Neural Networks (GNNs) have become increasingly popular and achieved impressive results in many graph-based applications. However, extensive manual work and domain knowledge are required to design effective architectures, and the results of GNN models have high variance with different training setups, which limits the application of existing GNN models. In this paper, we present AutoHEnsGNN, a framework to build effective and robust models for graph tasks without any human intervention. AutoHEnsGNN won first place in the AutoGraph Challenge for KDD Cup 2020, and achieved the best rank score of five real-life datasets in the final phase. Given a task, AutoHEnsGNN first applies a fast proxy evaluation to automatically select a pool of promising GNN models. Then it builds a hierarchical ensemble framework: 1) We propose graph self-ensemble (GSE), which can reduce the variance of weight initialization and efficiently exploit the information of local and global neighborhoods; 2) Based on GSE, a weighted ensemble of different types of GNN models is used to effectively learn more discriminative node representations. To efficiently search the architectures and ensemble weights, we propose AutoHEnsGNN$_{\text{Gradient}}$, which treats the architectures and ensemble weights as architecture parameters and uses gradient-based architecture search to obtain optimal configurations, and AutoHEnsGNN$_{\text{Adaptive}}$, which can adaptively adjust the ensemble weight based on the model accuracy. Extensive experiments on node classification, graph classification, edge prediction and KDD Cup challenge demonstrate the effectiveness and generality of AutoHEnsGNN

cs.LG↗
arXiv2021

ML-EXray: Visibility into ML Deployment on the Edge

Benefiting from expanding cloud infrastructure, deep neural networks (DNNs) today have increasingly high performance when trained in the cloud. Researchers spend months of effort competing for an extra few percentage points of model accuracy. However, when these models are actually deployed on edge devices in practice, very often, the performance can abruptly drop over 10% without obvious reasons. The key challenge is that there is not much visibility into ML inference execution on edge devices, and very little awareness of potential issues during the edge deployment process. We present ML-EXray, an end-to-end framework, which provides visibility into layer-level details of the ML execution, and helps developers analyze and debug cloud-to-edge deployment issues. More often than not, the reason for sub-optimal edge performance does not only lie in the model itself, but every operation throughout the data flow and the deployment process. Evaluations show that ML-EXray can effectively catch deployment issues, such as pre-processing bugs, quantization issues, suboptimal kernels, etc. Using ML-EXray, users need to write less than 15 lines of code to fully examine the edge deployment pipeline. Eradicating these issues, ML-EXray can correct model performance by up to 30%, pinpoint error-prone layers, and guide users to optimize kernel execution latency by two orders of magnitude. Code and APIs will be released as an open-source multi-lingual instrumentation library and a Python deployment validation library.

cs.DC↗
arXiv2021

Nonlinear Inclusion Theory with Application to the Growth and Morphogenesis of a Confined Body

One of the most celebrated contributions to the study of the mechanical behavior of materials is due to J.D. Eshelby, who in the late 50s revolutionized our understanding of the elastic stress and strain fields due to an ellipsoidal inclusion/inhomogeneity that undergoes a transformation of shape and size. While Eshelby's work laid the foundation for significant advancements in various fields, including fracture mechanics, theory of phase transitions, and homogenization methods, its extension into the range of large deformations, and to situations in which the material can actively reorganize in response to the finite transformation strain, is in a nascent state. Beyond the theoretical difficulties imposed by highly nonlinear material response, a major hindrance has been the absence of experimental observations that can elucidate the intricacies that arise in this regime. To address this limitation, our experimental observations reveal the key morphogenesis steps of Vibrio cholerae biofilms embedded in hydrogels, as they grow by four orders of magnitude from their initial size. Using the biofilm growth as a case study, our theoretical model considers various growth scenarios and employs two different and complimentary methods -- a minimal analytical model and finite element computations -- to obtain approximate equilibrium solutions. A particular emphasis is put on determining the natural growth path of an inclusion that optimizes its shape in response to the confinement, and the onset of damage in the matrix, which together explain the observed behavior of biofilms. Beyond bacterial biofilms, this work sheds light on the role of mechanics in determining the morphogenesis pathways of confined growing bodies and thus applies to a broad range of phenomena that are ubiquitous in both natural and engineered material systems.

cond-mat.soft↗