arXiv subjects

Lei Yang

Publications and source records attributed to Lei Yang.

At least 397 records · Page 22Linked to original sources

arXiv2021

Can Noise on Qubits Be Learned in Quantum Neural Network? A Case Study on QuantumFlow

In the noisy intermediate-scale quantum (NISQ) era, one of the key questions is how to deal with the high noise level existing in physical quantum bits (qubits). Quantum error correction is promising but requires an extensive number (e.g., over 1,000) of physical qubits to create one "perfect" qubit, exceeding the capacity of the existing quantum computers. This paper aims to tackle the noise issue from another angle: instead of creating perfect qubits for general quantum algorithms, we investigate the potential to mitigate the noise issue for dedicate algorithms. Specifically, this paper targets quantum neural network (QNN), and proposes to learn the errors in the training phase, so that the identified QNN model can be resilient to noise. As a result, the implementation of QNN needs no or a small number of additional physical qubits, which is more realistic for the near-term quantum computers. To achieve this goal, an application-specific compiler is essential: on the one hand, the error cannot be learned if the mapping from logical qubits to physical qubits exists randomness; on the other hand, the compiler needs to be efficient so that the lengthy training procedure can be completed in a reasonable time. In this paper, we utilize the recent QNN framework, QuantumFlow, as a case study. Experimental results show that the proposed approach can optimize QNN models for different errors in qubits, achieving up to 28% accuracy improvement compared with the model obtained by the error-agnostic training.

quant-ph↗

arXiv2021

Robust Event Classification Using Imperfect Real-world PMU Data

This paper studies robust event classification using imperfect real-world phasor measurement unit (PMU) data. By analyzing the real-world PMU data, we find it is challenging to directly use this dataset for event classifiers due to the low data quality observed in PMU measurements and event logs. To address these challenges, we develop a novel machine learning framework for training robust event classifiers, which consists of three main steps: data preprocessing, fine-grained event data extraction, and feature engineering. Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e.g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers. Based on the proposed framework, we develop a workflow for event classification using the real-world PMU data streaming into the system in real-time. Using the proposed framework, robust event classifiers can be efficiently trained based on many off-the-shelf lightweight machine learning models. Numerical experiments using the real-world dataset from the Western Interconnection of the U.S power transmission grid show that the event classifiers trained under the proposed framework can achieve high classification accuracy while being robust against low-quality data.

cs.LG↗

arXiv2021

Detecting Gender Bias in Transformer-based Models: A Case Study on BERT

In this paper, we propose a novel gender bias detection method by utilizing attention map for transformer-based models. We 1) give an intuitive gender bias judgement method by comparing the different relation degree between the genders and the occupation according to the attention scores, 2) design a gender bias detector by modifying the attention module, 3) insert the gender bias detector into different positions of the model to present the internal gender bias flow, and 4) draw the consistent gender bias conclusion by scanning the entire Wikipedia, a BERT pretraining dataset. We observe that 1) the attention matrices, Wq and Wk introduce much more gender bias than other modules (including the embedding layer) and 2) the bias degree changes periodically inside of the model (attention matrix Q, K, V, and the remaining part of the attention layer (including the fully-connected layer, the residual connection, and the layer normalization module) enhance the gender bias while the averaged attentions reduces the bias).

cs.CL↗

arXiv2021

DispersedLedger: High-Throughput Byzantine Consensus on Variable Bandwidth Networks

The success of blockchains has sparked interest in large-scale deployments of Byzantine fault tolerant (BFT) consensus protocols over wide area networks. A central feature of such networks is variable communication bandwidth across nodes and across time. We present DispersedLedger, an asynchronous BFT protocol that provides near-optimal throughput in the presence of such variable network bandwidth. The core idea of DispersedLedger is to enable nodes to propose, order, and agree on blocks of transactions without having to download their full content. By enabling nodes to agree on an ordered log of blocks, with a guarantee that each block is available within the network and unmalleable, DispersedLedger decouples bandwidth-intensive block downloads at different nodes, allowing each to make progress at its own pace. We build a full system prototype and evaluate it on real-world and emulated networks. Our results on a geo-distributed wide-area deployment across the Internet shows that DispersedLedger achieves 2x better throughput and 74% reduction in latency compared to HoneyBadger, the state-of-the-art asynchronous protocol.

cs.NI↗

arXiv2021

Sparse Solutions of a Class of Constrained Optimization Problems

In this paper, we consider a well-known sparse optimization problem that aims to find a sparse solution of a possibly noisy underdetermined system of linear equations. Mathematically, it can be modeled in a unified manner by minimizing $\|\bf{x}\|_p^p$ subject to $\|A\bf{x}-\bf{b}\|_q\leqσ$ for given $A \in \mathbb{R}^{m \times n}$, $\bf{b}\in\mathbb{R}^m$, $σ\geq0$, $0\leq p\leq 1$ and $q \geq 1$. We then study various properties of the optimal solutions of this problem. Specifically, without any condition on the matrix $A$, we provide upper bounds in cardinality and infinity norm for the optimal solutions, and show that all optimal solutions must be on the boundary of the feasible set when $0<p<1$. Moreover, for $q \in \{1,\infty\}$, we show that the problem with $0<p<1$ has a finite number of optimal solutions and prove that there exists $0<p^*<1$ such that the solution set of the problem with any $0<p<p^*$ is contained in the solution set of the problem with $p=0$ and there further exists $0<\bar{p}<p^*$ such that the solution set of the problem with any $0<p\leq\bar{p}$ remains unchanged. An estimation of such $p^*$ is also provided. In addition, to solve the constrained nonconvex non-Lipschitz $L_p$-$L_1$ problem ($0<p<1$ and $q=1$), we propose a smoothing penalty method and show that, under some mild conditions, any cluster point of the sequence generated is a KKT point of our problem. Some numerical examples are given to implicitly illustrate the theoretical results and show the efficiency of the proposed algorithm for the constrained $L_p$-$L_1$ problem under different noises.

math.OC↗

arXiv2021

iShape: A First Step Towards Irregular Shape Instance Segmentation

In this paper, we introduce a brand new dataset to promote the study of instance segmentation for objects with irregular shapes. Our key observation is that though irregularly shaped objects widely exist in daily life and industrial scenarios, they received little attention in the instance segmentation field due to the lack of corresponding datasets. To fill this gap, we propose iShape, an irregular shape dataset for instance segmentation. iShape contains six sub-datasets with one real and five synthetics, each represents a scene of a typical irregular shape. Unlike most existing instance segmentation datasets of regular objects, iShape has many characteristics that challenge existing instance segmentation algorithms, such as large overlaps between bounding boxes of instances, extreme aspect ratios, and large numbers of connected components per instance. We benchmark popular instance segmentation methods on iShape and find their performance drop dramatically. Hence, we propose an affinity-based instance segmentation algorithm, called ASIS, as a stronger baseline. ASIS explicitly combines perception and reasoning to solve Arbitrary Shape Instance Segmentation including irregular objects. Experimental results show that ASIS outperforms the state-of-the-art on iShape. Dataset and code are available at https://ishape.github.io

cs.CV↗

arXiv2021

Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems

In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To solve this class of problems, we propose a proximal gradient method with extrapolation and line search (PGels). This method is developed based on a special potential function and successfully incorporates both extrapolation and non-monotone line search, which are two simple and efficient accelerating techniques for the proximal gradient method. Thanks to the line search, this method allows more flexibilities in choosing the extrapolation parameters and updates them adaptively at each iteration if a certain line search criterion is not satisfied. Moreover, with proper choices of parameters, our PGels reduces to many existing algorithms. We also show that, under some mild conditions, our line search criterion is well defined and any cluster point of the sequence generated by PGels is a stationary point of our problem. In addition, by assuming the Kurdyka-$Ł$ojasiewicz exponent of the objective in our problem, we further analyze the local convergence rate of two special cases of PGels, including the widely used non-monotone proximal gradient method as one case. Finally, we conduct some numerical experiments for solving the $\ell_1$ regularized logistic regression problem and the $\ell_{1\text{-}2}$ regularized least squares problem. Our numerical results illustrate the efficiency of PGels and show the potential advantage of combining two accelerating techniques.

math.OC↗

arXiv2021

Capillary interactions, aggregate formation and the rheology of particle-laden flows: a lattice Boltzmann study

The agglomeration of particles caused by the formation of capillary bridges has a decisive impact on the transport properties of a variety of at a first sight very different systems such as capillary suspensions, fluidized beds in chemical reactors, or even sand castles. Here, we study the connection between the microstructure of the agglomerates and the rheology of fluidized suspensions using a coupled lattice Boltzmann and discrete element method approach. We address the influence of the shear rate, the secondary fluid surface tension, and the suspending liquid viscosity. The presence of capillary interactions promotes the formation of either filaments or globular clusters, leading to an increased suspension viscosity. Unexpectedly, filaments have the opposite effect on the viscosity as compared to globular clusters, decreasing the suspension viscosity at larger capillary interaction strengths. In addition, we show that the suspending fluid viscosity also has a non-trivial influence on the effective viscosity of the suspension, a fact usually not taken into account by empirical models.

physics.flu-dyn↗

arXiv2021

Dislocation induced energy dissipation in a tunable trilayer graphene resonator

In crystalline materials, the creation and modulation of dislocations are often associated with plastic deformation and energy dissipation. Here we report a study on the energy dissipation of a trilayer graphene ribbon resonator. The vibration of the ribbon generates cyclic mechanical loading to the graphene ribbon, during which mechanical energy is dissipated as heat. Measuring the quality factor of the graphene resonator provides a way to evaluate the energy dissipation. The graphene ribbon is integrated with silicon micro actuators, allowing its in-plane tension to be finely tuned. As we gradually increased the tension, we observed, in addition to the well-known resonance frequency increase, a large change in the energy dissipation. We propose that the dominating energy dissipation mechanism shifts over three regions. With small applied tension, the graphene is in elastic region, and the major energy dissipation is through graphene edge folding; as the tension increases, dislocations start to develop in the sample to gradually dominate the energy dissipation; finally, at large enough tension, graphene layers become decoupled and start to slide and cause friction, which induces the more severe energy dissipation. The generation and modulation of dislocations are modeled by molecular dynamics calculation and a method to count the energy loss is proposed and compared to the experiment.

cond-mat.mes-hall↗

arXiv2021

Distributed Attention for Grounded Image Captioning

We study the problem of weakly supervised grounded image captioning. That is, given an image, the goal is to automatically generate a sentence describing the context of the image with each noun word grounded to the corresponding region in the image. This task is challenging due to the lack of explicit fine-grained region word alignments as supervision. Previous weakly supervised methods mainly explore various kinds of regularization schemes to improve attention accuracy. However, their performances are still far from the fully supervised ones. One main issue that has been ignored is that the attention for generating visually groundable words may only focus on the most discriminate parts and can not cover the whole object. To this end, we propose a simple yet effective method to alleviate the issue, termed as partial grounding problem in our paper. Specifically, we design a distributed attention mechanism to enforce the network to aggregate information from multiple spatially different regions with consistent semantics while generating the words. Therefore, the union of the focused region proposals should form a visual region that encloses the object of interest completely. Extensive experiments have demonstrated the superiority of our proposed method compared with the state-of-the-arts.

cs.CV↗

arXiv2021

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

Various deep learning techniques have been proposed to solve the single-view 2D-to-3D pose estimation problem. While the average prediction accuracy has been improved significantly over the years, the performance on hard poses with depth ambiguity, self-occlusion, and complex or rare poses is still far from satisfactory. In this work, we target these hard poses and present a novel skeletal GNN learning solution. To be specific, we propose a hop-aware hierarchical channel-squeezing fusion layer to effectively extract relevant information from neighboring nodes while suppressing undesired noises in GNN learning. In addition, we propose a temporal-aware dynamic graph construction procedure that is robust and effective for 3D pose estimation. Experimental results on the Human3.6M dataset show that our solution achieves 10.3\% average prediction accuracy improvement and greatly improves on hard poses over state-of-the-art techniques. We further apply the proposed technique on the skeleton-based action recognition task and also achieve state-of-the-art performance. Our code is available at https://github.com/ailingzengzzz/Skeletal-GNN.

cs.CV↗

arXiv2021

Geometry Uncertainty Projection Network for Monocular 3D Object Detection

Geometry Projection is a powerful depth estimation method in monocular 3D object detection. It estimates depth dependent on heights, which introduces mathematical priors into the deep model. But projection process also introduces the error amplification problem, in which the error of the estimated height will be amplified and reflected greatly at the output depth. This property leads to uncontrollable depth inferences and also damages the training efficiency. In this paper, we propose a Geometry Uncertainty Projection Network (GUP Net) to tackle the error amplification problem at both inference and training stages. Specifically, a GUP module is proposed to obtains the geometry-guided uncertainty of the inferred depth, which not only provides high reliable confidence for each depth but also benefits depth learning. Furthermore, at the training stage, we propose a Hierarchical Task Learning strategy to reduce the instability caused by error amplification. This learning algorithm monitors the learning situation of each task by a proposed indicator and adaptively assigns the proper loss weights for different tasks according to their pre-tasks situation. Based on that, each task starts learning only when its pre-tasks are learned well, which can significantly improve the stability and efficiency of the training process. Extensive experiments demonstrate the effectiveness of the proposed method. The overall model can infer more reliable object depth than existing methods and outperforms the state-of-the-art image-based monocular 3D detectors by 3.74% and 4.7% AP40 of the car and pedestrian categories on the KITTI benchmark.

cs.CV↗

arXiv2021

A second-order semi-implicit method for the inertial Landau-Lifshitz-Gilbert equation

Recent theoretical and experimental advances show that the inertia of magnetization emerges at sub-picoseconds and contributes to the ultrafast magnetization dynamics which cannot be captured intrinsically by the LLG equation. Therefore, as a generalization, the inertial Landau-Lifshitz-Gilbert (iLLG) equation is proposed to model the ultrafast magnetization dynamics. Mathematically, the LLG equation is a nonlinear system of parabolic type with (possible) degeneracy. However, the iLLG equation is a nonlinear system of mixed hyperbolic-parabolic type with degeneracy, and exhibits more complicated structures. It behaves like a hyperbolic system at the sub-picosecond scale while behaves like a parabolic system at larger timescales. Such hybrid behaviors impose additional difficulties on designing numerical methods for the iLLG equation. In this work, we propose a second-order semi-implicit scheme to solve the iLLG equation. The second temporal derivative of magnetization is approximated by the standard centered difference scheme and the first derivative is approximated by the midpoint scheme involving three time steps. The nonlinear terms are treated semi-implicitly using one-sided interpolation with the second-order accuracy. At each step, the unconditionally unique solvability of the unsymmetric linear system of equations in the proposed method is proved with a detailed discussion on the condition number. Numerically, the second-order accuracy in both time and space is verified. Using the proposed method, the inertial effect of ferromagnetics is observed in micromagnetics simulations at small timescales, in consistency with the hyperbolic property of the model at sub-picoseconds. For long time simulations, the results of the iLLG model are in nice agreements with those of the LLG model, in consistency with the parabolic feature of the iLLG model at larger timescales.

math.NA↗

arXiv2021

The Automatic Calibration Unit in JUNO

This paper describes the design and construction of the automatic calibration unit (ACU) for the JUNO experiment. The ACU is a fully automated mechanical system. It is capable of deploying multiple radioactive sources, an ultraviolet (UV) laser source, or an auxiliary sensor such as a temperature sensor, one at a time, into the central detector of JUNO along the central axis. It is designed as a primary tool to precisely calibrate the energy scale of detector, aligning timing for the photosensors, and partially monitoring the position-dependent energy scale variations.

physics.ins-det↗

arXiv2021

Capillary-bridge Forces Between Solid Particles: Insights from Lattice Boltzmann Simulations

Liquid capillary-bridge formation between solid particles has a critical influence on the rheological properties of granular materials and, in particular, on the efficiency of fluidized bed reactors. The available analytical and semi-analytical methods have inherent limitations, and often do not cover important aspects, like the presence of non-axisymmetric bridges. Here, we conduct numerical simulations of the capillary bridge formation between equally and unequally-sized solid particles using the lattice Boltzmann method, and provide an assessment of the accuracy of different families of analytical models. We find that some of the models taken into account are shown to perform better than others. However, all of them fail to predict the capillary force for contact angles larger than $π/2$, where a repulsive capillary force attempts to push the solid particle outwards to minimize the surface energy, especially at a small separation distance.

cond-mat.soft↗

arXiv2021

Lite-FPN for Keypoint-based Monocular 3D Object Detection

3D object detection with a single image is an essential and challenging task for autonomous driving. Recently, keypoint-based monocular 3D object detection has made tremendous progress and achieved great speed-accuracy trade-off. However, there still exists a huge gap with LIDAR-based methods in terms of accuracy. To improve their performance without sacrificing efficiency, we propose a sort of lightweight feature pyramid network called Lite-FPN to achieve multi-scale feature fusion in an effective and efficient way, which can boost the multi-scale detection capability of keypoint-based detectors. Besides, the misalignment between classification score and localization precision is further relieved by introducing a novel regression loss named attention loss. With the proposed loss, predictions with high confidence but poor localization are treated with more attention during the training phase. Comparative experiments based on several state-of-the-art keypoint-based detectors on the KITTI dataset show that our proposed methods manage to achieve significant improvements in both accuracy and frame rate. The code and pretrained models will be released at \url{https://github.com/yanglei18/Lite-FPN}.

cs.CV↗

arXiv2021

IPS300+: a Challenging Multimodal Dataset for Intersection Perception System

Due to the high complexity and occlusion, insufficient perception in the crowded urban intersection can be a serious safety risk for both human drivers and autonomous algorithms, whereas CVIS (Cooperative Vehicle Infrastructure System) is a proposed solution for full-participants perception in this scenario. However, the research on roadside multimodal perception is still in its infancy, and there is no open-source dataset for such scenario. Accordingly, this paper fills the gap. Through an IPS (Intersection Perception System) installed at the diagonal of the intersection, this paper proposes a high-quality multimodal dataset for the intersection perception task. The center of the experimental intersection covers an area of 3000m2, and the extended distance reaches 300m, which is typical for CVIS. The first batch of open-source data includes 14198 frames, and each frame has an average of 319.84 labels, which is 9.6 times larger than the most crowded dataset (H3D dataset in 2019) by now. In order to facilitate further study, this dataset tries to keep the label documents consistent with the KITTI dataset, and a standardized benchmark is created for algorithm evaluation. Our dataset is available at: http://www.openmpd.com/column/other_datasets.

cs.CV↗

arXiv2021

Securing Parallel-chain Protocols under Variable Mining Power

Several emerging PoW blockchain protocols rely on a "parallel-chain" architecture for scaling, where instead of a single chain, multiple chains are run in parallel and aggregated. A key requirement of practical PoW blockchains is to adapt to mining power variations over time. In this paper, we consider the design of provably secure parallel-chain protocols which can adapt to such mining power variations. The Bitcoin difficulty adjustment rule adjusts the difficulty target of block mining periodically to get a constant mean inter-block time. While superficially simple, the rule has proved itself to be sophisticated and successfully secure, both in practice and in theory. We show that natural adaptations of the Bitcoin adjustment rule to the parallel-chain case open the door to subtle, but catastrophic safety and liveness breaches. We uncover a meta-design principle that allow us to design variable mining difficulty protocols for three popular PoW blockchain proposals (Prism, OHIE, and Fruitchains) inside a common rubric. The principle has three components:(M1) a pivot chain, based on which blocks in all chains choose difficulty, (M2) a monotonicity condition for referencing pivot chain blocks and (M3) translating additional protocol aspects from using levels (depth) to using "difficulty levels". We show that protocols employing a subset of these principles may have catastrophic failures. The security of the designs is also proved using a common rubric - the key technical challenge involves analyzing the interaction between the pivot chain and the other chains, as well as bounding the sudden changes in difficulty target experienced in non-pivot chains. We empirically investigate the responsivity of the new mining difficulty rule via simulations based on historical Bitcoin data, and find that the protocol very effectively controls the forking rate across all the chains.

cs.CR↗