arXiv subjects

Lu Wang

Publications and source records attributed to Lu Wang.

At least 325 records · Page 18Linked to original sources

arXiv2021

Unsupervised Learning of Multi-level Structures for Anomaly Detection

The main difficulty in high-dimensional anomaly detection tasks is the lack of anomalous data for training. And simply collecting anomalous data from the real world, common distributions, or the boundary of normal data manifold may face the problem of missing anomaly modes. This paper first introduces a novel method to generate anomalous data by breaking up global structures while preserving local structures of normal data at multiple levels. It can efficiently expose local abnormal structures of various levels. To fully exploit the exposed multi-level abnormal structures, we propose to train multiple level-specific patch-based detectors with contrastive losses. Each detector learns to detect local abnormal structures of corresponding level at all locations and outputs patchwise anomaly scores. By aggregating the outputs of all level-specific detectors, we obtain a model that can detect all potential anomalies. The effectiveness is evaluated on MNIST, CIFAR10, and ImageNet10 dataset, where the results surpass the accuracy of state-of-the-art methods. Qualitative experiments demonstrate our model is robust that it unbiasedly detects all anomaly modes.

cs.CV↗

arXiv2021

Efficient Attentions for Long Document Summarization

The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source. We further conduct a systematic study of existing efficient self-attentions. Combined with Hepos, we are able to process ten times more tokens than existing models that use full attentions. For evaluation, we present a new dataset, GovReport, with significantly longer documents and summaries. Results show that our models produce significantly higher ROUGE scores than competitive comparisons, including new state-of-the-art results on PubMed. Human evaluation also shows that our models generate more informative summaries with fewer unfaithful errors.

cs.CL↗

arXiv2021

Attention Head Masking for Inference Time Content Selection in Abstractive Summarization

How can we effectively inform content selection in Transformer-based abstractive summarization models? In this work, we present a simple-yet-effective attention head masking technique, which is applied on encoder-decoder attentions to pinpoint salient content at inference time. Using attention head masking, we are able to reveal the relation between encoder-decoder attentions and content selection behaviors of summarization models. We then demonstrate its effectiveness on three document summarization datasets based on both in-domain and cross-domain settings. Importantly, our models outperform prior state-of-the-art models on CNN/Daily Mail and New York Times datasets. Moreover, our inference-time masking technique is also data-efficient, requiring only 20% of the training samples to outperform BART fine-tuned on the full CNN/DailyMail dataset.

cs.CL↗

arXiv2021

Inference Time Style Control for Summarization

How to generate summaries of different styles without requiring corpora in the target styles, or training separate models? We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model. (1) Decoder state adjustment instantly modifies decoder final states with externally trained style scorers, to iteratively refine the output against a target style. (2) Word unit prediction constrains the word usage to impose strong lexical control during generation. In experiments of summarizing with simplicity control, automatic evaluation and human judges both find our models producing outputs in simpler languages while still informative. We also generate news headlines with various ideological leanings, which can be distinguished by humans with a reasonable probability.

cs.CL↗

arXiv2021

Schrödinger-cat states in Landau-Zener-Stückelberg-Majorana interferometry: a multiple Davydov Ansatz approach

Employing the time-dependent variational principle combined with the multiple Davydov $\mathrm{D}_2$ Ansatz, we investigate Landau-Zener (LZ) transitions in a qubit coupled to a photon mode with various initial photon states at zero temperature. Thanks to the multiple Davydov trial states, exact photonic dynamics taking place in the course of the LZ transition is also studied efficiently. With the qubit driven by a linear external field and the photon mode initialized with Schrödinger-cat states, asymptotic behavior of the transition probability beyond the rotating-wave approximation is uncovered for a variety of initial states. Using a sinusoidal external driving field, we also explore the photon-assisted dynamics of Landau-Zener-Stückelberg-Majorana interferometry. Transition pathways involving multiple energy levels are unveiled by analyzing the photon dynamics.

quant-ph↗

arXiv2021

Modeling Heterogeneous Edges to Represent Networks with Graph Auto-Encoder

In the real world, networks often contain multiple relationships among nodes, manifested as the heterogeneity of the edges in the networks. We convert the heterogeneous networks into multiple views by using each view to describe a specific type of relationship between nodes, so that we can leverage the collaboration of multiple views to learn the representation of networks with heterogeneous edges. Given this, we propose a \emph{regularized graph auto-encoders} (RGAE) model, committed to utilizing abundant information in multiple views to learn robust network representations. More specifically, RGAE designs shared and private graph auto-encoders as main components to capture high-order nonlinear structure information of the networks. Besides, two loss functions serve as regularization to extract consistent and unique information, respectively. Concrete experimental results on realistic datasets indicate that our model outperforms state-of-the-art baselines in practical applications.

cs.SI↗

arXiv2021

MD-MTL: An Ensemble Med-Multi-Task Learning Package for DiseaseScores Prediction and Multi-Level Risk Factor Analysis

While many machine learning methods have been used for medical prediction and risk factor analysis on healthcare data, most prior research has involved single-task learning (STL) methods. However, healthcare research often involves multiple related tasks. For instance, implementation of disease scores prediction and risk factor analysis in multiple subgroups of patients simultaneously and risk factor analysis at multi-levels synchronously. In this paper, we developed a new ensemble machine learning Python package based on multi-task learning (MTL), referred to as the Med-Multi-Task Learning (MD-MTL) package and applied it in predicting disease scores of patients, and in carrying out risk factor analysis on multiple subgroups of patients simultaneously. Our experimental results on two datasets demonstrate the utility of the MD-MTL package, and show the advantage of MTL (vs. STL), when analyzing data that is organized into different categories (tasks, which can be various age groups, different levels of disease severity, etc.).

cs.LG↗

arXiv2021

Revealing spatial spillover effect in high-tech industry agglomeration from a high-skilled labor flow network perspective

Understanding the high-tech industrial agglomeration from a spatial-spillover perspective is essential for cities to gain economic and technological competitive advantages. Along with rapid urbanization and the development of fast transportation networks, socioeconomic interactions between cities have been ever-increasing, traditional spatial metrics are not enough to describe actual inter-city connections. High-skilled labor flow between cities strongly influences the high-tech industrial agglomeration, yet receives less attention. By exploiting unique large-scale datasets and tools from complex network and data mining, we construct an inter-city high-skilled labor flow network, which was integrated into spatial econometric models. Our regression results indicate that spatial-spillover effects exist in the development of high-tech industries in the Yangtze River Delta Urban Agglomeration region. Moreover, the spatial-spillover effects are stronger among cities with a higher volume of high-skilled labor flows than among cities with just stronger geographic connections. Additionally, we investigate the channels for the spillover effects and discover that inadequate local government expenses on science and technology likely hamper the high-tech industrial agglomeration, so does the inadequate local educational provision. The increasing foreign direct investments in one city likely encourages the high-tech industrial agglomeration in other cities because of the policy inertia toward traditional industries.

physics.soc-ph↗

arXiv2020

Annotation-Efficient Learning for Medical Image Segmentation based on Noisy Pseudo Labels and Adversarial Learning

Despite that deep learning has achieved state-of-the-art performance for medical image segmentation, its success relies on a large set of manually annotated images for training that are expensive to acquire. In this paper, we propose an annotation-efficient learning framework for segmentation tasks that avoids annotations of training images, where we use an improved Cycle-Consistent Generative Adversarial Network (GAN) to learn from a set of unpaired medical images and auxiliary masks obtained either from a shape model or public datasets. We first use the GAN to generate pseudo labels for our training images under the implicit high-level shape constraint represented by a Variational Auto-encoder (VAE)-based discriminator with the help of the auxiliary masks, and build a Discriminator-guided Generator Channel Calibration (DGCC) module which employs our discriminator's feedback to calibrate the generator for better pseudo labels. To learn from the pseudo labels that are noisy, we further introduce a noise-robust iterative learning method using noise-weighted Dice loss. We validated our framework with two situations: objects with a simple shape model like optic disc in fundus images and fetal head in ultrasound images, and complex structures like lung in X-Ray images and liver in CT images. Experimental results demonstrated that 1) Our VAE-based discriminator and DGCC module help to obtain high-quality pseudo labels. 2) Our proposed noise-robust learning method can effectively overcome the effect of noisy pseudo labels. 3) The segmentation performance of our method without using annotations of training images is close or even comparable to that of learning from human annotations.

eess.IV↗

arXiv2020

Provably Robust Metric Learning

Metric learning is an important family of algorithms for classification and similarity search, but the robustness of learned metrics against small adversarial perturbations is less studied. In this paper, we show that existing metric learning algorithms, which focus on boosting the clean accuracy, can result in metrics that are less robust than the Euclidean distance. To overcome this problem, we propose a novel metric learning algorithm to find a Mahalanobis distance that is robust against adversarial perturbations, and the robustness of the resulting model is certifiable. Experimental results show that the proposed metric learning algorithm improves both certified robust errors and empirical robust errors (errors under adversarial attacks). Furthermore, unlike neural network defenses which usually encounter a trade-off between clean and robust errors, our method does not sacrifice clean errors compared with previous metric learning methods. Our code is available at https://github.com/wangwllu/provably_robust_metric_learning.

cs.LG↗

arXiv2020

Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data

Adversarial black-box attacks aim to craft adversarial perturbations by querying input-output pairs of machine learning models. They are widely used to evaluate the robustness of pre-trained models. However, black-box attacks often suffer from the issue of query inefficiency due to the high dimensionality of the input space, and therefore incur a false sense of model robustness. In this paper, we relax the conditions of the black-box threat model, and propose a novel technique called the spanning attack. By constraining adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset, the spanning attack significantly improves the query efficiency of a wide variety of existing black-box attacks. Extensive experiments show that the proposed method works favorably in both soft-label and hard-label black-box attacks. Our code is available at https://github.com/wangwllu/spanning_attack.

cs.LG↗

arXiv2020

Adaptive Federated Learning and Digital Twin for Industrial Internet of Things

Industrial Internet of Things (IoT) enables distributed intelligent services varying with the dynamic and realtime industrial devices to achieve Industry 4.0 benefits. In this paper, we consider a new architecture of digital twin empowered Industrial IoT where digital twins capture the characteristics of industrial devices to assist federated learning. Noticing that digital twins may bring estimation deviations from the actual value of device state, a trusted based aggregation is proposed in federated learning to alleviate the effects of such deviation. We adaptively adjust the aggregation frequency of federated learning based on Lyapunov dynamic deficit queue and deep reinforcement learning, to improve the learning performance under the resource constraints. To further adapt to the heterogeneity of Industrial IoT, a clustering-based asynchronous federated learning framework is proposed. Numerical results show that the proposed framework is superior to the benchmark in terms of learning accuracy, convergence, and energy saving.

cs.LG↗

arXiv2020

PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation

Pre-trained Transformers have enabled impressive breakthroughs in generating long and fluent text, yet their outputs are often "rambling" without coherently arranged content. In this work, we present a novel content-controlled text generation framework, PAIR, with planning and iterative refinement, which is built upon a large model, BART. We first adapt the BERT model to automatically construct the content plans, consisting of keyphrase assignments and their corresponding sentence-level positions. The BART model is employed for generation without modifying its structure. We then propose a refinement algorithm to gradually enhance the generation quality within the sequence-to-sequence framework. Evaluation with automatic metrics shows that adding planning consistently improves the generation quality on three distinct domains, with an average of 20 BLEU points and 12 METEOR points improvements. In addition, human judges rate our system outputs to be more relevant and coherent than comparisons without planning.

cs.CL↗

arXiv2020

Efficient inverse design and spectrum prediction for nanophotonic devices based on deep recurrent neural networks

In recent years, the development of nanophotonic devices has presented a revolutionary means to manipulate light at nanoscale. Recently, artificial neural networks (ANNs) have displayed powerful ability in the inverse design of nanophotonic devices. However, there is limited research on the inverse design for modeling and learning the sequence characteristics of a spectrum. In this work, we propose a novel deep learning method based on an improved recurrent neural networks to extract the sequence characteristics of a spectrum and achieve inverse design and spectrum prediction. A key feature of the network is that the memory or feedback loops it comprises allow it to effectively recognize time series data. In the context of nanorods hyperbolic metamaterials, we demonstrated the high consistency between the target spectrum and the predicted spectrum, and the network learned the deep physical relationship concerning the structural parameter changes reflected on the spectrum. Moreover, the proposed model is capable of predicting an unknown spectrum based on a known spectrum with only 0.32% mean relative error. We propose this method as an effective and accurate alternative to the application of ANNs in nanophotonics, paving way for fast and accurate design of desired devices.

physics.optics↗

arXiv2020

A uniqueness theorem for asymptotically cylindrical shrinking Ricci solitons

We prove that a shrinking gradient Ricci soliton which agrees to infinite order at spatial infinity with one of the standard cylindrical metrics on $S^k\times \RR^{n-k}$ for $k\geq 2$ along some end must be isometric to the cylinder on that end. When the underlying manifold is complete, it must be globally isometric either to the cylinder or (when $k=n-1$) to its $\ZZ_2$-quotient.

math.DG↗

arXiv2020

Vision-Based Fall Event Detection in Complex Background Using Attention Guided Bi-directional LSTM

Fall event detection, as one of the greatest risks to the elderly, has been a hot research issue in the solitary scene in recent years. Nevertheless, there are few researches on the fall event detection in complex background. Different from most conventional background subtraction methods which depend on background modeling, Mask R-CNN method based on deep learning technique can clearly extract the moving object in noise background. We further propose an attention guided Bi-directional LSTM model for the final fall event detection. To demonstrate the efficiency, the proposed method is verified in the public dataset and self-build dataset. Evaluation of the algorithm performances in comparison with other state-of-the-art methods indicates that the proposed design is accurate and robust, which means it is suitable for the task of fall event detection in complex situation.

cs.CV↗

arXiv2020

Intrinsic current drive by electromagnetic electron drift wave turbulence in tokamak pedestal region

The local intrinsic parallel current density driven by electron drift wave (DW) turbulence including electromagnetic (EM) effects is analytically studied. The scalings of the ratios of intrinsic current density driven by residual turbulent flux and by turbulent source to the bootstrap current density with electron density and temperature are predicted to be $T_e^{3/4}T_i/n_e$ and $T_eT_i/n_e$, respectively. Based on the typical parameters in DIII-D pedestal region, the local intrinsic current density driven by both the residual turbulent flux and the turbulent source is negligible. However, despite the negligible turbulent source driven current, the residual turbulent flux driven local intrinsic current density by EM DW turbulence can reach about 66% of the bootstrap current density for ITER pedestal parameters due to much lower collisionality in ITER than in DIII-D. Moreover, the contributions from adiabatic ES parts, non-adiabatic ES parts and non-adiabatic EM parts of the plasma response to electromagnetic fluctuations are analyzed. It is found that there exists strong cancelation between non-adiabatic ES response and the non-adiabatic EM response for the ITER pedestal case, and thus the kinetic stress contributed by the adiabatic ES response of parallel electron pressure dominates the intrinsic current drive. This is different from the ES electron DW case. Therefore, the EM effects on turbulence driven intrinsic current density should be carefully considered in the future reactor with high ratio of electron pressure to the magnetic pressure and steep pressure profile.

physics.plasm-ph↗

arXiv2020

Intrinsic current drive by electromagnetic electron temperature gradient turbulence in tokamak plasmas

The mean parallel current density evolution equation is presented using electromagnetic (EM) gyrokinetic equation. There exist two types of intrinsic current driving mechanisms resulted from EM electron temperature gradient (ETG) turbulence. The first type is the divergence of residual turbulent flux including a residual stress-like term and a kinetic stress-like term. The second type is named as residual turbulent source, which is driven by the correlation between density and parallel electric field fluctuations. The intrinsic current density driven by the residual turbulent source is negligible as compared to that driven by the residual turbulent flux. The ratio of intrinsic current density driven by EM ETG turbulence to the background bootstrap current density is estimated. The local intrinsic current density driven by the residual turbulent flux for mesoscale variation of turbulent flux can reach about 80% of the bootstrap current density in the core region of ITER standard scenario, but there is no net intrinsic current on a global scale. Based on this, the local intrinsic current driven by EM micro-turbulence and its effects on local modification of the profile of safety factor may be needed to be carefully taken into account in the future device with high beta_e which is the ratio between electron pressure to the magnetic pressure.

physics.plasm-ph↗