arXiv subjects

Shuai Lu

Publications and source records attributed to Shuai Lu.

At least 109 records · Page 6Linked to original sources

arXiv2022

Increasing stability of the first order linearized inverse Schrödinger potential problem with integer power type nonlinearities

We investigate the increasing stability of the inverse Schrödinger potential problem with integer power type nonlinearities at a large wavenumber. By considering the first order linearized system with respect to the unknown potential function, a combination formula of the first order linearization is proposed, which provides a Lipschitz type stability for the recovery of the Fourier coefficients of the unknown potential function in low frequency mode. These stability results highlight the advantage of nonlinearity in solving this inverse potential problem by explicitly quantifying the dependence to the wavenumber and the nonlinearities index. A reconstruction algorithm for general power type nonlinearities is also provided. Several numerical examples illuminate the efficiency of our proposed algorithm.

math.AP↗

arXiv2022

Conditional Goal-oriented Trajectory Prediction for Interacting Vehicles with Vectorized Representation

This paper aims to tackle the interactive behavior prediction task, and proposes a novel Conditional Goal-oriented Trajectory Prediction (CGTP) framework to jointly generate scene-compliant trajectories of two interacting agents. Our CGTP framework is an end to end and interpretable model, including three main stages: context encoding, goal interactive prediction and trajectory interactive prediction. First, a Goals-of-Interest Network (GoINet) is designed to extract the interactive features between agent-to-agent and agent-to-goals using a graph-based vectorized representation. Further, the Conditional Goal Prediction Network (CGPNet) focuses on goal interactive prediction via a combined form of marginal and conditional goal predictors. Finally, the Goaloriented Trajectory Forecasting Network (GTFNet) is proposed to implement trajectory interactive prediction via the conditional goal-oriented predictors, with the predicted future states of the other interacting agent taken as inputs. In addition, a new goal interactive loss is developed to better learn the joint probability distribution over goal candidates between two interacting agents. In the end, the proposed method is conducted on Argoverse motion forecasting dataset, In-house cut-in dataset, and Waymo open motion dataset. The comparative results demonstrate the superior performance of our proposed CGTP model than the mainstream prediction methods.

cs.RO↗

arXiv2022

Automating Code Review Activities by Large-Scale Pre-training

Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing, understanding and even running the programs to assess logic, functionality, latency, style and other factors. It turns out that developers have to spend far too much time reviewing the code of their peers. Accordingly, it is in significant demand to automate the code review process. In this research, we focus on utilizing pre-training techniques for the tasks in the code review scenario. We collect a large-scale dataset of real-world code changes and code reviews from open-source projects in nine of the most popular programming languages. To better understand code diffs and reviews, we propose CodeReviewer, a pre-trained model that utilizes four pre-training tasks tailored specifically for the code review scenario. To evaluate our model, we focus on three key tasks related to code review activities, including code change quality estimation, review comment generation and code refinement. Furthermore, we establish a high-quality benchmark dataset based on our collected data for these three tasks and conduct comprehensive experiments on it. The experimental results demonstrate that our model outperforms the previous state-of-the-art pre-training approaches in all tasks. Further analysis show that our proposed pre-training tasks and the multilingual pre-training dataset benefit the model on the understanding of code changes and reviews.

cs.SE↗

arXiv2022

On a Dynamic Variant of the Iteratively Regularized Gauss-Newton Method with Sequential Data

For numerous parameter and state estimation problems, assimilating new data as they become available can help produce accurate and fast inference of unknown quantities. While most existing algorithms for solving those kind of ill-posed inverse problems can only be used with a single instance of the observed data, in this work we propose a new framework that enables existing algorithms to invert multiple instances of data in a sequential fashion. Specifically we will work with the well-known iteratively regularized Gauss-Newton method (IRGNM), a variational methodology for solving nonlinear inverse problems. We develop a theory of convergence analysis for a proposed dynamic IRGNM algorithm in the presence of Gaussian white noise. We combine this algorithm with the classical IRGNM to deliver a practical (hybrid) algorithm that can invert data sequentially while producing fast estimates. Our work includes the proof of well-definedness of the proposed iterative scheme, as well as various error bounds that rely on standard assumptions for nonlinear inverse problems. We use several numerical experiments to verify our theoretical findings, and to highlight the benefits of incorporating sequential data. The context of the numerical experiments comprises various parameter identification problems including a Darcy flow elliptic PDE example, and that of electrical impedance tomography.

math.NA↗

arXiv2022

Discrimination-Based Double Auction for Maximizing Social Welfare in the Electricity and Heating Market Considering Privacy Preservation

This paper proposes a doubled-sided auction mechanism with price discrimination for social welfare (SW) maximization in the electricity and heating market. In this mechanism, energy service providers (ESPs) submit offers and load aggregators (LAs) submit bids to an energy trading center (ETC) to maximize their utility; in turn, the selfless ETC as an auctioneer leverages dis-criminatory price weights to regulate the behaviors of ESPs and LAs, which combines the individual benefits of each stakeholder with the overall social welfare to achieve the global optimum. Nash games are employed to describe the interactions between players with the same market role. Theoretically, we first prove the existence and uniqueness of the Nash equilibrium; then, considering the requirement of game players to preserve privacy, a distributed algorithm based on the alternating direction method of multipliers is developed to implement distributed bidding and analytical target cascading algorithm is applied to reach the balance of demand and supply. We validated the proposed mechanism using case studies on a city-level distribution system. The results indicated that the achieved SW improved by 4%-15% compared with other mechanisms, and also verified the effectiveness of the distributed algorithm.

cs.GT↗

arXiv2022

Increasing stability in the linearized inverse Schrödinger potential problem with power type nonlinearities

We consider increasing stability in the inverse Schrödinger potential problem with power type nonlinearities at a large wavenumber. Two linearization approaches, with respect to small boundary data and small potential function, are proposed and their performance on the inverse Schrödinger potential problem is investigated. It can be observed that higher order linearization for small boundary data can provide an increasing stability for an arbitrary power type nonlinearity term if the wavenumber is chosen large. Meanwhile, linearization with respect to the potential function leads to increasing stability for a quadratic nonlinearity term, which highlights the advantage of nonlinearity in solving the inverse Schrödinger potential problem. Noticing that both linearization approaches can be numerically approximated, we provide several reconstruction algorithms for the quadratic and general power type nonlinearity terms, where one of these algorithms is designed based on boundary measurements of multiple wavenumbers. Several numerical examples shed light on the efficiency of our proposed algorithms.

math.AP↗

arXiv2022

ReACC: A Retrieval-Augmented Code Completion Framework

Code completion, which aims to predict the following code token(s) according to the code context, can improve the productivity of software development. Recent work has proved that statistical language modeling with transformers can greatly improve the performance in the code completion task via learning from large-scale source code datasets. However, current approaches focus only on code context within the file or project, i.e. internal context. Our distinction is utilizing "external" context, inspired by human behaviors of copying from the related code snippets when writing code. Specifically, we propose a retrieval-augmented code completion framework, leveraging both lexical copying and referring to code with similar semantics by retrieval. We adopt a stage-wise training approach that combines a source code retriever and an auto-regressive language model for programming language. We evaluate our approach in the code completion task in Python and Java programming languages, achieving a state-of-the-art performance on CodeXGLUE benchmark.

cs.SE↗

arXiv2022

Learning to Recommend Method Names with Global Context

In programming, the names for the program entities, especially for the methods, are the intuitive characteristic for understanding the functionality of the code. To ensure the readability and maintainability of the programs, method names should be named properly. Specifically, the names should be meaningful and consistent with other names used in related contexts in their codebase. In recent years, many automated approaches are proposed to suggest consistent names for methods, among which neural machine translation (NMT) based models are widely used and have achieved state-of-the-art results. However, these NMT-based models mainly focus on extracting the code-specific features from the method body or the surrounding methods, the project-specific context and documentation of the target method are ignored. We conduct a statistical analysis to explore the relationship between the method names and their contexts. Based on the statistical results, we propose GTNM, a Global Transformer-based Neural Model for method name suggestion, which considers the local context, the project-specific context, and the documentation of the method simultaneously. Experimental results on java methods show that our model can outperform the state-of-the-art results by a large margin on method name suggestion, demonstrating the effectiveness of our proposed model.

cs.SE↗

arXiv2022

UniXcoder: Unified Cross-Modal Pre-training for Code Representation

Pre-trained models for programming languages have recently demonstrated great success on code intelligence. To support both code-related understanding and generation tasks, recent works attempt to pre-train unified encoder-decoder models. However, such encoder-decoder framework is sub-optimal for auto-regressive tasks, especially code completion that requires a decoder-only manner for efficient inference. In this paper, we present UniXcoder, a unified cross-modal pre-trained model for programming language. The model utilizes mask attention matrices with prefix adapters to control the behavior of the model and leverages cross-modal contents like AST and code comment to enhance code representation. To encode AST that is represented as a tree in parallel, we propose a one-to-one mapping method to transform AST in a sequence structure that retains all structural information from the tree. Furthermore, we propose to utilize multi-modal contents to learn representation of code fragment with contrastive learning, and then align representations among programming languages using a cross-modal generation task. We evaluate UniXcoder on five code-related tasks over nine datasets. To further evaluate the performance of code fragment representation, we also construct a dataset for a new task, called zero-shot code-to-code search. Results show that our model achieves state-of-the-art performance on most tasks and analysis reveals that comment and AST can both enhance UniXcoder.

cs.CL↗

arXiv2021

Suppressing Andreev bound state zero bias peaks using a strongly dissipative lead

Hybrid semiconductor-superconductor nanowires are predicted to host Majorana zero modes, manifested as zero-bias peaks (ZBPs) in tunneling conductance. ZBPs alone, however, are not sufficient evidence due to the ubiquitous presence of Andreev bound states in the same system. Here, we implement a strongly resistive normal lead in our InAs-Al nanowire devices and show that most of the expected ZBPs, corresponding to zero-energy Andreev bound states, can be suppressed, a phenomenon known as environmental Coulomb blockade. Our result is the first experimental demonstration of this dissipative interaction effect on Andreev bound states and can serve as a possible filter to narrow down ZBP phase diagram in future Majorana searches.

cond-mat.mes-hall↗

arXiv2021

Honeycomb lattice in metal-rich chalcogenide Fe2Te

Two-dimensional honeycomb crystals have inspired intense research interest for their novel properties and great potential in electronics and optoelectronics. Here, through molecular beam epitaxy on SrTiO3(001), we report successful epitaxial growth of metal-rich chalcogenide Fe2Te, a honeycomb-structured film that has no direct bulk analogue, under Te-limited growth conditions. The structural morphology and electronic properties of Fe2Te are explored with scanning tunneling microscopy and angle resolved photoemission spectroscopy, which reveal electronic bands cross the Fermi level and nearly flat bands. Moreover, we find a weak interfacial interaction between Fe2Te and the underlying substrates, paving a newly developed alternative avenue for honeycomb-based electronic devices.

cond-mat.mtrl-sci↗

arXiv2021

Oxygen adsorption induced superconductivity in ultrathin FeTe film on SrTiO3(001)

The phenomenon of oxygen incorporation induced superconductivity in iron telluride (Fe1+yTe, with antiferromagnetic (AFM) orders) is intriguing and quite different from the case of FeSe. Until now, the microscopic origin of the induced superconductivity and the role of oxygen are far from clear. Here, by combining in-situ scanning tunneling microscopy/spectroscopy (STM/STS) and x-ray photoemission spectroscopy (XPS) on oxygenated FeTe, we found physically adsorbed O2 molecules crystallized into c(2/3x2) structure as an oxygen overlayer at low temperature, which was vital for superconductivity. The O2 overlayer were not epitaxial on the FeTe lattice, which implied weak O2-FeTe interaction but strong molecular interactions. Energy shift observed in the STS and XPS measurements indicated hole doping effect from the O2 overlayer to the FeTe layer, leading to a superconducting gap of 4.5 meV opened across the Fermi level. Our direct microscopic probe clarified the role of oxygen on FeTe and emphasized the importance of charge transfer effect to induce superconductivity in iron-chalcogenide thin films.

cond-mat.supr-con↗

arXiv2021

Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy

Statistical language modeling and translation with transformers have found many successful applications in program understanding and generation tasks, setting high benchmarks for tools in modern software development environments. The finite context window of these neural models means, however, that they will be unable to leverage the entire relevant context of large files and packages for any given task. While there are many efforts to extend the context window, we introduce an architecture-independent approach for leveraging the syntactic hierarchies of source code for incorporating entire file-level context into a fixed-length window. Using concrete syntax trees of each source file we extract syntactic hierarchies and integrate them into context window by selectively removing from view more specific, less relevant scopes for a given task. We evaluate this approach on code generation tasks and joint translation of natural language and source code in Python programming language, achieving a new state-of-the-art in code completion and summarization for Python in the CodeXGLUE benchmark. We also introduce new CodeXGLUE benchmarks for user-experience-motivated tasks: code completion with normalized literals, method body completion/code summarization conditioned on file-level context.

cs.LG↗

arXiv2021

Hierarchical Electricity and Carbon Trading in Transmission and Distribution Networks Based on Virtual Federated Prosumer

Facing the dilemma of growing energy demand and mitigating carbon emissions, this paper proposes an energy sharing mechanism based on virtual federated prosumers (VFPs) with budget allocation for joint electricity and carbon market to incentivize distributed energy resources to participate in the hierarchical market and reduce carbon emissions. At the transmission level, the regional transmission operator coordinates transactions between two markets, the inter-VFP energy sharing market and the wholesale market, intending to minimize the overall cost of VFPs. The energy sharing market clearing problem is formulated as a generalized Nash game, for which we develop a first-order response algorithm to obtain the equilibrium. At the distribution level, the VFPs play the role of selfless auctioneer that leverage discriminatory weights and benchmark prices to allocate the electricity-carbon budget among entities in the VFP to maximize social welfare. The Nash game is exploited to characterize the budget allocation problem, for which a distributed feedback allocation algorithm is proposed. The entire hierarchical electricity and carbon trading is modeled as an equilibrium problem and is solved iteratively. Case studies based on a practical regional grid verify the effectiveness of the proposed algorithm and show that the mechanism is effective in improving energy efficiency and reducing carbon emissions.

math.OC↗

arXiv2021

GraphCodeBERT: Pre-training Code Representations with Data Flow

Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarization, etc. However, existing pre-trained models regard a code snippet as a sequence of tokens, while ignoring the inherent structure of code, which provides crucial code semantics and would enhance the code understanding process. We present GraphCodeBERT, a pre-trained model for programming language that considers the inherent structure of code. Instead of taking syntactic-level structure of code like abstract syntax tree (AST), we use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables. Such a semantic-level structure is neat and does not bring an unnecessarily deep hierarchy of AST, the property of which makes the model more efficient. We develop GraphCodeBERT based on Transformer. In addition to using the task of masked language modeling, we introduce two structure-aware pre-training tasks. One is to predict code structure edges, and the other is to align representations between source code and code structure. We implement the model in an efficient way with a graph-guided masked attention function to incorporate the code structure. We evaluate our model on four tasks, including code search, clone detection, code translation, and code refinement. Results show that code structure and newly introduced pre-training tasks can improve GraphCodeBERT and achieves state-of-the-art performance on the four downstream tasks. We further show that the model prefers structure-level attentions over token-level attentions in the task of code search.

cs.SE↗

arXiv2021

Determination of source terms in diffusion and wave equations by observations after incidents: uniqueness and stability

We consider a diffusion and a wave equations: $$ \partial_t^ku(x,t) = Δu(x,t) + μ(t)f(x), \quad x\in Ω, \, t>0, \quad k=1,2 $$ with the zero initial and boundary conditions, where $Ω\subset \mathbb{R}^d$ is a bounded domain. We establish uniqueness and/or stability results for inverse problems of 1. determining $μ(t)$, $0<t<T$ with given $f(x)$; 2. determining $f(x)$, $x\in Ω$ with given $μ(t)$ \end{itemize} by data of $u$: $u(x_0,\cdot)$ with fixed point $x_0\in Ω$ or Neumann data on subboundary over time interval. In our inverse problems, data are taken over time interval $T_1<t<T_1$, by assuming that $T<T_1<T_2$ and $μ(t)=0$ for $t\ge T$, which means that the source stops to be active after the time $T$ and the observations are started only after $T$. This assumption is practical by such a posteriori data after incidents, although inverse problems had been well studied in the case of $T=0$. We establish the non-uniqueness, the uniqueness and conditional stability for a diffusion and a wave equations. The proofs are based on eigenfunction expansions of the solutions $u(x,t)$, and we rely on various knowledge of the generalized Weierstrass theorem on polynomial approximation, almost periodic functions, Carleman estimate, non-harmonic Fourier series.

math.AP↗

arXiv2021

WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach

Producing the embedding of a sentence in an unsupervised way is valuable to natural language matching and retrieval problems in practice. In this work, we conduct a thorough examination of pretrained model based unsupervised sentence embeddings. We study on four pretrained models and conduct massive experiments on seven datasets regarding sentence semantics. We have there main findings. First, averaging all tokens is better than only using [CLS] vector. Second, combining both top andbottom layers is better than only using top layers. Lastly, an easy whitening-based vector normalization strategy with less than 10 lines of code consistently boosts the performance.

cs.CL↗

arXiv2021

CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.

cs.SE↗