arXiv subjects

Xiao Yu

Publications and source records attributed to Xiao Yu.

At least 91 records · Page 5Linked to original sources

arXiv2023

Multi-Frame Self-Supervised Depth Estimation with Multi-Scale Feature Fusion in Dynamic Scenes

Multi-frame methods improve monocular depth estimation over single-frame approaches by aggregating spatial-temporal information via feature matching. However, the spatial-temporal feature leads to accuracy degradation in dynamic scenes. To enhance the performance, recent methods tend to propose complex architectures for feature matching and dynamic scenes. In this paper, we show that a simple learning framework, together with designed feature augmentation, leads to superior performance. (1) A novel dynamic objects detecting method with geometry explainability is proposed. The detected dynamic objects are excluded during training, which guarantees the static environment assumption and relieves the accuracy degradation problem of the multi-frame depth estimation. (2) Multi-scale feature fusion is proposed for feature matching in the multi-frame depth network, which improves feature matching, especially between frames with large camera motion. (3) The robust knowledge distillation with a robust teacher network and reliability guarantee is proposed, which improves the multi-frame depth estimation without computation complexity increase during the test. The experiments show that our proposed methods achieve great performance improvement on the multi-frame depth estimation.

cs.CV↗

arXiv2023

KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning

In task-oriented dialogs (TOD), reinforcement learning (RL) algorithms train a model to directly optimize response for task-related metrics. However, RL needs to perform exploration, which can be time-consuming due to the slow auto-regressive sequence generation process. We investigate an approach to create a more efficient RL-based algorithm to improve TOD performance in an offline setting. First, we use a faster generation procedure that samples from independent next-word distributions after training the language model (LM) with supervised learning. We then introduce a fine-grained reward function to help the model focus on learning key information in a dialog, by measuring the importance and semantic closeness of each generated token. Experiments on the MultiWoZ dataset show our new training algorithm, Keywords Reinforcement Learning with Next-word Sampling (KRLS), achieves state-of-the-art performance on the end-to-end response generation task, with a 15% training time reduction compared to a standard RL algorithm using auto-regressive generation.

cs.CL↗

arXiv2023

Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning

Planning for goal-oriented dialogue often requires simulating future dialogue interactions and estimating task progress. Many approaches thus consider training neural networks to perform look-ahead search algorithms such as A* search and Monte Carlo Tree Search (MCTS). However, this training often requires abundant annotated data, which creates challenges when faced with noisy annotations or low-resource settings. We introduce GDP-Zero, an approach using Open-Loop MCTS to perform goal-oriented dialogue policy planning without any model training. GDP-Zero prompts a large language model to act as a policy prior, value function, user simulator, and system model during the tree search. We evaluate GDP-Zero on the goal-oriented task PersuasionForGood, and find that its responses are preferred over ChatGPT up to 59.32% of the time, and are rated more persuasive than ChatGPT during interactive evaluations.

cs.CL↗

arXiv2023

Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data

Jointly extracting entity pairs and their relations is challenging when working on distantly-supervised data with ambiguous or noisy labels. To mitigate such impact, we propose uncertainty-aware bootstrap learning, which is motivated by the intuition that the higher uncertainty of an instance, the more likely the model confidence is inconsistent with the ground truths. Specifically, we first explore instance-level data uncertainty to create an initial high-confident examples. Such subset serves as filtering noisy instances and facilitating the model to converge fast at the early stage. During bootstrap learning, we propose self-ensembling as a regularizer to alleviate inter-model uncertainty produced by noisy labels. We further define probability variance of joint tagging probabilities to estimate inner-model parametric uncertainty, which is used to select and build up new reliable training instances for the next iteration. Experimental results on two large datasets reveal that our approach outperforms existing strong baselines and related methods.

cs.CL↗

arXiv2023

Controllable Mixed-Initiative Dialogue Generation through Prompting

Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Conversational agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. The standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. However, these supervised generation models are limited by the cost and quality of data annotation. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation. We formalize prompt construction for controllable mixed-initiative dialogue. Our findings show improvements over fine-tuning and ground truth responses according to human evaluation and automatic metrics for two tasks: PersuasionForGood and Emotional Support Conversations.

cs.CL↗

arXiv2023

FastKASSIM: A Fast Tree Kernel-Based Syntactic Similarity Metric

Syntax is a fundamental component of language, yet few metrics have been employed to capture syntactic similarity or coherence at the utterance- and document-level. The existing standard document-level syntactic similarity metric is computationally expensive and performs inconsistently when faced with syntactically dissimilar documents. To address these challenges, we present FastKASSIM, a metric for utterance- and document-level syntactic similarity which pairs and averages the most similar constituency parse trees between a pair of documents based on tree kernels. FastKASSIM is more robust to syntactic dissimilarities and runs up to to 5.32 times faster than its predecessor over documents in the r/ChangeMyView corpus. FastKASSIM's improvements allow us to examine hypotheses in two settings with large documents. We find that syntactically similar arguments on r/ChangeMyView tend to be more persuasive, and that syntax is predictive of authorship attribution in the Australian High Court Judgment corpus.

cs.CL↗

arXiv2022

Semileptonic decays of doubly charmed baryons with $Ξ_c-Ξ_c'$ mixing

We study the $Ξ_c- Ξ_c'$ mixing effects in the semileptonic decays the doubly charm baryons of $Ξ_{cc}$. We focus on the ratio of ${\cal R}(θ_c) \equiv {\cal B}( Ξ_{cc} \to Ξ_c' e^+ ν_e)/ {\cal B}( Ξ_{cc} \to Ξ_c e^+ ν_e) $ and find that $({\cal R}(θ_0),{\cal R}(- θ_0)) =(0.46 \pm 0.01,7.33 \pm 0.23)$ with $θ_0 = 0.137π$, which are in sharp contrast to ${\cal R}(0)=2.15\pm0.11$ without the mixing. The ratio is enhanced~(suppressed) by a factor of four for a negative~(positive) $θ_c$. In addition, the polarization asymmetries of $Ξ_c^{(\prime)}$ are found to be $α(-θ_0) = 0.32 ~(-0.76)$ and $α(θ_0) = -0.82~(-0.38)$. As ${\cal R}$ and $α$ are highly sensitive to $θ_c$ and unaffected by the $W$-exchange contributions, they provide excellent opportunities to determine $θ_c$ in the ongoing experiments.

hep-ph↗

arXiv2022

Semileptonic decays of doubly charmed baryons with bag model

We study the semileptonic decays of $B_{cc}$ ${\rightarrow}$ $B_c\ell^+ν_\ell$ with the bag model, where $\ell$ = $(e, μ)$, $B_{cc}$ = $(Ξ_{cc}^{++}$, $Ξ_{cc}^+$, $Ω_{cc}^+$), and $ B_c$ are the singly charmed baryons with $J^P= 1/2^+$. We obtain the decay widths of $Γ(Ξ_{cc}^{++}{\rightarrow}Ξ_c^+e^+ν_e, Ξ_c^{\prime+}e^+ν_e, Λ_c^+e^+ν_e, Ω_c^+ e^+ν_e) =(5.1\pm 0.1 , 11\pm 1, 0.34\pm 0.06, 0.76\pm 0.06)\times 10^{-14}$~GeV, $Γ(Ξ_{cc}^+\rightarrow Ξ_c^0e^+ν_e, Ξ_c^{\prime0}e^+ν_e , Σ_c^0e^+ν_e) = (5.1\pm 0.6, 11\pm 1, 1.5\pm 0.1) \times 10^{-14}$~GeV, and $Γ(Ω_{cc}^+\rightarrow Ω_c^0 e^+ν_e, Ξ_c^0e^+ν_e , Ξ_c^{\prime0} e^+ν_e) = (22\pm 2, 0.32 \pm 0.04, 0.77\pm 0.06)\times 10^{-14}$~GeV. We also get that $Γ$($B_{cc}$ ${\rightarrow}$ $B_cμ^+ν_μ$)/$Γ$($B_{cc}$ ${\rightarrow}$ $B_ce^+ν_e$) = $0.97\sim 1.00$. In addition, we discuss the $SU(3)$ flavor breaking effects, classified into three aspects: phase space differences, spectator quarks, and overlappings of the transited quarks. In particular, we show that the breaking effects are dominated by the phase space differences, which can be as large as 25\%. Explicitly, we find that $Γ(Ξ_{cc}^{++} \to Λ_c ^+ e^+ ν_e) V_{cs}^2/Γ(Ξ_{cc}^{++} \to Ξ_c ^+ e^+ ν_e )V_{cd}^2 = 1.24$, which is expected as $1$ under the exact $SU(3)$ flavor symmetry.

hep-ph↗

arXiv2022

Improving Model Training via Self-learned Label Representations

Modern neural network architectures have shown remarkable success in several large-scale classification and prediction tasks. Part of the success of these architectures is their flexibility to transform the data from the raw input representations (e.g. pixels for vision tasks, or text for natural language processing tasks) to one-hot output encoding. While much of the work has focused on studying how the input gets transformed to the one-hot encoding, very little work has examined the effectiveness of these one-hot labels. In this work, we demonstrate that more sophisticated label representations are better for classification than the usual one-hot encoding. We propose Learning with Adaptive Labels (LwAL) algorithm, which simultaneously learns the label representation while training for the classification task. These learned labels can significantly cut down on the training time (usually by more than 50%) while often achieving better test accuracies. Our algorithm introduces negligible additional parameters and has a minimal computational overhead. Along with improved training times, our learned labels are semantically meaningful and can reveal hierarchical relationships that may be present in the data.

cs.LG↗

arXiv2022

Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information

Context: Stack Overflow is very helpful for software developers who are seeking answers to programming problems. Previous studies have shown that a growing number of questions are of low quality and thus obtain less attention from potential answerers. Gao et al. proposed an LSTM-based model (i.e., BiLSTM-CC) to automatically generate question titles from the code snippets to improve the question quality. However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens. Objective: This paper proposes CCBERT, a deep learning based novel model to enhance the performance of question title generation by making full use of the bi-modal information of the entire question body. Method: CCBERT follows the encoder-decoder paradigm and uses CodeBERT to encode the question body into hidden representations, a stacked Transformer decoder to generate predicted tokens, and an additional copy attention layer to refine the output distribution. Both the encoder and decoder perform the multi-head self-attention operation to better capture the long-range dependencies. This paper builds a dataset containing around 200,000 high-quality questions filtered from the data officially published by Stack Overflow to verify the effectiveness of the CCBERT model. Results: CCBERT outperforms all the baseline models on the dataset. Experiments on both code-only and low-resource datasets show the superiority of CCBERT with less performance degradation. The human evaluation also shows the excellent performance of CCBERT concerning both readability and correlation criteria.

cs.CL↗

arXiv2022

Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer

Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems. Nevertheless, if inexperienced developers fail to describe their problems clearly, it is hard for them to attract sufficient attention and get the anticipated answers. We propose M$_3$NSCT5, a novel approach to automatically generate multiple post titles from the given code snippets. Developers may use the generated titles to find closely related posts and complete their problem descriptions. M$_3$NSCT5 employs the CodeT5 backbone, which is a pre-trained Transformer model having an excellent language understanding and generation ability. To alleviate the ambiguity issue that the same code snippets could be aligned with different titles under varying contexts, we propose the maximal marginal multiple nucleus sampling strategy to generate multiple high-quality and diverse title candidates at a time for the developers to choose from. We build a large-scale dataset with 890,000 question posts covering eight programming languages to validate the effectiveness of M$_3$NSCT5. The automatic evaluation results on the BLEU and ROUGE metrics demonstrate the superiority of M$_3$NSCT5 over six state-of-the-art baseline models. Moreover, a human evaluation with trustworthy results also demonstrates the great potential of our approach for real-world application.

cs.SE↗

arXiv2022

A recipe of training neural network-based LDPC decoders

It is known belief propagation decoding variants of LDPC codes can be unrolled easily as neural networks after assigning differed weights to message passing edges flexibly. In this paper we focus on how to determine these weights, in the form of trainable paramters, within a framework of deep learning. Firstly, a new method is proposed to generate high-quality training data via exploiting an approximation to the targeted mixture density. Then the strong positive correlation between training loss and decoding metrics is fully exposed after tracing the training evolution curves. Lastly, for the purpose of facilitating training convergence and reducing decoding complexity, we highlight the necessity of slashing the number of trainable parameters while emphasizing the locations of these survived ones, which is justified in the extensive simulation.

cs.IT↗

arXiv2022

Bilinear integral operator on Morrey-Banach spaces and its application

In this paper, we give the definability of bilinear singular and fractional integral operators on Morrey-Banach space, as well as their commutators and we prove the boundedness of such operators on Morrey-Banach spaces. Moreover, the necessary condition for BMO via the bounedness of bilinear commutators on Morrey-Banach space is also given. As a application of our main results, we get the necessary conditions for BMO via the bounedness of bilinear integral operators on weighted Morrey space and Morrey space with variable exponents. Finally, we obtain the boundedness of bilinear C-Z operator on Morrey space with variable exponents.

math.FA↗

arXiv2022

Robust single-sideband-modulated Raman light generation for atom interferometry by FBG-based optical rectangular filtration

Low-phase-noise and pure-spectrum Raman light is vital for high-precision atom interferometry by two-photon Raman transition. A preferred and prevalent solution for Raman light generation is electro-optic phase modulation. However, phase modulation inherently brings in double sidebands, resulting in residual sideband effects of multiple laser pairs beside Raman light in atom interferometry. Based on a well-designed rectangular fiber Bragg grating and an electro-optic modulator, optical single-sideband modulation has been realized at 1560 nm with a stable suppression ratio better than -25 dB despite of intense temperature variations. After optical filtration and frequency doubling, a robust phase-coherent Raman light at 780 nm is generated with a stable SNR of better than -19 dB and facilitates measuring the local gravity successfully. This proposed all-fiber single-sideband-modulated Raman light source, characterized as robust, compact and low-priced, is practical and potential for field applications of portable atom interferometry.

physics.atom-ph↗

arXiv2021

SIGL: Securing Software Installations Through Deep Graph Learning

Many users implicitly assume that software can only be exploited after it is installed. However, recent supply-chain attacks demonstrate that application integrity must be ensured during installation itself. We introduce SIGL, a new tool for detecting malicious behavior during software installation. SIGL collects traces of system call activity, building a data provenance graph that it analyzes using a novel autoencoder architecture with a graph long short-term memory network (graph LSTM) for the encoder and a standard multilayer perceptron for the decoder. SIGL flags suspicious installations as well as the specific installation-time processes that are likely to be malicious. Using a test corpus of 625 malicious installers containing real-world malware, we demonstrate that SIGL has a detection accuracy of 96%, outperforming similar systems from industry and academia by up to 87% in precision and recall and 45% in accuracy. We also demonstrate that SIGL can pinpoint the processes most likely to have triggered malicious behavior, works on different audit platforms and operating systems, and is robust to training data contamination and adversarial attack. It can be used with application-specific models, even in the presence of new software versions, as well as application-agnostic meta-models that encompass a wide range of applications and installers.

cs.CR↗

arXiv2021

A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts

Code comment has been an important part of computer programs, greatly facilitating the understanding and maintenance of source code. However, high-quality code comments are often unavailable in smart contracts, the increasingly popular programs that run on the blockchain. In this paper, we propose a Multi-Modal Transformer-based (MMTrans) code summarization approach for smart contracts. Specifically, the MMTrans learns the representation of source code from the two heterogeneous modalities of the Abstract Syntax Tree (AST), i.e., Structure-based Traversal (SBT) sequences and graphs. The SBT sequence provides the global semantic information of AST, while the graph convolution focuses on the local details. The MMTrans uses two encoders to extract both global and local semantic information from the two modalities respectively, and then uses a joint decoder to generate code comments. Both the encoders and the decoder employ the multi-head attention structure of the Transformer to enhance the ability to capture the long-range dependencies between code tokens. We build a dataset with over 300K pairs of smart contracts, and evaluate the MMTrans on it. The experimental results demonstrate that the MMTrans outperforms the state-of-the-art baselines in terms of four evaluation metrics by a substantial margin, and can generate higher quality comments.

cs.SE↗

arXiv2021

Asymptotic spreading of KPP reactive fronts in heterogeneous shifting environments

We study the asymptotic spreading of Kolmogorov-Petrovsky-Piskunov (KPP) fronts in heterogeneous shifting habitats, with any number of shifting speeds, by further developing the method based on the theory of viscosity solutions of Hamilton-Jacobi equations. Our framework addresses both reaction-diffusion equation and integro-differential equations with a distributed time-delay. The latter leads to a class of limiting equations of Hamilton-Jacobi-type depending on the variable $x/t$ and in which the time and space derivatives are coupled together. We will first establish uniqueness results for these Hamilton-Jacobi equations using elementary arguments, and then characterize the spreading speed in terms of a reduced equation on a one-dimensional domain in the variable $s=x/t$. In terms of the standard Fisher-KPP equation, our results leads to a new class of "asymptotically homogeneous" environments which share the same spreading speed with the corresponding homogeneous environments.

math.AP↗

arXiv2020

Commutators of weighted Hardy operator on weighted $λ$-central Morrey space

In this paper, the authors prove the boundedness of commutators generated by the weighted Hardy operator on weighted $λ$-central Morrey space with the weight $ω$ satisfying the doubling condition. Moreover, the authors give the characterization for the weighted $λ$-central Campanato space by introducing a new kind of operator which is related to the commutator of weighted Hardy operator.

math.CA↗