arXiv subjects

Xiao Liu

Publications and source records attributed to Xiao Liu.

At least 343 records · Page 19Linked to original sources

arXiv2023

Weighted Point Cloud Normal Estimation

Existing normal estimation methods for point clouds are often less robust to severe noise and complex geometric structures. Also, they usually ignore the contributions of different neighbouring points during normal estimation, which leads to less accurate results. In this paper, we introduce a weighted normal estimation method for 3D point cloud data. We innovate in two key points: 1) we develop a novel weighted normal regression technique that predicts point-wise weights from local point patches and use them for robust, feature-preserving normal regression; 2) we propose to conduct contrastive learning between point patches and the corresponding ground-truth normals of the patches' central points as a pre-training process to facilitate normal regression. Comprehensive experiments demonstrate that our method can robustly handle noisy and complex point clouds, achieving state-of-the-art performance on both synthetic and real-world datasets.

cs.CV↗

arXiv2023

Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention

Compared with standard text, understanding dialogue is more challenging for machines as the dynamic and unexpected semantic changes in each turn. To model such inconsistent semantics, we propose a simple but effective Hierarchical Dialogue Understanding model, HiDialog. Specifically, we first insert multiple special tokens into a dialogue and propose the turn-level attention to learn turn embeddings hierarchically. Then, a heterogeneous graph module is leveraged to polish the learned embeddings. We evaluate our model on various dialogue understanding tasks including dialogue relation extraction, dialogue emotion recognition, and dialogue act classification. Results show that our simple approach achieves state-of-the-art performance on all three tasks above. All our source code is publicly available at https://github.com/ShawX825/HiDialog.

cs.CL↗

arXiv2023

GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner

Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world graph data. Among both sets of graph SSL techniques, the masked graph autoencoders (e.g., GraphMAE)--one type of generative method--have recently produced promising results. The idea behind this is to reconstruct the node features (or structures)--that are randomly masked from the input--with the autoencoder architecture. However, the performance of masked feature reconstruction naturally relies on the discriminability of the input features and is usually vulnerable to disturbance in the features. In this paper, we present a masked self-supervised learning framework GraphMAE2 with the goal of overcoming this issue. The idea is to impose regularization on feature reconstruction for graph SSL. Specifically, we design the strategies of multi-view random re-mask decoding and latent representation prediction to regularize the feature reconstruction. The multi-view random re-mask decoding is to introduce randomness into reconstruction in the feature space, while the latent representation prediction is to enforce the reconstruction in the embedding space. Extensive experiments show that GraphMAE2 can consistently generate top results on various public datasets, including at least 2.45% improvements over state-of-the-art baselines on ogbn-Papers100M with 111M nodes and 1.6B edges.

cs.LG↗

arXiv2023

Anomalous quantized plateaus in two-dimensional electron gas with gate confinement

Quantum information can be coded by the topologically protected edges of fractional quantum Hall (FQH) states. Investigation on FQH edges in the hope of searching and utilizing non-Abelian statistics has been a focused challenge for years. Manipulating the edges, e.g. to bring edges close to each other or to separate edges spatially, is a common and essential step for such studies. The FQH edge structures in a confined region are typically presupposed to be the same as that in the open region in analysis of experimental results, but whether they remain unchanged with extra confinement is obscure. In this work, we present a series of unexpected plateaus in a confined single-layer two-dimensional electron gas (2DEG), which are quantized at anomalous fractions such as 9/4, 17/11, 16/13 and the reported 3/2. We explain all the plateaus by assuming surprisingly larger filling factors in the confined region. Our findings enrich the understanding of edge states in the confined region and in the applications of gate manipulation, which is crucial for the experiments with quantum point contact and interferometer.

cond-mat.mes-hall↗

arXiv2023

DeepStruct: Pretraining of Language Models for Structure Prediction

We introduce a method for improving the structural understanding abilities of language models. Unlike previous approaches that finetune the models with task-specific augmentation, we pretrain language models on a collection of task-agnostic corpora to generate structures from text. Our structure pretraining enables zero-shot transfer of the learned knowledge that models have about the structure tasks. We study the performance of this approach on 28 datasets, spanning 10 structure prediction tasks including open information extraction, joint entity and relation extraction, named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, factual probe, intent detection, and dialogue state tracking. We further enhance the pretraining with the task-specific training sets. We show that a 10B parameter language model transfers non-trivially to most tasks and obtains state-of-the-art performance on 21 of 28 datasets that we evaluate.

cs.CL↗

arXiv2023

Learning Explicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning via Polarization Policy Gradient

Cooperative multi-agent policy gradient (MAPG) algorithms have recently attracted wide attention and are regarded as a general scheme for the multi-agent system. Credit assignment plays an important role in MAPG and can induce cooperation among multiple agents. However, most MAPG algorithms cannot achieve good credit assignment because of the game-theoretic pathology known as \textit{centralized-decentralized mismatch}. To address this issue, this paper presents a novel method, \textit{\underline{M}ulti-\underline{A}gent \underline{P}olarization \underline{P}olicy \underline{G}radient} (MAPPG). MAPPG takes a simple but efficient polarization function to transform the optimal consistency of joint and individual actions into easily realized constraints, thus enabling efficient credit assignment in MAPG. Theoretically, we prove that individual policies of MAPPG can converge to the global optimum. Empirically, we evaluate MAPPG on the well-known matrix game and differential game, and verify that MAPPG can converge to the global optimum for both discrete and continuous action spaces. We also evaluate MAPPG on a set of StarCraft II micromanagement tasks and demonstrate that MAPPG outperforms the state-of-the-art MAPG algorithms.

cs.LG↗

arXiv2023

Inverse Models for Estimating the Initial Condition of Spatio-Temporal Advection-Diffusion Processes

Inverse problems involve making inference about unknown parameters of a physical process using observational data. This paper investigates an important class of inverse problems -- the estimation of the initial condition of a spatio-temporal advection-diffusion process using spatially sparse data streams. Three spatial sampling schemes are considered, including irregular, non-uniform and shifted uniform sampling. The irregular sampling scheme is the general scenario, while computationally efficient solutions are available in the spectral domain for non-uniform and shifted uniform sampling. For each sampling scheme, the inverse problem is formulated as a regularized convex optimization problem that minimizes the distance between forward model outputs and observations. The optimization problem is solved by the Alternating Direction Method of Multipliers algorithm, which also handles the situation when a linear inequality constraint (e.g., non-negativity) is imposed on the model output. Numerical examples are presented, code is made available on GitHub, and discussions are provided to generate some useful insights of the proposed inverse modeling approaches.

stat.ME↗

arXiv2023

Uplifting Message Passing Neural Network with Graph Original Information

Message passing neural networks (MPNNs) learn the representation of graph-structured data based on graph original information, including node features and graph structures, and have shown astonishing improvement in node classification tasks. However, the expressive power of MPNNs is upper bounded by the first-order Weisfeiler-Leman test and its accuracy still has room for improvement. This work studies how to improve MPNNs' expressiveness and generalizability by fully exploiting graph original information both theoretically and empirically. It further proposes a new GNN model called INGNN (INformation-enhanced Graph Neural Network) that leverages the insights to improve node classification performance. Extensive experiments on both synthetic and real datasets demonstrate the superiority (average rank 1.78) of our INGNN compared with state-of-the-art methods.

cs.LG↗

arXiv2023

Imperceptible Adversarial Attack via Invertible Neural Networks

Adding perturbations via utilizing auxiliary gradient information or discarding existing details of the benign images are two common approaches for generating adversarial examples. Though visual imperceptibility is the desired property of adversarial examples, conventional adversarial attacks still generate traceable adversarial perturbations. In this paper, we introduce a novel Adversarial Attack via Invertible Neural Networks (AdvINN) method to produce robust and imperceptible adversarial examples. Specifically, AdvINN fully takes advantage of the information preservation property of Invertible Neural Networks and thereby generates adversarial examples by simultaneously adding class-specific semantic information of the target class and dropping discriminant information of the original class. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet-1K demonstrate that the proposed AdvINN method can produce less imperceptible adversarial images than the state-of-the-art methods and AdvINN yields more robust adversarial examples with high confidence compared to other adversarial attacks.

cs.CV↗

arXiv2023

Capillary gravity water waves linearized at monotone shear flows: eigenvalues and inviscid damping

This paper is concerned with the eigenvalues and linear inviscid damping of the 2D capillary gravity water waves of finite depth $x_2\in(-h,0)$ linearized at a monotone shear flow $U(x_2)$. Unlike the linearized Euler equation in a fixed channel where eigenvalues exist only in low horizontal wave number $k$, we first prove the linearized capillary gravity wave has two branches of eigenvalues $-ikc^\pm(k)$, where the wave speeds $c^\pm(k)=O(\sqrt{|k|})$ for $|k|\gg1$ have the same asymptotics as the those of the linear irrotational capillary gravity waves. Under the additional assumption of $U"\ne0$, we obtain the complete continuation of these two branches, which are all the eigenvalues in this (and some other) case(s). Particularly $-ikc^-(k)$ could bifurcate into unstable eigenvalues at $c^-(k)=U(-h)$. The bifurcation of unstable eigenvalues from inflection values of $U$ is also proved. Assuming no singular modes, i.e. no embedded eigenvalues for any wave number $k$, linear solutions $(v(t,x),η(t,x_1))$ are studieded in both periodic-in-$x_1$ and $x_1\in R$ cases, where $v$ is the velocity and $η$ the surface profile. Solutions can be split into $(v^p,η^p)$ and $(v^c,η^c)$ whose $k$-th Fourier mode in $x_1$ correspond to the eigenvalues and the continuous spectra of wave number $k$, respectively. The component $(v^p,η^p)$ is governed by a (possibly unstable) dispersion relation given by the eigenvalues, which are simply $k\to-ikc^\pm(k)$ in the case of $x_1\in R$. The other component $(v^c,η^c)$ satisfies the inviscid damping as fast as $|v_1^c|_{L_x^2},|η^c|_{L_x^2}=O(|t|^{-1})$ and $|v_2^c|_{L_x^2}=O(t^{-2})$ as $|t|\gg1$. Additional decay of $tv_1^c,t^2v_2^c$ in $L_x^2L_t^q$, $q\in(2,\infty]$, is obtained after leading asymptotic terms are removed, which are in the forms of $t$-dependent translations in $x_1$ of certain functions of $x$.

math.AP↗

arXiv2022

Convolutional Non-homogeneous Poisson Process with Application to Wildfire Risk Quantification for Power Delivery Networks

The current projection shows that much of the continental U.S. will have significantly hotter and drier days in the following decades, leading to more wildfire hazards that threaten the safety of power grid. Unfortunately, the U.S. power industry is not well prepared and still predominantly relies on empirical fire indices which do not consider the full spectrum of dynamic environmental factors. This paper proposes a new spatio-temporal point process model, Convolutional Non-homogeneous Poisson Process (cNHPP), to quantify wildfire risks for power delivery networks. The proposed model captures both the current short-term and cumulative long-term effects of covariates on wildfire risks, and the spatio-temporal dependency among different segments of the power delivery network. The computation and interpretation of the intensity function are thoroughly investigated, and the connection between cNHPP and Recurrent Neural Network is also discussed. We apply the proposed approach to estimate wildfire risks on major transmission lines in California, utilizing historical fire data, meteorological and vegetation data obtained from the National Oceanic and Atmospheric Administration and National Aeronautics and Space Administration. Comparison studies are performed to show the applicability and predictive capability of the proposed approach. Useful insights are obtained that potentially enhance power grid resilience against wildfires.

stat.ME↗

arXiv2022

Unbox the Blackbox: Predict and Interpret YouTube Viewership Using Deep Learning

Predicting video viewership is a top priority for content creators and video-sharing sites. Content creators live on such predictions to maximize influences and minimize budgets. Video-sharing sites rely on this prediction to promote credible videos and curb violative videos. Although deep learning champions viewership prediction, it lacks interpretability, which is fundamental to increasing the adoption of predictive models and prescribing measurements to improve viewership. Following the design-science paradigm, we propose a novel interpretable IT system, Precise Wide and Deep Learning (PrecWD), to precisely interpret viewership prediction. Improving upon state-of-the-art frameworks, PrecWD offers precise feature effects and designs an unstructured component. PrecWD outperforms benchmarks in two contexts: health video viewership prediction and misinformation viewership prediction. A user study confirms the superior interpretability of PrecWD. This study contributes to IS design theory with generalizable design principles and an interpretable predictive framework. Our findings provide implications to improve video viewership and credibility.

cs.LG↗

arXiv2022

An adaptive human-in-the-loop approach to emission detection of Additive Manufacturing processes and active learning with computer vision

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation.

cs.LG↗

arXiv2022

ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information

ClueWeb22, the newest iteration of the ClueWeb line of datasets, provides 10 billion web pages affiliated with rich information. Its design was influenced by the need for a high quality, large scale web corpus to support a range of academic and industry research, for example, in information systems, retrieval-augmented AI systems, and model pretraining. Compared with earlier ClueWeb corpora, the ClueWeb22 corpus is larger, more varied, of higher-quality, and aligned with the document distributions in commercial web search. Besides raw HTML, ClueWeb22 includes rich information about the web pages provided by industry-standard document understanding systems, including the visual representation of pages rendered by a web browser, parsed HTML structure information from a neural network parser, and pre-processed cleaned document text to lower the barrier to entry. Many of these signals have been widely used in industry but are available to the research community for the first time at this scale.

cs.IR↗

arXiv2022

Efficient Calculation of NMR Shielding Constants Using Composite Method Approximations and Locally Dense Basis Sets

This paper presents a systematic study of applying composite method approximations with locally dense basis sets (LDBS) to efficiently calculate NMR shielding constants in small and medium-sized molecules. The pcSseg-n series of basis sets are shown to have similar accuracy to the pcS-n series when n $\geq1$ and can slightly reduce compute costs. We identify two different LDBS partition schemes that perform very effectively for density functional calculations. We select a large subset of the recent NS372 database containing 290 H, C, N, and O shielding values evaluated by reference methods on 106 molecules to carefully assess methods of the high, medium, and low compute costs to make practical recommendations. Our assessment covers conventional electronic structure methods (DFT and wavefunction) with global basis calculations, as well as their use in one of the satisfactory LDBS approaches, and a range of composite approaches, also with and without LDBS. Altogether 99 methods are evaluated. On this basis, we recommend different methods to reach three different levels of accuracy and time requirements across the four nuclei considered.

physics.chem-ph↗

arXiv2022

AMFlow: a Mathematica package for Feynman integrals computation via Auxiliary Mass Flow

AMFlow is a Mathematica package to numerically compute dimensionally regularized Feynman integrals via the recently proposed auxiliary mass flow method. In this framework, integrals are treated as functions of an auxiliary mass parameter and their results can be obtained by constructing and solving differential systems with respect to this parameter, in an automatic way. The usage of this package is described in detail through an explicit example of double-box family involved in two-loop $t\bar{t}$ hadroproduction.

hep-ph↗

arXiv2022

IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models

We introduce a new open information extraction (OIE) benchmark for pre-trained language models (LM). Recent studies have demonstrated that pre-trained LMs, such as BERT and GPT, may store linguistic and relational knowledge. In particular, LMs are able to answer ``fill-in-the-blank'' questions when given a pre-defined relation category. Instead of focusing on pre-defined relations, we create an OIE benchmark aiming to fully examine the open relational information present in the pre-trained LMs. We accomplish this by turning pre-trained LMs into zero-shot OIE systems. Surprisingly, pre-trained LMs are able to obtain competitive performance on both standard OIE datasets (CaRB and Re-OIE2016) and two new large-scale factual OIE datasets (TAC KBP-OIE and Wikidata-OIE) that we establish via distant supervision. For instance, the zero-shot pre-trained LMs outperform the F1 score of the state-of-the-art supervised OIE methods on our factual OIE datasets without needing to use any training sets. Our code and datasets are available at https://github.com/cgraywang/IELM

cs.CL↗

arXiv2022

SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval

Sampling proper negatives from a large document pool is vital to effectively train a dense retrieval model. However, existing negative sampling strategies suffer from the uninformative or false negative problem. In this work, we empirically show that according to the measured relevance scores, the negatives ranked around the positives are generally more informative and less likely to be false negatives. Intuitively, these negatives are not too hard (\emph{may be false negatives}) or too easy (\emph{uninformative}). They are the ambiguous negatives and need more attention during training. Thus, we propose a simple ambiguous negatives sampling method, SimANS, which incorporates a new sampling probability distribution to sample more ambiguous negatives. Extensive experiments on four public and one industry datasets show the effectiveness of our approach. We made the code and models publicly available in \url{https://github.com/microsoft/SimXNS}.

cs.CL↗