arXiv subjects

Yu Xia

Publications and source records attributed to Yu Xia.

At least 109 records · Page 6Linked to original sources

arXiv2023

Contrastive Bootstrapping for Label Refinement

Traditional text classification typically categorizes texts into pre-defined coarse-grained classes, from which the produced models cannot handle the real-world scenario where finer categories emerge periodically for accurate services. In this work, we investigate the setting where fine-grained classification is done only using the annotation of coarse-grained categories and the coarse-to-fine mapping. We propose a lightweight contrastive clustering-based bootstrapping method to iteratively refine the labels of passages. During clustering, it pulls away negative passage-prototype pairs under the guidance of the mapping from both global and local perspectives. Experiments on NYT and 20News show that our method outperforms the state-of-the-art methods by a large margin.

cs.CL↗

arXiv2023

The Closeness of In-Context Learning and Weight Shifting for Softmax Regression

Large language models (LLMs) are known for their exceptional performance in natural language processing, making them highly effective in many human life-related or even job-related tasks. The attention mechanism in the Transformer architecture is a critical component of LLMs, as it allows the model to selectively focus on specific input parts. The softmax unit, which is a key part of the attention mechanism, normalizes the attention scores. Hence, the performance of LLMs in various NLP tasks depends significantly on the crucial role played by the attention mechanism with the softmax unit. In-context learning, as one of the celebrated abilities of recent LLMs, is an important concept in querying LLMs such as ChatGPT. Without further parameter updates, Transformers can learn to predict based on few in-context examples. However, the reason why Transformers becomes in-context learners is not well understood. Recently, several works [ASA+22,GTLV22,ONR+22] have studied the in-context learning from a mathematical perspective based on a linear regression formulation $\min_x\| Ax - b \|_2$, which show Transformers' capability of learning linear functions in context. In this work, we study the in-context learning based on a softmax regression formulation $\min_{x} \| \langle \exp(Ax), {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2$ of Transformer's attention mechanism. We show the upper bounds of the data transformations induced by a single self-attention layer and by gradient-descent on a $\ell_2$ regression loss for softmax prediction function, which imply that when training self-attention-only Transformers for fundamental regression tasks, the models learned by gradient-descent and Transformers show great similarity.

cs.CL↗

arXiv2023

DeePMD-kit v2: A software package for Deep Potential models

DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics simulations using machine learning potentials (MLP) known as Deep Potential (DP) models. This package, which was released in 2017, has been widely used in the fields of physics, chemistry, biology, and material science for studying atomistic systems. The current version of DeePMD-kit offers numerous advanced features such as DeepPot-SE, attention-based and hybrid descriptors, the ability to fit tensile properties, type embedding, model deviation, Deep Potential - Range Correction (DPRc), Deep Potential Long Range (DPLR), GPU support for customized operators, model compression, non-von Neumann molecular dynamics (NVNMD), and improved usability, including documentation, compiled binary packages, graphical user interfaces (GUI), and application programming interfaces (API). This article presents an overview of the current major version of the DeePMD-kit package, highlighting its features and technical details. Additionally, the article benchmarks the accuracy and efficiency of different models and discusses ongoing developments.

physics.chem-ph↗

arXiv2023

DocRED-FE: A Document-Level Fine-Grained Entity And Relation Extraction Dataset

Joint entity and relation extraction (JERE) is one of the most important tasks in information extraction. However, most existing works focus on sentence-level coarse-grained JERE, which have limitations in real-world scenarios. In this paper, we construct a large-scale document-level fine-grained JERE dataset DocRED-FE, which improves DocRED with Fine-Grained Entity Type. Specifically, we redesign a hierarchical entity type schema including 11 coarse-grained types and 119 fine-grained types, and then re-annotate DocRED manually according to this schema. Through comprehensive experiments we find that: (1) DocRED-FE is challenging to existing JERE models; (2) Our fine-grained entity types promote relation classification. We make DocRED-FE with instruction and the code for our baselines publicly available at https://github.com/PKU-TANGENT/DOCRED-FE.

cs.CL↗

arXiv2022

Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models

Structural probing work has found evidence for latent syntactic information in pre-trained language models. However, much of this analysis has focused on monolingual models, and analyses of multilingual models have employed correlational methods that are confounded by the choice of probing tasks. In this study, we causally probe multilingual language models (XGLM and multilingual BERT) as well as monolingual BERT-based models across various languages; we do this by performing counterfactual perturbations on neuron activations and observing the effect on models' subject-verb agreement probabilities. We observe where in the model and to what extent syntactic agreement is encoded in each language. We find significant neuron overlap across languages in autoregressive multilingual language models, but not masked language models. We also find two distinct layer-wise effect patterns and two distinct sets of neurons used for syntactic agreement, depending on whether the subject and verb are separated by other tokens. Finally, we find that behavioral analyses of language models are likely underestimating how sensitive masked language models are to syntactic information.

cs.CL↗

arXiv2022

Block-STM: Scaling Blockchain Execution by Turning Ordering Curse to a Performance Blessing

Block-STM is a parallel execution engine for smart contracts, built around the principles of Software Transactional Memory. Transactions are grouped in blocks, and every execution of the block must yield the same deterministic outcome. Block-STM further enforces that the outcome is consistent with executing transactions according to a preset order, leveraging this order to dynamically detect dependencies and avoid conflicts during speculative transaction execution. At the core of Block-STM is a novel, low-overhead collaborative scheduler of execution and validation tasks. Block-STM is implemented on the main branch of the Diem Blockchain code-base and runs in production at Aptos. Our evaluation demonstrates that Block-STM is adaptive to workloads with different conflict rates and utilizes the inherent parallelism therein. Block-STM achieves up to $110k$ tps in the Diem benchmarks and up to $170k$ tps in the Aptos Benchmarks, which is a $20$x and $17$x improvement over the sequential baseline with $32$ threads, respectively. The throughput on a contended workload is up to $50k$ tps and $80k$ tps in Diem and Aptos benchmarks, respectively.

cs.DC↗

arXiv2022

Low Resource Style Transfer via Domain Adaptive Meta Learning

Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of non-parallel data to guide transferring different text styles. (ii) colossal performance degradation when fine-tuning the model in new domains. In this work, we propose DAML-ATM (Domain Adaptive Meta-Learning with Adversarial Transfer Model), which consists of two parts: DAML and ATM. DAML is a domain adaptive meta-learning approach to learn general knowledge in multiple heterogeneous source domains, capable of adapting to new unseen domains with a small amount of data. Moreover, we propose a new unsupervised TST approach Adversarial Transfer Model (ATM), composed of a sequence-to-sequence pre-trained language model and uses adversarial style training for better content preservation and style transfer. Results on multi-domain datasets demonstrate that our approach generalizes well on unseen low-resource domains, achieving state-of-the-art results against ten strong baselines.

cs.CL↗

arXiv2021

Exploring Text-transformers in AAAI 2021 Shared Task: COVID-19 Fake News Detection in English

In this paper, we describe our system for the AAAI 2021 shared task of COVID-19 Fake News Detection in English, where we achieved the 3rd position with the weighted F1 score of 0.9859 on the test set. Specifically, we proposed an ensemble method of different pre-trained language models such as BERT, Roberta, Ernie, etc. with various training strategies including warm-up,learning rate schedule and k-fold cross-validation. We also conduct an extensive analysis of the samples that are not correctly classified. The code is available at:https://github.com/archersama/3rd-solution-COVID19-Fake-News-Detection-in-English.

cs.CL↗

arXiv2020

Taurus: Lightweight Parallel Logging for In-Memory Database Management Systems (Extended Version)

Existing single-stream logging schemes are unsuitable for in-memory database management systems (DBMSs) as the single log is often a performance bottleneck. To overcome this problem, we present Taurus, an efficient parallel logging scheme that uses multiple log streams, and is compatible with both data and command logging. Taurus tracks and encodes transaction dependencies using a vector of log sequence numbers (LSNs). These vectors ensure that the dependencies are fully captured in logging and correctly enforced in recovery. Our experimental evaluation with an in-memory DBMS shows that Taurus's parallel logging achieves up to 9.9x and 2.9x speedups over single-streamed data logging and command logging, respectively. It also enables the DBMS to recover up to 22.9x and 75.6x faster than these baselines for data and command logging, respectively. We also compare Taurus with two state-of-the-art parallel logging schemes and show that the DBMS achieves up to 2.8x better performance on NVMe drives and 9.2x on HDDs.

cs.DB↗

arXiv2020

Sparse phase retrieval via Phaseliftoff

The aim of sparse phase retrieval is to recover a $k$-sparse signal $\mathbf{x}_0\in \mathbb{C}^{d}$ from quadratic measurements $|\langle \mathbf{a}_i,\mathbf{x}_0\rangle|^2$ where $\mathbf{a}_i\in \mathbb{C}^d, i=1,\ldots,m$. Noting $|\langle \mathbf{a}_i,\mathbf{x}_0\rangle|^2={\text{Tr}}(A_iX_0)$ with $A_i=\mathbf{a}_i\mathbf{a}_i^*\in \mathbb{C}^{d\times d}, X_0=\mathbf{x}_0\mathbf{x}_0^*\in \mathbb{C}^{d\times d}$, one can recast sparse phase retrieval as a problem of recovering a rank-one sparse matrix from linear measurements. Yin and Xin introduced PhaseLiftOff which presents a proxy of rank-one condition via the difference of trace and Frobenius norm. By adding sparsity penalty to PhaseLiftOff, in this paper, we present a novel model to recover sparse signals from quadratic measurements. Theoretical analysis shows that the solution to our model provides the stable recovery of $\mathbf{x}_0$ under almost optimal sampling complexity $m=O(k\log(d/k))$. The computation of our model is carried out by the difference of convex function algorithm (DCA). Numerical experiments demonstrate that our algorithm outperforms other state-of-the-art algorithms used for solving sparse phase retrieval.

math.FA↗

arXiv2019

Cluster Analysis of High-Dimensional scRNA Sequencing Data

With ongoing developments and innovations in single-cell RNA sequencing methods, advancements in sequencing performance could empower significant discoveries as well as new emerging possibilities to address biological and medical investigations. In the study, we will be using the dataset collected by the authors of Systematic comparative analysis of single cell RNA-sequencing methods. The dataset consists of single-cell and single nucleus profiling from three types of samples - cell lines, peripheral blood mononuclear cells, and brain tissue, which offers 36 libraries in six separate experiments in a single center. Our quantitative comparison aims to identify unique characteristics associated with different single-cell sequencing methods, especially among low-throughput sequencing methods and high-throughput sequencing methods. Our procedures also incorporate evaluations of every method's capacity for recovering known biological information in the samples through clustering analysis.

stat.AP↗

arXiv2019

Visual-UWB Navigation System for Unknown Environments

Navigation applications relying on the Global Navigation Satellite System (GNSS) are limited in indoor environments and GNSS-denied outdoor terrains such as dense urban or forests. In this paper, we present a novel accurate, robust and low-cost GNSS-independent navigation system, which is composed of a monocular camera and Ultra-wideband (UWB) transceivers. Visual techniques have gained excellent results when computing the incremental motion of the sensor, and UWB methods have proved to provide promising localization accuracy due to the high time resolution of the UWB ranging signals. However, the monocular visual techniques with scale ambiguity are not suitable for applications requiring metric results, and UWB methods assume that the positions of the UWB transceiver anchor are pre-calibrated and known, thus precluding their application in unknown and challenging environments. To this end, we advocate leveraging the monocular camera and UWB to create a map of visual features and UWB anchors. We propose a visual-UWB Simultaneous Localization and Mapping (SLAM) algorithm which tightly combines visual and UWB measurements to form a joint non-linear optimization problem on Lie-Manifold. The 6 Degrees of Freedom (DoF) state of the vehicles and the map are estimated by minimizing the UWB ranging errors and landmark reprojection errors. Our navigation system starts with an exploratory task which performs the real-time visual-UWB SLAM to obtain the global map, then the navigation task by reusing this global map. The tasks can be performed by different vehicles in terms of equipped sensors and payload capability in a heterogeneous team. We validate our system on the public datasets, achieving typical centimeter accuracy and 0.1% scale error.

cs.RO↗

arXiv2019

The recovery of complex sparse signals from few phaseless measurements

We study the stable recovery of complex $k$-sparse signals from as few phaseless measurements as possible. The main result is to show that one can employ $\ell_1$ minimization to stably recover complex $k$-sparse signals from $m\geq O(k\log (n/k))$ complex Gaussian random quadratic measurements with high probability. To do that, we establish that Gaussian random measurements satisfy the restricted isometry property over rank-$2$ and sparse matrices with high probability. This paper presents the first theoretical estimation of the measurement number for stably recovering complex sparse signals from complex Gaussian quadratic measurements.

math.FA↗

arXiv2019

Monolithic shape-programmable dielectric liquid crystal elastomer actuators

Macroscale robotic systems have demonstrated great capabilities of high speed, precise, and agile functions. However, the ability of soft robots to perform complex tasks, especially in centimeter and millimeter scale, remains limited due to the unavailability of fast, energy-efficient soft actuators that can programmably change shape. Here, we combine desirable characteristics from two distinct active materials: fast and efficient actuation from dielectric elastomers and facile shape programmability from liquid crystal elastomers into a single shape changing electrical actuator. Uniaxially aligned monoliths achieve strain rates over 120%/s with energy conversion efficiency of 20% while moving loads over 700 times the actuator weight. The combined actuator technology offers unprecedented opportunities towards miniaturization with precision, efficiency, and more degrees of freedom for applications in soft robotics and beyond.

cond-mat.soft↗

arXiv2018

Weighted total variation based convex clustering

Data clustering is a fundamental problem with a wide range of applications. Standard methods, eg the $k$-means method, usually require solving a non-convex optimization problem. Recently, total variation based convex relaxation to the $k$-means model has emerged as an attractive alternative for data clustering. However, the existing results on its exact clustering property, ie, the condition imposed on data so that the method can provably give correct identification of all cluster memberships, is only applicable to very specific data and is also much more restrictive than that of some other methods. This paper aims at the revisit of total variation based convex clustering, by proposing a weighted sum-of-$\ell_1$-norm relating convex model. Its exact clustering property established in this paper, in both deterministic and probabilistic context, is applicable to general data and is much sharper than the existing results. These results provided good insights to advance the research on convex clustering. Moreover, the experiments also demonstrated that the proposed convex model has better empirical performance when be compared to standard clustering methods, and thus it can see its potential in practice.

math.OC↗

arXiv2017

Making Faces: Universal Inverse Design of Surfaces with Thin Nematic Elastomer Sheets

Programmable shape-shifting materials can take different physical forms to achieve multifunctionality in a dynamic and controllable manner. Although morphing a shape from 2D to 3D via programmed inhomogeneous local deformations has been demonstrated in various ways, the inverse problem -- programming a sheet to take an arbitrary desired 3D shape -- is much harder yet critical to realize specific functions. Here, we address this inverse problem in thin liquid crystal elastomer (LCE) sheets, where the shape is preprogrammed by precise and local control of the molecular orientation of the liquid crystal monomers. We show how blueprints for arbitrary surface geometries as well as local extrinsic curvatures can be generated using approximate numerical methods. Backed by faithfully alignable and rapidly lockable LCE chemistry, we precisely embed our designs in LCE sheets using advanced top-down microfabrication techniques. We thus successfully produce flat sheets that, upon thermal activation, take an arbitrary desired shape, such as a face. The general design principles presented here for creating an arbitrary 3D shape will allow for exploration of unmet needs in flexible electronics, metamaterials, aerospace and medical devices, and more.

cond-mat.soft↗

arXiv2017

CeMon: A Cost-effective Flow Monitoring System in Software Defined Networks

Network monitoring and measurement are crucial in network management to facilitate quality of service routing and performance evaluation. Software Defined Networking (SDN) makes network management easier by separating the control plane and data plane. Network monitoring in SDN is relatively light-weight since operators only need to install a monitoring module into the controller. Active monitoring techniques usually introduce extra overhead into the network. The state-of-the-art approaches utilize sampling, aggregation and passive measurement techniques to reduce measurement overhead. However, little work has focused on reducing the communication cost of network monitoring. Moreover, most of the existing approaches select polling switch nodes by sub-optimal local heuristics.Inspired by the visibility and central control of SDN, we propose CeMon, a generic low-cost high-accuracy monitoring system that supports various network management tasks. We first propose a Maximum Coverage Polling Scheme (MCPS) to optimize the polling cost for all active flows. The problem is formulated as a weighted set cover problem which is proved to be NP-hard. Heuristics are presented to obtain the polling scheme efficiently and handle traffic dynamics practically. In order to balance the cost and flexibility, an Adaptive Fine-grained Polling Scheme (AFPS) is proposed as a complementary method to implement flow level measurement tasks. Three sampling algorithms are further proposed to eliminate measurement overhead while maintain high accuracy. Both emulation and simulation results show that MCPS reduces more than 50% of the communication cost with negligible loss of accuracy for different topologies and traffics. We also use real traces to demonstrate that AFPS reduces the cost by up to 20% with only 1.5% loss in accuracy.

cs.NI↗

arXiv2017

FlowCover: Low-cost Flow Monitoring Scheme in Software Defined Networks

Network monitoring and measurement are crucial in network management to facilitate quality of service routing and performance evaluation. Software Defined Networking (SDN) makes network management easier by separating the control plane and data plane. Network monitoring in SDN is lightweight as operators only need to install a monitoring module into the controller. Active monitoring techniques usually introduce too many overheads into the network. The state-of-the-art approaches utilize sampling method, aggregation flow statistics and passive measurement techniques to reduce overheads. However, little work in literature has focus on reducing the communication cost of network monitoring. Moreover, most of the existing approaches select the polling switch nodes by sub-optimal local heuristics. Inspired by the visibility and central control of SDN, we propose FlowCover, a low-cost high-accuracy monitoring scheme to support various network management tasks. We leverage the global view of the network topology and active flows to minimize the communication cost by formulating the problem as a weighted set cover, which is proved to be NP-hard. Heuristics are presented to obtain the polling scheme efficiently and handle flow changes practically. We build a simulator to evaluate the performance of FlowCover. Extensive experiment results show that FlowCover reduces roughly 50% communication cost without loss of accuracy in most cases.

cs.NI↗