arXiv subjects

Bing Liu

Publications and source records attributed to Bing Liu.

At least 181 records · Page 10Linked to original sources

arXiv2021

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks. Although some CL techniques have been proposed for document sentiment classification, we are not aware of any CL work on ASC. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues: (1) transfer knowledge learned from previous tasks to the new task to help it learn a better model, and (2) maintain the performance of the models for previous tasks so that they are not forgotten. This paper proposes a novel capsule network based model called B-CL to address these issues. B-CL markedly improves the ASC performance on both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of B-CL is demonstrated through extensive experiments.

cs.CL↗

arXiv2021

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR

cs.CL↗

arXiv2021

CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

This paper studies continual learning (CL) of a sequence of aspect sentiment classification(ASC) tasks in a particular CL setting called domain incremental learning (DIL). Each task is from a different domain or product. The DIL setting is particularly suited to ASC because in testing the system needs not know the task/domain to which the test data belongs. To our knowledge, this setting has not been studied before for ASC. This paper proposes a novel model called CLASSIC. The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing. Experimental results show the high effectiveness of CLASSIC.

cs.CL↗

arXiv2021

On the surface brightness radial profile of the extended $γ$-ray sources

The morphology of the extended $γ$-ray source is governed by the propagation process of parent relativistic particles. In this paper, we investigate the surface brightness radial profile of extended $γ$-ray sources illuminated by cosmic ray protons and electrons, considering the radiation mechanisms, projection effects, and the response of instruments. We found that the parent particle species and the propagation process can cause considerable differences in the observed radial profiles. Thus, the surface brightness profile can be used as a unique tool to identify the radiation mechanism and the propagation process of the parent particles. In addition, We also discuss the possible implications regarding the latest discoveries from very/ultra-high energy $γ$-ray instruments like LHAASO and HAWC.

astro-ph.HE↗

arXiv2021

Building an Application Independent Natural Language Interface

Traditional approaches to building natural language (NL) interfaces typically use a semantic parser to parse the user command and convert it to a logical form, which is then translated to an executable action in an application. However, it is still challenging for a semantic parser to correctly parse natural language. For a different domain, the parser may need to be retrained or tuned, and a new translator also needs to be written to convert the logical forms to executable actions. In this work, we propose a novel and application independent approach to building NL interfaces that does not need a semantic parser or a translator. It is based on natural language to natural language matching and learning, where the representation of each action and each user command are both in natural language. To perform a user intended action, the system only needs to match the user command with the correct action representation, and then execute the corresponding action. The system also interactively learns new (paraphrased) commands for actions to expand the action representations over time. Our experimental results show the effectiveness of the proposed approach.

cs.CL↗

arXiv2021

Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification

The ability of neural networks (NNs) to learn and remember multiple tasks sequentially is facing tough challenges in achieving general artificial intelligence due to their catastrophic forgetting (CF) issues. Fortunately, the latest OWM Orthogonal Weights Modification) and other several continual learning (CL) methods suggest some promising ways to overcome the CF issue. However, none of existing CL methods explores the following three crucial questions for effectively overcoming the CF issue: that is, what knowledge does it contribute to the effective weights modification of the NN during its sequential tasks learning? When the data distribution of a new learning task changes corresponding to the previous learned tasks, should a uniform/specific weight modification strategy be adopted or not? what is the upper bound of the learningable tasks sequentially for a given CL method? ect. To achieve this, in this paper, we first reveals the fact that of the weight gradient of a new learning task is determined by both the input space of the new task and the weight space of the previous learned tasks sequentially. On this observation and the recursive least square optimal method, we propose a new efficient and effective continual learning method EOWM via enhanced OWM. And we have theoretically and definitively given the upper bound of the learningable tasks sequentially of our EOWM. Extensive experiments conducted on the benchmarks demonstrate that our EOWM is effectiveness and outperform all of the state-of-the-art CL baselines.

cs.LG↗

arXiv2021

Kekule Lattice in Graphdiyne: Coexistence of Phononic and Electronic Higher-Order Band Topology

The topological physics has been extensively studied in different kinds of bosonic and fermionic systems, ranging from artificial structures to natural materials. However, the coexistence of topological phonon and electron in one single material is seldom reported. Recently, graphdiyne is proposed to be a two-dimensional (2D) electronic second-order topological insulator (SOTI). In this work, based on density-functional tight-binding calculations, we found that graphdiyne is equivalent to the Kekule lattice, also realizing a 2D phononic SOTI in both out-of-plane and in-plane modes. Depending on edge terminations, the characterized topological corner states can be either inside or outside the bulk gap, which are tunable by local corner potential. Most remarkably, a unique selectivity of space and symmetry is revealed in electron-phonon coupling between the localized phononic and electronic topological corner states. Our results not only demonstrate the phononic higher-order band topology in a real carbon material, but also provide an opportunity to investigate the interplay between phononic and electronic higher-order topological states.

cond-mat.mes-hall↗

arXiv2021

ActiveEA: Active Learning for Neural Entity Alignment

Entity Alignment (EA) aims to match equivalent entities across different Knowledge Graphs (KGs) and is an essential step of KG fusion. Current mainstream methods -- neural EA models -- rely on training with seed alignment, i.e., a set of pre-aligned entity pairs which are very costly to annotate. In this paper, we devise a novel Active Learning (AL) framework for neural EA, aiming to create highly informative seed alignment to obtain more effective EA models with less annotation cost. Our framework tackles two main challenges encountered when applying AL to EA: (1) How to exploit dependencies between entities within the AL strategy. Most AL strategies assume that the data instances to sample are independent and identically distributed. However, entities in KGs are related. To address this challenge, we propose a structure-aware uncertainty sampling strategy that can measure the uncertainty of each entity as well as its impact on its neighbour entities in the KG. (2) How to recognise entities that appear in one KG but not in the other KG (i.e., bachelors). Identifying bachelors would likely save annotation budget. To address this challenge, we devise a bachelor recognizer paying attention to alleviate the effect of sampling bias. Empirical results show that our proposed AL strategy can significantly improve sampling quality with good generality across different datasets, EA models and amount of bachelors.

cs.CL↗

arXiv2021

Zero-Shot Dialogue State Tracking via Cross-Task Transfer

Zero-shot transfer learning for dialogue state tracking (DST) enables us to handle a variety of task-oriented dialogue domains without the expense of collecting in-domain data. In this work, we propose to transfer the \textit{cross-task} knowledge from general question answering (QA) corpora for the zero-shot DST task. Specifically, we propose TransferQA, a transferable generative QA model that seamlessly combines extractive QA and multi-choice QA via a text-to-text transformer framework, and tracks both categorical slots and non-categorical slots in DST. In addition, we introduce two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation, which enable our model to handle "none" value slots in the zero-shot DST setting. The extensive experiments show that our approaches substantially improve the existing zero-shot and few-shot results on MultiWoz. Moreover, compared to the fully trained baseline on the Schema-Guided Dialogue dataset, our approach shows better generalization ability in unseen domains.

cs.CL↗

arXiv2021

NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions

Existing conversational systems are mostly agent-centric, which assumes the user utterances would closely follow the system ontology (for NLU or dialogue state tracking). However, in real-world scenarios, it is highly desirable that the users can speak freely in their own way. It is extremely hard, if not impossible, for the users to adapt to the unknown system ontology. In this work, we attempt to build a user-centric dialogue system. As there is no clean mapping for a user's free form utterance to an ontology, we first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions. Learning such a mapping poses new challenges on reasoning over existing knowledge, ranging from factoid knowledge, commonsense knowledge to the users' own situations. To this end, we build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation. Collected via dialogue simulation and paraphrasing, NUANCED contains 5.1k dialogues, 26k turns of high-quality user responses. We conduct experiments, showing both the usefulness and challenges of our problem setting. We believe NUANCED can serve as a valuable resource to push existing research from the agent-centric system to the user-centric system. The code and data is publicly available at \url{https://github.com/facebookresearch/nuanced}.

cs.CL↗

arXiv2021

Non-inductive plasma current sustainment with stochastic electron cyclotron in EXL-50 spherical torus

The start-up and sustainment of a stochastic wave non-inductive current on a spherical torus was experimentally demonstrated for the first time using only electron cyclotron waves. The plasma current is insensitive to the injection angle of ECWs and approximately linearly correlated with the slope of the X-ray spectrum. Its direction is determined by the vertical magnetic field (BV). The temporal development in the number of X-ray bremsstrahlung photons with a specified energy is consistent with the stochastic heating model. Moreover, the ratio of Amps to Watts of the ECW is generally >1 kA/kW under normal conditions (maximum plasma current: 150 kA, ECW: 140 kW). The experimental results are explained using the stochastic heating model of the asymmetric electron velocity distribution in stochastic electromagnetic waves.

physics.plasm-ph↗

arXiv2021

Observation of a strong correlation between the positive floating potential near the edge and plasma current on EXL-50 ECW plasma

Fully non-inductive plasma current start-up without the central solenoid in ECW plasma was used on EXL-50 Spherical Torus with a weak external vertical field (Bv). Generally, the number of electrons leaving to the vessel wall by the gradient Bt is larger than ions, and the positive potential was built up in plasma. The relationship between floating potential and the plasma current was studied using the Langmuir probes near the boundary. The results show that the floating potential is positive (about 200V) and has a strong correlation with plasma current. In open magnetic field, the plasma current is driven by the high energy electrons in preferential confinement, the plasma current and potential approximately positively correlated with total electron density. After forming the closed flux surface, the plasma current consists mainly of the ECW driven current, and potential is negatively correlated with plasma current. By actively adjusting the Bv, it demonstrated that the positive voltage is approximately inversely correlated with the Bv and plasma current (Ip). Considering that the plasma temperature near the boundary is quite low (~eV), the positive voltage near the boundary caused by the high-energy electron loss. Therefore, the measurements of the boundary potential are important for the study of high-energy electron confinement performance, noninductive plasma current start-up and current driven.

physics.plasm-ph↗

arXiv2021

A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining

Classifying and resolving coreferences of objects (e.g., product names) and attributes (e.g., product aspects) in opinionated reviews is crucial for improving the opinion mining performance. However, the task is challenging as one often needs to consider domain-specific knowledge (e.g., iPad is a tablet and has aspect resolution) to identify coreferences in opinionated reviews. Also, compiling a handcrafted and curated domain-specific knowledge base for each domain is very time consuming and arduous. This paper proposes an approach to automatically mine and leverage domain-specific knowledge for classifying objects and attribute coreferences. The approach extracts domain-specific knowledge from unlabeled review data and trains a knowledgeaware neural coreference classification model to leverage (useful) domain knowledge together with general commonsense knowledge for the task. Experimental evaluation on realworld datasets involving five domains (product types) shows the effectiveness of the approach.

cs.CL↗

arXiv2021

Detecting Changed-Hands Online Review Accounts

A reputable social media or review account can be a good cover for spamming activities. It has become prevalent that spammers buy/sell such accounts openly on the Web. We call these sold/bought accounts the changed-hands (CH) accounts. They are hard to detect by existing spam detection algorithms as their spamming activities are under the disguise of clean histories. In this paper, we first propose the problem of detecting CH accounts, and then design an effective detection algorithm which exploits changes in content and writing styles of individual accounts, and a proposed novel feature selection method that works at a fine-grained level within each individual account. The proposed method not only determines if an account has changed hands, but also pinpoints the change point. Experimental results with online review accounts demonstrate the high effectiveness of our approach.

cs.SI↗

arXiv2021

Discovery of carbon-based strongest and hardest amorphous material

Carbon is likely the most fascinating element of the periodic table because of the diversity of its allotropes stemming from its variable (sp, sp2, and sp3) bonding motifs. Exploration of new forms of carbon has been an eternal theme of contemporary scientific research. Here we report on novel amorphous carbon phases containing high fraction of sp3 bonded atoms recovered after compressing fullerene C60 to previously unexplored high pressure and temperature. The synthesized carbons are the hardest and strongest amorphous materials known to date, capable of scratching diamond crystal and approaching its strength which is evidenced by complimentary mechanical tests. Photoluminescence and absorption spectra of the materials demonstrate they are semiconductors with tunable bandgaps in the range of 1.5-2.2 eV, comparable to that of amorphous silicon. A remarkable combination of the outstanding mechanical and electronic properties makes this class of amorphous carbons an excellent candidate for photovoltaic applications demanding ultrahigh strength and wear resistance.

cond-mat.mtrl-sci↗

arXiv2021

The low cosmic-ray density in Polaris Flare

We reported the gamma-ray observation towards the giant molecular cloud Polaris Flare. Together with the dust column density map, we derived the cosmic ray density and spectrum in this cloud. Compared with the CR measured locally, the CR density in Polaris Flare is significantly lower and the spectrum is softer. Such a different CR spectrum reveals either a rather large gradient of CR distribution in the direction perpendicular to the Galactic plane or a suppression of CR inside molecular clouds.

astro-ph.HE↗

arXiv2021

Narrow-gap Semiconducting Superhard Amorphous Carbon with Superior Toughness

New carbon forms exhibiting extraordinary physico-chemical properties can be generated from nanostructured precursors under extreme pressure. Nevertheless, synthesis of such fascinating materials is often not well understood that results, as is the case of C60 precursor, in irreproducibility of the results and impeding further progress in the materials design. Here the semiconducting amorphous carbon having bandgaps of 0.1-0.3 eV and the advantages of isotropic superhardness and superior toughness over single-crystal diamond and inorganic glasses are produced from transformation of fullerene at high pressure and moderate temperatures. A systematic investigation of the structure and bonding evolution was carried out by using rich arsenal of complimentary characterization methods, which helps to build a model of the transformation that can be used in further high p,T synthesis of novel nanocarbon systems for advanced applications. The produced amorphous carbon materials have the potential of demanding optoelectronic applications that diamond and graphene cannot achieve

cond-mat.mtrl-sci↗

arXiv2021

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking

Zero-shot cross-domain dialogue state tracking (DST) enables us to handle task-oriented dialogue in unseen domains without the expense of collecting in-domain data. In this paper, we propose a slot description enhanced generative approach for zero-shot cross-domain DST. Specifically, our model first encodes dialogue context and slots with a pre-trained self-attentive encoder, and generates slot values in an auto-regressive manner. In addition, we incorporate Slot Type Informed Descriptions that capture the shared information across slots to facilitate cross-domain knowledge transfer. Experimental results on the MultiWOZ dataset show that our proposed method significantly improves existing state-of-the-art results in the zero-shot cross-domain setting.

cs.CL↗