arXiv subjects

Zheng Zhang

Publications and source records attributed to Zheng Zhang.

At least 577 records · Page 32Linked to original sources

arXiv2020

Parametric Instance Classification for Unsupervised Visual Feature Learning

This paper presents parametric instance classification (PIC) for unsupervised visual feature learning. Unlike the state-of-the-art approaches which do instance discrimination in a dual-branch non-parametric fashion, PIC directly performs a one-branch parametric instance classification, revealing a simple framework similar to supervised classification and without the need to address the information leakage issue. We show that the simple PIC framework can be as effective as the state-of-the-art approaches, i.e. SimCLR and MoCo v2, by adapting several common component settings used in the state-of-the-art approaches. We also propose two novel techniques to further improve effectiveness and practicality of PIC: 1) a sliding-window data scheduler, instead of the previous epoch-based data scheduler, which addresses the extremely infrequent instance visiting issue in PIC and improves the effectiveness; 2) a negative sampling and weight update correction approach to reduce the training time and GPU memory consumption, which also enables application of PIC to almost unlimited training images. We hope that the PIC framework can serve as a simple baseline to facilitate future study.

cs.CV↗

arXiv2020

Recent Advances and Challenges in Task-oriented Dialog System

Due to the significance and value in human-computer interaction and natural language processing, task-oriented dialog systems are attracting more and more attention in both academic and industrial communities. In this paper, we survey recent advances and challenges in task-oriented dialog systems. We also discuss three critical topics for task-oriented dialog systems: (1) improving data efficiency to facilitate dialog modeling in low-resource settings, (2) modeling multi-turn dynamics for dialog policy learning to achieve better task-completion performance, and (3) integrating domain ontology knowledge into the dialog model. Besides, we review the recent progresses in dialog evaluation and some widely-used corpora. We believe that this survey, though incomplete, can shed a light on future research in task-oriented dialog systems.

cs.CL↗

arXiv2020

Dual-level Semantic Transfer Deep Hashing for Efficient Social Image Retrieval

Social network stores and disseminates a tremendous amount of user shared images. Deep hashing is an efficient indexing technique to support large-scale social image retrieval, due to its deep representation capability, fast retrieval speed and low storage cost. Particularly, unsupervised deep hashing has well scalability as it does not require any manually labelled data for training. However, owing to the lacking of label guidance, existing methods suffer from severe semantic shortage when optimizing a large amount of deep neural network parameters. Differently, in this paper, we propose a Dual-level Semantic Transfer Deep Hashing (DSTDH) method to alleviate this problem with a unified deep hash learning framework. Our model targets at learning the semantically enhanced deep hash codes by specially exploiting the user-generated tags associated with the social images. Specifically, we design a complementary dual-level semantic transfer mechanism to efficiently discover the potential semantics of tags and seamlessly transfer them into binary hash codes. On the one hand, instance-level semantics are directly preserved into hash codes from the associated tags with adverse noise removing. Besides, an image-concept hypergraph is constructed for indirectly transferring the latent high-order semantic correlations of images and tags into hash codes. Moreover, the hash codes are obtained simultaneously with the deep representation learning by the discrete hash optimization strategy. Extensive experiments on two public social image retrieval datasets validate the superior performance of our method compared with state-of-the-art hashing methods. The source codes of our method can be obtained at https://github.com/research2020-1/DSTDH

cs.IR↗

arXiv2020

Repurpose Open Data to Discover Therapeutics for COVID-19 using Deep Learning

There have been more than 850,000 confirmed cases and over 48,000 deaths from the human coronavirus disease 2019 (COVID-19) pandemic, caused by novel severe acute respiratory syndrome coronavirus (SARS-CoV-2), in the United States alone. However, there are currently no proven effective medications against COVID-19. Drug repurposing offers a promising way for the development of prevention and treatment strategies for COVID-19. This study reports an integrative, network-based deep learning methodology to identify repurposable drugs for COVID-19 (termed CoV-KGE). Specifically, we built a comprehensive knowledge graph that includes 15 million edges across 39 types of relationships connecting drugs, diseases, genes, pathways, and expressions, from a large scientific corpus of 24 million PubMed publications. Using Amazon AWS computing resources, we identified 41 repurposable drugs (including indomethacin, toremifene and niclosamide) whose therapeutic association with COVID-19 were validated by transcriptomic and proteomic data in SARS-CoV-2 infected human cells and data from ongoing clinical trials. While this study, by no means recommends specific drugs, it demonstrates a powerful deep learning methodology to prioritize existing drugs for further investigation, which holds the potential of accelerating therapeutic development for COVID-19.

q-bio.QM↗

arXiv2020

Dense RepPoints: Representing Visual Objects with Dense Point Sets

We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level. Techniques are proposed to efficiently process these dense points, maintaining near-constant complexity with increasing point numbers. Dense RepPoints is shown to represent and learn object segments well, with the use of a novel distance transform sampling method combined with set-to-set supervision. The distance transform sampling combines the strengths of contour and grid representations, leading to performance that surpasses counterparts based on contours or grids. Code is available at \url{https://github.com/justimyhxu/Dense-RepPoints}.

cs.CV↗

arXiv2020

Finite-volume formalism in the $2 \xrightarrow[]{H_I+H_I} 2$ transition: an application to the lattice QCD calculation of double beta decays

We present the formalism for connecting a second-order electroweak $2\xrightarrow[]{H_I+H_I}2$ transition amplitudes in the finite volume (with two hadrons in the initial and final states) to the physical amplitudes in the infinite volume. Our study mainly focus on the case where the low-lying intermediate state consists of two scattering hadrons. As a side product we also reproduce the finite-volume formula for $2\xrightarrow[]{H_I}2$ transition, originally obtained by Briceño and Hansen. With the available finite-volume formalism, we further discuss how to treat with the finite-volume problem in the double beta decays $nn\to pp ee\barν\barν$ and $nn\to pp ee$.

hep-lat↗

arXiv2020

Active Subspace of Neural Networks: Structural Analysis and Universal Attacks

Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each intermediate layer and reduce the number of neurons from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as {ASNet}, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98$\times$ parameter and 7.30$\times$ flops reduction. The universal active subspace attack vector can achieve around 20% higher attack ratio compared with the existing approach in all of our numerical experiments. The PyTorch codes for this paper are available online.

cs.LG↗

arXiv2020

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems

We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems. As the successor of ConvLab (Lee et al., 2019b), ConvLab-2 inherits ConvLab's framework but integrates more powerful dialogue models and supports more datasets. Besides, we have developed an analysis tool and an interactive tool to assist researchers in diagnosing dialogue systems. The analysis tool presents rich statistics and summarizes common mistakes from simulated dialogues, which facilitates error analysis and system improvement. The interactive tool provides a user interface that allows developers to diagnose an assembled dialogue system by interacting with the system and modifying the output of each system component.

cs.CL↗

arXiv2020

Chiral phase transition in a rotating sphere

We study the chiral phase transition of the two-flavor Nambu-Jona-Lasinio (NJL) model in a rotating sphere, which includes both rotation and finite size effects. We find that rotation leads to a suppression of the chiral condensate at a finite temperature, while its effects are smaller than the finite size effects. Our work can be helpful to study the effects relevant to rotation in heavy-ion collisions in a more realistic way.

nucl-th↗

arXiv2020

Chance-Constrained and Yield-aware Optimization of Photonic ICs with Non-Gaussian Correlated Process Variations

Uncertainty quantification has become an efficient tool for uncertainty-aware prediction, but its power in yield-aware optimization has not been well explored from either theoretical or application perspectives. Yield optimization is a much more challenging task. On one side, optimizing the generally non-convex probability measure of performance metrics is difficult. On the other side, evaluating the probability measure in each optimization iteration requires massive simulation data, especially when the process variations are non-Gaussian correlated. This paper proposes a data-efficient framework for the yield-aware optimization of photonic ICs. This framework optimizes the design performance with a yield guarantee, and it consists of two modules: a modeling module that builds stochastic surrogate models for design objectives and chance constraints with a few simulation samples, and a novel yield optimization module that handles probabilistic objectives and chance constraints in an efficient deterministic way. This deterministic treatment avoids repeatedly evaluating probability measures at each iteration, thus it only requires a few simulations in the whole optimization flow. We validate the accuracy and efficiency of the whole framework by a synthetic example and two photonic ICs. Our optimization method can achieve more than $30\times$ reduction of simulation cost and better design performance on the test cases compared with a Bayesian yield optimization approach developed recently.

math.OC↗

arXiv2020

Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness

Most existing approaches for goal-oriented dialogue policy learning used reinforcement learning, which focuses on the target agent policy and simply treat the opposite agent policy as part of the environment. While in real-world scenarios, the behavior of an opposite agent often exhibits certain patterns or underlies hidden policies, which can be inferred and utilized by the target agent to facilitate its own decision making. This strategy is common in human mental simulation by first imaging a specific action and the probable results before really acting it. We therefore propose an opposite behavior aware framework for policy learning in goal-oriented dialogues. We estimate the opposite agent's policy from its behavior and use this estimation to improve the target agent by regarding it as part of the target policy. We evaluate our model on both cooperative and competitive dialogue tasks, showing superior performance over state-of-the-art baselines.

cs.CL↗

arXiv2020

DGL-KE: Training Knowledge Graph Embeddings at Scale

Knowledge graphs have emerged as a key abstraction for organizing information in diverse domains and their embeddings are increasingly used to harness their information in various information retrieval and machine learning tasks. However, the ever growing size of knowledge graphs requires computationally efficient algorithms capable of scaling to graphs with millions of nodes and billions of edges. This paper presents DGL-KE, an open-source package to efficiently compute knowledge graph embeddings. DGL-KE introduces various novel optimizations that accelerate training on knowledge graphs with millions of nodes and billions of edges using multi-processing, multi-GPU, and distributed parallelism. These optimizations are designed to increase data locality, reduce communication overhead, overlap computations with memory accesses, and achieve high operation efficiency. Experiments on knowledge graphs consisting of over 86M nodes and 338M edges show that DGL-KE can compute embeddings in 100 minutes on an EC2 instance with 8 GPUs and 30 minutes on an EC2 cluster with 4 machines with 48 cores/machine. These results represent a 2x~5x speedup over the best competing approaches. DGL-KE is available on https://github.com/awslabs/dgl-ke.

cs.DC↗

arXiv2020

Negative Margin Matters: Understanding Margin in Few-shot Classification

This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and achieves state-of-the-art accuracy on three standard few-shot classification benchmarks with few bells and whistles. These results are contrary to the common practice in the metric learning field, that the margin is zero or positive. To understand why the negative margin loss performs well for the few-shot classification, we analyze the discriminability of learned features w.r.t different margins for training and novel classes, both empirically and theoretically. We find that although negative margin reduces the feature discriminability for training classes, it may also avoid falsely mapping samples of the same novel class to multiple peaks or clusters, and thus benefit the discrimination of novel classes. Code is available at https://github.com/bl0/negative-margin.few-shot.

cs.CV↗

arXiv2020

Pressure induced metallization and possible unconventional superconductivity in spin liquid $NaYbSe_{2}$

Beyond the conventional electron pairing mediated by phonons, high-temperature superconductivity in cuprates is believed to stem from quantum spin liquid (QSL). The unconventional superconductivity by doping a spin liquid/Mott insulator, is a long-sought goal but a principal challenge in condensed matter physics because of the lack of an ideal QSL platform. Here we report the pressure induced metallization and possible unconventional superconductivity in $NaYbSe_{2}$, which belongs to a large and ideal family of triangular lattice spin liquid we revealed recently and is evidenced to possess a QSL ground state. The charge gap of NaYbSe2 is gradually reduced by applying pressures, and at ~20 GPa the crystal jumps into a superconducting (SC) phase with Tc ~ 5.8 K even before the insulating gap is completely closed. The metallization is confirmed by further high-pressure experiments but the sign of superconductivity is not well repeated. No symmetry breaking accompanies the SC transition, as indicated by X-ray diffraction and low-temperature Raman experiments under high pressures. This intrinsically connects QSL and SC phases, and suggests an unconventional superconductivity developed from QSL. We further observed the magnetic-field-tuned superconductor-insulator transition which is analogous to that found in the underdoped cuprate superconductor $La_{2-x}Sr_{x}CuO_{4}$. The study is expected to inspire interest in exploring new types of superconductors and sheds light into the intriguing physics from a spin liquid/Mott insulator to a superconductor.

cond-mat.supr-con↗

arXiv2020

CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset

To advance multi-domain (cross-domain) dialogue modeling as well as alleviate the shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first large-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dataset. It contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains rich annotation of dialogue states and dialogue acts at both user and system sides. About 60% of the dialogues have cross-domain user goals that favor inter-domain dependency and encourage natural transition across domains in conversation. We also provide a user simulator and several benchmark models for pipelined task-oriented dialogue systems, which will facilitate researchers to compare and evaluate their models on this corpus. The large size and rich annotation of CrossWOZ make it suitable to investigate a variety of tasks in cross-domain dialogue modeling, such as dialogue state tracking, policy learning, user simulation, etc.

cs.CL↗

arXiv2020

Crystalline Electric-Field Excitations in Quantum Spin Liquids Candidate $NaYbSe_{2}$

Very recently we revealed a large family of triangular lattice quantum spin liquid candidates named rare-earth chalcogenides, which features a high-symmetry structure without structural/charge disorders and spin impurities, and may serve as an ideal platform exploring spin liquid physics. The knowledge of crystalline electric-field (CEF) excitations is an essential step to explore the fundamental magnetism of rare-earth spin systems. Here we employed inelastic neutron scattering (INS) and Raman scattering (RS) to carry out a comprehensive CFE investigation on $NaYbSe_{2}$, a promising representative of the family. By comparison with its nonmagnetic compound $NaLuSe_{2}$, we are able to identify the CEF excitations at 15.8, 24.3 and 30.5 meV at 5K. The selected cuts of the INS spectra are well re-produced with a large anisotropy of $g$ factors ($g_{ab}:g_{c}\sim3:1$). Further, the CEF excitations are explained well by our calculations based on the point charge model. Interestingly, $NaYbSe_{2}$ exhibits an unusual CEF shift to higher energies with increasing temperatures, and the Raman mode close to the first CEF excitation shows an anomalously large softening with decreasing temperatures. The absence of the anomalies in $NaLuSe_{2}$ clearly demonstrates a CEF-phonon coupling not reported in the family. It can be understood in term of the weaker electronegativity of Se. The fact that the smallest first CEF excitation in the sub-family of $NaYbCh_{2}$ is $\sim$ 180K (Ch=O, S, Se), guarantees that the sub-family can be strictly described with an effective S=1/2 picture at sufficiently low temperatures. Interestingly the CEF-phonon coupling revealed here may present alternative possibilities to manipulate the spin systems.

cond-mat.str-el↗

arXiv2020

Transformer on a Diet

Transformer has been widely used thanks to its ability to capture sequence information in an efficient way. However, recent developments, such as BERT and GPT-2, deliver only heavy architectures with a focus on effectiveness. In this paper, we explore three carefully-designed light Transformer architectures to figure out whether the Transformer with less computations could produce competitive results. Experimental results on language model benchmark datasets hint that such trade-off is promising, and the light Transformer reduces 70% parameters at best, while obtains competitive perplexity compared to standard Transformer. The source code is publicly available.

cs.CL↗

arXiv2020

Nambu-Jona-Lasinio model in a sphere

We study the chiral phase transition of the two flavor Nambu-Jona-Lasinio (NJL) model in a sphere with the MIT boundary condition. We find that the MIT boundary condition results in much stronger finite size effects than the antiperiodic boundary condition. Our work may be helpful to study the finite size effects in heavy-ion collision in a more realistic way.

hep-ph↗