arXiv subjects

Yiming Yang

Publications and source records attributed to Yiming Yang.

216 records · Page 12Linked to original sources

arXiv2018

Asymmetric Variational Autoencoders

Variational inference for latent variable models is prevalent in various machine learning problems, typically solved by maximizing the Evidence Lower Bound (ELBO) of the true data likelihood with respect to a variational distribution. However, freely enriching the family of variational distribution is challenging since the ELBO requires variational likelihood evaluations of the latent variables. In this paper, we propose a novel framework to enrich the variational family by incorporating auxiliary variables to the variational family. The resulting inference network doesn't require density evaluations for the auxiliary variables and thus complex implicit densities over the auxiliary variables can be constructed by neural networks. It can be shown that the actual variational posterior of the proposed approach is essentially modeling a rich probabilistic mixture of simple variational posterior indexed by auxiliary variables, thus a flexible inference model can be built. Empirical evaluations on several density estimation tasks demonstrates the effectiveness of the proposed method.

cs.LG↗

arXiv2018

Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data

How to model distribution of sequential data, including but not limited to speech and human motions, is an important ongoing research problem. It has been demonstrated that model capacity can be significantly enhanced by introducing stochastic latent variables in the hidden states of recurrent neural networks. Simultaneously, WaveNet, equipped with dilated convolutions, achieves astonishing empirical performance in natural speech generation task. In this paper, we combine the ideas from both stochastic latent variables and dilated convolutions, and propose a new architecture to model sequential data, termed as Stochastic WaveNet, where stochastic latent variables are injected into the WaveNet structure. We argue that Stochastic WaveNet enjoys powerful distribution modeling capacity and the advantage of parallel training from dilated convolutions. In order to efficiently infer the posterior distribution of the latent variables, a novel inference network structure is designed based on the characteristics of WaveNet architecture. State-of-the-art performances on benchmark datasets are obtained by Stochastic WaveNet on natural speech modeling and high quality human handwriting samples can be generated as well.

cs.LG↗

arXiv2018

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. Temporal data arise in these real-world applications often involves a mixture of long-term and short-term patterns, for which traditional approaches such as Autoregressive models and Gaussian Process may fail. In this paper, we proposed a novel deep learning framework, namely Long- and Short-term Time-series network (LSTNet), to address this open challenge. LSTNet uses the Convolution Neural Network (CNN) and the Recurrent Neural Network (RNN) to extract short-term local dependency patterns among variables and to discover long-term patterns for time series trends. Furthermore, we leverage traditional autoregressive model to tackle the scale insensitive problem of the neural network model. In our evaluation on real-world data with complex mixtures of repetitive patterns, LSTNet achieved significant performance improvements over that of several state-of-the-art baseline methods. All the data and experiment codes are available online.

cs.LG↗

arXiv2018

Cross-lingual Distillation for Text Classification

Cross-lingual text classification(CLTC) is the task of classifying documents written in different languages into the same taxonomy of categories. This paper presents a novel approach to CLTC that builds on model distillation, which adapts and extends a framework originally proposed for model compression. Using soft probabilistic predictions for the documents in a label-rich language as the (induced) supervisory labels in a parallel corpus of documents, we train classifiers successfully for new languages in which labeled training data are not available. An adversarial feature adaptation technique is also applied during the model training to reduce distribution mismatch. We conducted experiments on two benchmark CLTC datasets, treating English as the source language and German, French, Japan and Chinese as the unlabeled target languages. The proposed approach had the advantageous or comparable performance of the other state-of-art methods.

cs.CL↗

arXiv2018

Fabrication and Characterization of Superconducting Quantum Interference Device using (Bi_{1-x}Sb_x)_2Se_3 Topological Insulator Nanoribbons

We report on the fabrication and electrical transport properties of superconducting quantum interference devices (SQUIDs) made from a (Bi_{1-x}Sb_x)_2Se_3 topological insulator (TI) nanoribbon (NR) connected with Pb0.5In0.5 superconducting electrodes. Below the transition temperature of the superconducting Pb0.5In0.5 electrodes, periodic oscillations of the critical current are observed in the TI NR SQUID under a magnetic field applied perpendicular to the plane owing to flux quantization. Also the output voltage modulates as a function of the external magnetic field. Moreover, the SQUID the SQUID shows a voltage modulation envelope, which is considered to represent the Fraunhofer-like patterns of each single junction. These properties of the TI NR SQUID would provide a useful method to explore Majorana fermions.

cond-mat.supr-con↗

arXiv2017

RACE: Large-scale ReAding Comprehension Dataset From Examinations

We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task. Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students' ability in understanding and reasoning. In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models (43%) and the ceiling human performance (95%). We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension. The dataset is freely available at http://www.cs.cmu.edu/~glai1/data/race/ and the code is available at https://github.com/qizhex/RACE_AR_baselines.

cs.CL↗

arXiv2017

MMD GAN: Towards Deeper Understanding of Moment Matching Network

Generative moment matching network (GMMN) is a deep generative model that differs from Generative Adversarial Network (GAN) by replacing the discriminator in GAN with a two-sample test based on kernel maximum mean discrepancy (MMD). Although some theoretical guarantees of MMD have been studied, the empirical performance of GMMN is still not as competitive as that of GAN on challenging and large benchmark datasets. The computational efficiency of GMMN is also less desirable in comparison with GAN, partially due to its requirement for a rather large batch size during the training. In this paper, we propose to improve both the model expressiveness of GMMN and its computational efficiency by introducing adversarial kernel learning techniques, as the replacement of a fixed Gaussian kernel in the original GMMN. The new approach combines the key ideas in both GMMN and GAN, hence we name it MMD GAN. The new distance measure in MMD GAN is a meaningful loss that enjoys the advantage of weak topology and can be optimized via gradient descent with relatively small batch sizes. In our evaluation on multiple benchmark datasets, including MNIST, CIFAR- 10, CelebA and LSUN, the performance of MMD-GAN significantly outperforms GMMN, and is competitive with other representative GAN works.

cs.LG↗

arXiv2017

Analogical Inference for Multi-Relational Embeddings

Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs. An effective and scalable solution for this problem is crucial for the true success of knowledge-based inference in a broad range of applications. This paper proposes a novel framework for optimizing the latent representations with respect to the \textit{analogical} properties of the embedded entities and relations. By formulating the learning objective in a differentiable fashion, our model enjoys both theoretical power and computational scalability, and significantly outperformed a large number of representative baseline methods on benchmark datasets. Furthermore, the model offers an elegant unification of several well-known methods in multi-relational embedding, which can be proven to be special instantiations of our framework.

cs.LG↗

arXiv2017

Data-driven Random Fourier Features using Stein Effect

Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang et al., 2014]. A limitation of the current approaches is that all the features receive an equal weight summing to 1. In this paper, we propose a novel shrinkage estimator from "Stein effect", which provides a data-driven weighting strategy for random features and enjoys theoretical justifications in terms of lowering the empirical risk. We further present an efficient randomized algorithm for large-scale applications of the proposed method. Our empirical results on six benchmark data sets demonstrate the advantageous performance of this approach over representative baselines in both kernel approximation and supervised learning tasks.

cs.LG↗

arXiv2017

Co-Clustering for Multitask Learning

This paper presents a new multitask learning framework that learns a shared representation among the tasks, incorporating both task and feature clusters. The jointly-induced clusters yield a shared latent subspace where task relationships are learned more effectively and more generally than in state-of-the-art multitask learning methods. The proposed general framework enables the derivation of more specific or restricted state-of-the-art multitask methods. The paper also proposes a highly-scalable multitask learning algorithm, based on the new framework, using conjugate gradient descent and generalized \textit{Sylvester equations}. Experimental results on synthetic and benchmark datasets show that the proposed method systematically outperforms several state-of-the-art multitask learning methods.

stat.ML↗

arXiv2016

Scaling Sampling-based Motion Planning to Humanoid Robots

Planning balanced and collision-free motion for humanoid robots is non-trivial, especially when they are operated in complex environments, such as reaching targets behind obstacles or through narrow passages. We propose a method that allows us to apply existing sampling--based algorithms to plan trajectories for humanoids by utilizing a customized state space representation, biased sampling strategies, and a steering function based on a robust inverse kinematics solver. Our approach requires no prior offline computation, thus one can easily transfer the work to new robot platforms. We tested the proposed method solving practical reaching tasks on a 38 degrees-of-freedom humanoid robot, NASA Valkyrie, showing that our method is able to generate valid motion plans that can be executed on advanced full-size humanoid robots. We also present a benchmark between different motion planning algorithms evaluated on a variety of reaching motion problems. This allows us to find suitable algorithms for solving humanoid motion planning problems, and to identify the limitations of these algorithms.

cs.RO↗

arXiv2016

iDRM: Humanoid Motion Planning with Real-Time End-Pose Selection in Complex Environments

In this paper, we propose a novel inverse Dynamic Reachability Map (iDRM) that allows a floating base system to find valid end-poses in complex and dynamically changing environments in real-time. End-pose planning for valid stance pose and collision-free configuration is an essential problem for humanoid applications, such as providing goal states for walking and motion planners. However, this is non-trivial in complex environments, where standing locations and reaching postures are restricted by obstacles. Our proposed iDRM customizes the robot-to-workspace occupation list and uses an online update algorithm to enable efficient reconstruction of the reachability map to guarantee that the selected end-poses are always collision-free. The iDRM was evaluated in a variety of reaching tasks using the 38 degree-of-freedom (DoF) humanoid robot Valkyrie. Our results show that the approach is capable of finding valid end-poses in a fraction of a second. Significantly, we also demonstrate that motion planning algorithms integrating our end-pose planning method are more efficient than those not utilizing this technique.

cs.RO↗

arXiv2016

Very Strong Superconducting Proximity Effects in PbS Semiconductor Nanowires

We report the fabrication of strongly coupled nanohybrid superconducting junctions using PbS semiconductor nanowires and Pb0.5In0.5 superconducting electrodes. The maximum supercurrent in the junction reaches up to ~15 μA at 0.3 K, which is the highest value ever observed in semiconductor-nanowire-based superconducting junctions. The observation of microwave-induced constant voltage steps confirms the existence of genuine Josephson coupling through the nanowire. Monotonic suppression of the critical current under an external magnetic field is also in good agreement with the narrow junction model. The temperature-dependent stochastic distribution of the switching current exhibits a crossover from phase diffusion to a thermal activation process as the temperature decreases. These strongly coupled nanohybrid superconducting junctions would be advantageous to the development of gate-tunable superconducting quantum information devices.

cond-mat.supr-con↗

arXiv2016

Cross-Graph Learning of Multi-Relational Associations

Cross-graph Relational Learning (CGRL) refers to the problem of predicting the strengths or labels of multi-relational tuples of heterogeneous object types, through the joint inference over multiple graphs which specify the internal connections among each type of objects. CGRL is an open challenge in machine learning due to the daunting number of all possible tuples to deal with when the numbers of nodes in multiple graphs are large, and because the labeled training instances are extremely sparse as typical. Existing methods such as tensor factorization or tensor-kernel machines do not work well because of the lack of convex formulation for the optimization of CGRL models, the poor scalability of the algorithms in handling combinatorial numbers of tuples, and/or the non-transductive nature of the learning methods which limits their ability to leverage unlabeled data in training. This paper proposes a novel framework which formulates CGRL as a convex optimization problem, enables transductive learning using both labeled and unlabeled tuples, and offers a scalable algorithm that guarantees the optimal solution and enjoys a linear time complexity with respect to the sizes of input graphs. In our experiments with a subset of DBLP publication records and an Enzyme multi-source dataset, the proposed method successfully scaled to the large cross-graph inference problem, and outperformed other representative approaches significantly.

cs.LG↗

arXiv2016

Spin Generation Via Bulk Spin Current in Three Dimensional Topological Insulators

To date, spin generation in three-dimensional topological insulators is primarily modeled as a single-surface phenomenon, attributed to the momentum-spin locking on each individual surface. In this article we propose a mechanism of spin generation where the role of the insulating yet topologically non-trivial bulk becomes explicit: an external electric field creates a transverse pure spin current through the bulk of a three-dimensional topological insulator, which transports spins between the top and bottom surfaces. Under sufficiently high surface disorder, the spin relaxation time can be extended via the Dyakonov-Perel mechanism. Consequently both the spin generation efficiency and surface conductivity are largely enhanced. Numerical simulation confirms that this spin generation mechanism originates from the unique topological connection of the top and bottom surfaces and is absent in other two dimensional systems such as graphene, even though they possess a similar Dirac cone-type dispersion.

cond-mat.mtrl-sci↗

arXiv2015

Gate-tunable superconducting quantum interference devices of PbS nanowires

We report on the fabrication and electrical transport properties of gate-tunable superconducting quantum interference devices (SQUIDs), made of semiconducting PbS nanowire contacted with PbIn superconducting electrodes. Applied with a magnetic field perpendicular to the plane of the nano-hybrid SQUID, periodic oscillations of the critical current due to the flux quantization in SQUID are observed up to T = 4.0 K. Nonsinusoidal current-phase relationship is obtained as a function of temperature and gate voltage, which is consistent with a short and diffusive junction model.

cond-mat.mes-hall↗

arXiv2015

Hot Carrier Trapping Induced Negative Photoconductance in InAs Nanowires toward Novel Nonvolatile Memory

We report a novel negative photoconductivity (NPC) mechanism in n-type indium arsenide nanowires (NWs). Photoexcitation significantly suppresses the conductivity with a gain up to 10^5. The origin of NPC is attributed to the depletion of conduction channels by light assisted hot electron trapping, supported by gate voltage threshold shift and wavelength dependent photoconductance measurements. Scanning photocurrent microscopy excludes the possibility that NPC originates from the NW/metal contacts and reveals a competing positive photoconductivity. The conductivity recovery after illumination substantially slows down at low temperature, indicating a thermally activated detrapping mechanism. At 78 K, the spontaneous recovery of the conductance is completely quenched, resulting in a reversible memory device which can be switched by light and gate voltage pulses. The novel NPC based optoelectronics may find exciting applications in photodetection and nonvolatile memory with low power consumption.

cond-mat.mes-hall↗

arXiv2015

Broadband Quantum Efficiency Enhancement in High Index Nanowires Resonators

Light trapping in sub-wavelength semiconductor nanowires (NWs) offers a promising approach to simultaneously reducing material consumption and enhancing photovoltaic performance. Nevertheless, the absorption efficiency of a NW, defined by the ratio of optical absorption cross section to the NW diameter, lingers around 1 in existing NW photonic devices, and the absorption enhancement suffers from a narrow spectral width. Here, we show that the absorption efficiency can be significantly improved in NWs with higher refractive indices, by an experimental observation of up to 350% external quantum efficiency (EQE) in lead sulfide (PbS) NW resonators, a 3-fold increase compared to Si NWs. Furthermore, broadband absorption enhancement is achieved in single tapered NWs, where light of various wavelengths is absorbed at segments with different diameters analogous to a tandem solar cell. Overall, the single NW Schottky junction solar cells benefit from optical resonance, near bandgap open circuit voltage, and long minority carrier diffusion length, demonstrating power conversion efficiency (PCE) comparable to single Si NW coaxial p-n junction cells11, but with much simpler fabrication processes.

cond-mat.mes-hall↗