arXiv subjects

Wei Hu

Publications and source records attributed to Wei Hu.

At least 325 records · Page 18Linked to original sources

arXiv2018

Recurrent Skipping Networks for Entity Alignment

We consider the problem of learning knowledge graph (KG) embeddings for entity alignment (EA). Current methods use the embedding models mainly focusing on triple-level learning, which lacks the ability of capturing long-term dependencies existing in KGs. Consequently, the embedding-based EA methods heavily rely on the amount of prior (known) alignment, due to the identity information in the prior alignment cannot be efficiently propagated from one KG to another. In this paper, we propose RSN4EA (recurrent skipping networks for EA), which leverages biased random walk sampling for generating long paths across KGs and models the paths with a novel recurrent skipping network (RSN). RSN integrates the conventional recurrent neural network (RNN) with residual learning and can largely improve the convergence speed and performance with only a few more parameters. We evaluated RSN4EA on a series of datasets constructed from real-world KGs. Our experimental results showed that it outperformed a number of state-of-the-art embedding-based EA methods and also achieved comparable performance for KG completion.

cs.CL↗

arXiv2018

Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced

We study the implicit regularization imposed by gradient descent for learning multi-layer homogeneous functions including feed-forward fully connected and convolutional deep neural networks with linear, ReLU or Leaky ReLU activation. We rigorously prove that gradient flow (i.e. gradient descent with infinitesimal step size) effectively enforces the differences between squared norms across different layers to remain invariant without any explicit regularization. This result implies that if the weights are initially small, gradient flow automatically balances the magnitudes of all layers. Using a discretization argument, we analyze gradient descent with positive step size for the non-convex low-rank asymmetric matrix factorization problem without any regularization. Inspired by our findings for gradient flow, we prove that gradient descent with step sizes $η_t = O\left(t^{-\left( \frac12+δ\right)} \right)$ ($0<δ\le\frac12$) automatically balances two low-rank factors and converges to a bounded global optimum. Furthermore, for rank-$1$ asymmetric matrix factorization we give a finer analysis showing gradient descent with constant step size converges to the global minimum at a globally linear rate. We believe that the idea of examining the invariance imposed by first order algorithms in learning homogeneous models could serve as a fundamental building block for studying optimization for learning deep models.

cs.LG↗

arXiv2018

Light Reconfigurable Geometric Phase Optical Element with Multi-stable States

We present the design methodology of a light reconfigurable geometric phase optical element with multi-stable diffraction efficiency states, enabled by a photoresponsive self-organized chiral liquid crystal. Experimental demonstration shows the device exhibits a broad diffraction efficiency tunable range that can be smoothly modulated under alternate stimulation of ultraviolet and green lights. Distinctive to previous designs, the regulation of diffraction efficiency fundamentally stems from the modulation of geometric phase together with dynamical phase retardation, and any intermediate diffractive state is memorized. Such multi-stability facilitates applications including energy-saving all-optical signal processing in classical and quantum level, and phase hologram for anti-counterfeit.

physics.optics↗

arXiv2018

Local Frequency Interpretation and Non-Local Self-Similarity on Graph for Point Cloud Inpainting

As 3D scanning devices and depth sensors mature, point clouds have attracted increasing attention as a format for 3D object representation, with applications in various fields such as tele-presence, navigation and heritage reconstruction. However, point clouds usually exhibit holes of missing data, mainly due to the limitation of acquisition techniques and complicated structure. Further, point clouds are defined on irregular non-Euclidean domains, which is challenging to address especially with conventional signal processing tools. Hence, leveraging on recent advances in graph signal processing, we propose an efficient point cloud inpainting method, exploiting both the local smoothness and the non-local self-similarity in point clouds. Specifically, we first propose a frequency interpretation in graph nodal domain, based on which we introduce the local graph-signal smoothness prior in order to describe the local smoothness of point clouds. Secondly, we explore the characteristics of non-local self-similarity, by globally searching for the most similar area to the missing region. The similarity metric between two areas is defined based on the direct component and the anisotropic graph total variation of normals in each area. Finally, we formulate the hole-filling step as an optimization problem based on the selected most similar area and regularized by the graph-signal smoothness prior. Besides, we propose voxelization and automatic hole detection methods for the point cloud prior to inpainting. Experimental results show that the proposed approach outperforms four competing methods significantly, both in objective and subjective quality.

cs.CV↗

arXiv2018

TreeSegNet: Adaptive Tree CNNs for Subdecimeter Aerial Image Segmentation

For the task of subdecimeter aerial imagery segmentation, fine-grained semantic segmentation results are usually difficult to obtain because of complex remote sensing content and optical conditions. Recently, convolutional neural networks (CNNs) have shown outstanding performance on this task. Although many deep neural network structures and techniques have been applied to improve the accuracy, few have paid attention to better differentiating the easily confused classes. In this paper, we propose TreeSegNet which adopts an adaptive network to increase the classification rate at the pixelwise level. Specifically, based on the infrastructure of DeepUNet, a Tree-CNN block in which each node represents a ResNeXt unit is constructed adaptively according to the confusion matrix and the proposed TreeCutting algorithm. By transporting feature maps through concatenating connections, the Tree-CNN block fuses multiscale features and learns best weights for the model. In experiments on the ISPRS 2D semantic labeling Potsdam dataset, the results obtained by TreeSegNet are better than those of other published state-of-the-art methods. Detailed comparison and analysis show that the improvement brought by the adaptive Tree-CNN block is significant.

cs.CV↗

arXiv2018

Combinatorial Multi-Armed Bandit with General Reward Functions

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the $\max()$ function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve $O(\log{T})$ distribution-dependent regret and $\tilde{O}(\sqrt{T})$ distribution-independent regret, where $T$ is the time horizon. We apply our results to the $K$-MAX problem and expected utility maximization problems. In particular, for $K$-MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the first $\tilde{O}(\sqrt T)$ bound on the $(1-ε)$-approximation regret of its online problem, for any $ε>0$.

cs.LG↗

arXiv2018

Tunable superconductivity in parent cuprate Pr$_{2}$CuO$_{4\pmδ}$ thin films

In this article, we studied the role of oxygen in Pr$_{2}$CuO$_{4\pmδ}$ thin films fabricated by polymer assisted deposition method. The magnetoresistance and Hall resistivity of Pr$_{2}$CuO$_{4\pmδ}$ samples were systematically investigated. It is found that with decreasing the oxygen content, the low-temperature Hall coefficient ($R_H$) and magnetoresistance change from negative to positive, similar to those with the increase of Ce-doped concentration in R$_{2-x}$Ce$_{x}$CuO$_{4}$ (R= La, Nd, Pr, Sm, Eu). In addition, $T_c$ versus $R_H$ for both Pr$_{1-x}$LaCe$_{x}$CuO$_{4}$ and Pr$_{2}$CuO$_{4\pmδ}$ samples can coincide with each other. We conclude that the doped electrons induced by the oxygen removal are responsible for the superconductivity of $T^\prime$-phase parent compounds.

cond-mat.supr-con↗

arXiv2018

RGCNN: Regularized Graph CNN for Point Cloud Segmentation

Point cloud, an efficient 3D object representation, has become popular with the development of depth sensing and 3D laser scanning techniques. It has attracted attention in various applications such as 3D tele-presence, navigation for unmanned vehicles and heritage reconstruction. The understanding of point clouds, such as point cloud segmentation, is crucial in exploiting the informative value of point clouds for such applications. Due to the irregularity of the data format, previous deep learning works often convert point clouds to regular 3D voxel grids or collections of images before feeding them into neural networks, which leads to voluminous data and quantization artifacts. In this paper, we instead propose a regularized graph convolutional neural network (RGCNN) that directly consumes point clouds. Leveraging on spectral graph theory, we treat features of points in a point cloud as signals on graph, and define the convolution over graph by Chebyshev polynomial approximation. In particular, we update the graph Laplacian matrix that describes the connectivity of features in each layer according to the corresponding learned features, which adaptively captures the structure of dynamic graphs. Further, we deploy a graph-signal smoothness prior in the loss function, thus regularizing the learning process. Experimental results on the ShapeNet part dataset show that the proposed approach significantly reduces the computational complexity while achieving competitive performance with the state of the art. Also, experiments show RGCNN is much more robust to both noise and point cloud density in comparison with other methods. We further apply RGCNN to point cloud classification and achieve competitive results on ModelNet40 dataset.

cs.CV↗

arXiv2018

An Analysis of the t-SNE Algorithm for Data Visualization

A first line of attack in exploratory data analysis is data visualization, i.e., generating a 2-dimensional representation of data that makes clusters of similar points visually identifiable. Standard Johnson-Lindenstrauss dimensionality reduction does not produce data visualizations. The t-SNE heuristic of van der Maaten and Hinton, which is based on non-convex optimization, has become the de facto standard for visualization in a wide range of applications. This work gives a formal framework for the problem of data visualization - finding a 2-dimensional embedding of clusterable data that correctly separates individual clusters to make them visually identifiable. We then give a rigorous analysis of the performance of t-SNE under a natural, deterministic condition on the "ground-truth" clusters (similar to conditions assumed in earlier analyses of clustering) in the underlying data. These are the first provable guarantees on t-SNE for constructing good data visualizations. We show that our deterministic condition is satisfied by considerably general probabilistic generative models for clusterable data such as mixtures of well-separated log-concave distributions. Finally, we give theoretical evidence that t-SNE provably succeeds in partially recovering cluster structure even when the above deterministic condition is not met.

cs.LG↗

arXiv2018

Gorenstein projective bimodules via monomorphism categories and filtration categories

We generalize the monomorphism category from quiver (with monomial relations) to arbitrary finite dimensional algebras by a homological definition. Given two finite dimension algebras $A$ and $B$, we use the special monomorphism category Mon(B, A-Gproj) to describe some Gorenstein projective bimodules over the tensor product of $A$ and $B$. If one of the two algebras is Gorenstein, we give a sufficient and necessary condition for Mon(B, A-Gproj) being the category of all Gorenstein projective bimodules. In addition, If both $A$ and $B$ are Gorenstein, we can describe the category of all Gorenstein projective bimodules via filtration categories. Similarly, in this case, we get the same result for infinitely generated Gorenstein projective bimodules.

math.RT↗

arXiv2018

Online Improper Learning with an Approximation Oracle

We revisit the question of reducing online learning to approximate optimization of the offline problem. In this setting, we give two algorithms with near-optimal performance in the full information setting: they guarantee optimal regret and require only poly-logarithmically many calls to the approximation oracle per iteration. Furthermore, these algorithms apply to the more general improper learning problems. In the bandit setting, our algorithm also significantly improves the best previously known oracle complexity while maintaining the same regret.

cs.LG↗

arXiv2018

SeqFace: Make full use of sequence information for face recognition

Deep convolutional neural networks (CNNs) have greatly improved the Face Recognition (FR) performance in recent years. Almost all CNNs in FR are trained on the carefully labeled datasets containing plenty of identities. However, such high-quality datasets are very expensive to collect, which restricts many researchers to achieve state-of-the-art performance. In this paper, we propose a framework, called SeqFace, for learning discriminative face features. Besides a traditional identity training dataset, the designed SeqFace can train CNNs by using an additional dataset which includes a large number of face sequences collected from videos. Moreover, the label smoothing regularization (LSR) and a new proposed discriminative sequence agent (DSA) loss are employed to enhance discrimination power of deep face features via making full use of the sequence data. Our method achieves excellent performance on Labeled Faces in the Wild (LFW), YouTube Faces (YTF), only with a single ResNet. The code and models are publicly available on-line (https://github.com/huangyangyu/SeqFace).

cs.CV↗

arXiv2018

Distributed consensus of linear MASs with an unknown leader via a predictive extended state observer considering input delay and disturbances

The problem of disturbance rejection/attenuation for constant-input delayed linear multi-agent systems (MASs) with the directed communication topology is tackled in this paper, where a classic model reduction technique is introduced to transform the delayed MAS into the delay-free one. First, when the leader has no control input, a novel adaptive predictive extended state observer (ESO) using only relative state information of neighboring agents is designed to achieve disturbance-rejected consensus tracking. The stabilization analysis is presented via the Lyapunov function and sufficient conditions are derived in terms of linear matrix inequalities. Then the result is extended to the disturbance-attenuated case where the leader has bounded control input which is only known by a portion of followers. Finally, two numerical examples are presented to illustrate the effectiveness of proposed strategies. The main contribution focuses on the design of adaptive predictive ESO protocols with the fully distributed property.

eess.SY↗

arXiv2018

A Consistent Spatially Adaptive Smoothed Particle Hydrodynamics Method for Fluid-Structure Interactions

A new consistent, spatially adaptive, smoothed particle hydrodynamics (SPH) method for Fluid-Structure Interactions (FSI) is presented. The method combines several attributes that have not been simultaneously satisfied by other SPH methods. Specifically, it is second-order convergent; it allows for resolutions spatially adapted with moving (translating and rotating) boundaries of arbitrary geometries; and, it accelerates the FSI solution as the adaptive approach leads to fewer degrees of freedom without sacrificing accuracy. The key ingredients in the method are a consistent discretization of differential operators, a \textit{posteriori} error estimator/distance-based criterion of adaptivity, and a particle-shifting technique. The method is applied in simulating six different flows or FSI problems. The new method's convergence, accuracy, and efficiency attributes are assessed by comparing the results it produces with analytical, finite element, and consistent SPH uniform high-resolution solutions as well as experimental data.

physics.flu-dyn↗

arXiv2018

Accelerating Optical Absorption Spectra and Exciton Energy Computation for Nanosystems via Interpolative Separable Density Fitting

We present an efficient way to solve the Bethe-Salpeter equation (BSE), a model for the computation of absorption spectra in molecules and solids that includes electron-hole excitations. Standard approaches to construct and diagonalize the Bethe-Salpeter Hamiltonian require at least $Ø(N_e^5)$ operations, where $N_e$ is proportional to the number of electrons in the system, limiting its application to small systems. Our approach is based on the interpolative separable density fitting (ISDF) technique to construct low rank approximations to the bare and screened exchange operators associated with the BSE Hamiltonian. This approach reduces the complexity of the Hamiltonian construction to $Ø(N_e^3)$ with a much smaller pre-constant. Here, we implement the ISDF method for the BSE calculations within the Tamm-Dancoff approximation (TDA) in the BerkeleyGW software package. We show that ISDF-based BSE calculations in molecules and solids reproduce accurate exciton energies and optical absorption spectra with significantly reduced computational cost.

physics.comp-ph↗

arXiv2017

An perturbation-iteration method for multi-peak solitons in nonlocal nonlinear media

An perturbation-iteration method is developed for the computation of the Hermite-Gaussian-like solitons with arbitrary peak numbers in nonlocal nonlinear media. This method is based on the perturbed model of the Schrödinger equation for the harmonic oscillator, in which the minimum perturbation is obtained by the iteration. This method takes a few tens of iteration loops to achieve enough high accuracy, and the initial condition is fixed to the Hermite-Gaussian function. The method we developed might also be extended to the numerical integration of the Schrödinger equations in any type of potentials.

physics.optics↗

arXiv2017

Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls

We propose a rank-$k$ variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation ($1$-SVD) in Frank-Wolfe with a top-$k$ singular-vector computation ($k$-SVD), which can be done by repeatedly applying $1$-SVD $k$ times. Alternatively, our algorithm can be viewed as a rank-$k$ restricted version of projected gradient descent. We show that our algorithm has a linear convergence rate when the objective function is smooth and strongly convex, and the optimal solution has rank at most $k$. This improves the convergence rate and the total time complexity of the Frank-Wolfe method and its variants.

cs.LG↗

arXiv2017

Interpolative Separable Density Fitting through Centroidal Voronoi Tessellation With Applications to Hybrid Functional Electronic Structure Calculations

The recently developed interpolative separable density fitting (ISDF) decomposition is a powerful way for compressing the redundant information in the set of orbital pairs, and has been used to accelerate quantum chemistry calculations in a number of contexts. The key ingredient of the ISDF decomposition is to select a set of non-uniform grid points, so that the values of the orbital pairs evaluated at such grid points can be used to accurately interpolate those evaluated at all grid points. The set of non-uniform grid points, called the interpolation points, can be automatically selected by a QR factorization with column pivoting (QRCP) procedure. This is the computationally most expensive step in the construction of the ISDF decomposition. In this work, we propose a new approach to find the interpolation points based on the centroidal Voronoi tessellation (CVT) method, which offers a much less expensive alternative to the QRCP procedure when ISDF is used in the context of hybrid functional electronic structure calculations. The CVT method only uses information from the electron density, and can be efficiently implemented using a K-Means algorithm. We find that this new method achieves comparable accuracy to the ISDF-QRCP method, at a cost that is negligible in the overall hybrid functional calculations. For instance, for a system containing $1000$ silicon atoms simulated using the HSE06 hybrid functional on $2000$ computational cores, the cost of QRCP-based method for finding the interpolation points costs $434.2$ seconds, while the CVT procedure only takes $3.2$ seconds. We also find that the ISDF-CVT method also enhances the smoothness of the potential energy surface in the context of \emph{ab initio} molecular dynamics (AIMD) simulations with hybrid functionals.

physics.comp-ph↗