arXiv subjects

Lei Zhu

Publications and source records attributed to Lei Zhu.

At least 343 records · Page 19Linked to original sources

arXiv2019

Personalized Hashtag Recommendation for Micro-videos

Personalized hashtag recommendation methods aim to suggest users hashtags to annotate, categorize, and describe their posts. The hashtags, that a user provides to a post (e.g., a micro-video), are the ones which in her mind can well describe the post content where she is interested in. It means that we should consider both users' preferences on the post contents and their personal understanding on the hashtags. Most existing methods rely on modeling either the interactions between hashtags and posts or the interactions between users and hashtags for hashtag recommendation. These methods have not well explored the complicated interactions among users, hashtags, and micro-videos. In this paper, towards the personalized micro-video hashtag recommendation, we propose a Graph Convolution Network based Personalized Hashtag Recommendation (GCN-PHR) model, which leverages recently advanced GCN techniques to model the complicate interactions among and learn their representations. In our model, the users, hashtags, and micro-videos are three types of nodes in a graph and they are linked based on their direct associations. In particular, the message-passing strategy is used to learn the representation of a node (e.g., user) by aggregating the message passed from the directly linked other types of nodes (e.g., hashtag and micro-video). Because a user is often only interested in certain parts of a micro-video and a hashtag is typically used to describe the part (of a micro-video) that the user is interested in, we leverage the attention mechanism to filter the message passed from micro-videos to users and hashtags, which can significantly improve the representation capability. Extensive experiments have been conducted on two real-world micro-video datasets and demonstrate that our model outperforms the state-of-the-art approaches by a large margin.

cs.MM↗

arXiv2019

Color imaging through the scattering media based on phase retrieval with triple correlation

Light passing through scattering media will be strongly scattered and diffused into complex speckle pattern, which however contains almost all the spatial information and color information of the objects. Although various technologies have been proposed to realize color imaging through the scattering media, current technologies are still complex with long sequence of measurement for each imaging pixel or spectral point spread functions of optical system. Here we theoretically prove the spatial averaging of triple correlation technique can be used to retrieve the Fourier phase of object, and experimentally demonstrate it can be applied in color imaging through scattering media. Compared to other phase retrieval techniques, the phase retrieval with triple correlation technique can retain the orientation information of objects, and can composite color image without rotation operation. Furthermore, our approach has the potential of realizing spectral imaging through scattering media.

eess.IV↗

arXiv2019

Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration

Brain image registration transforms a pair of images into one system with the matched imaging contents, which is of essential importance for brain image analysis. This paper presents a novel framework for unsupervised 3D brain image registration by capturing the feature-level transformation relationships between the unaligned image and reference image. To achieve this, we develop a feature-level probabilistic model to provide the direct regularization to the hidden layers of two deep convolutional neural networks, which are constructed from two input images. This model design is developed into multiple layers of these two networks to capture the transformation relationships at different levels. We employ two common benchmark datasets for 3D brain image registration and perform various experiments to evaluate our method. Experimental results show that our method clearly outperforms state-of-the-art methods on both benchmark datasets by a large margin.

cs.CV↗

arXiv2019

From Zero-Shot Learning to Cold-Start Recommendation

Zero-shot learning (ZSL) and cold-start recommendation (CSR) are two challenging problems in computer vision and recommender system, respectively. In general, they are independently investigated in different communities. This paper, however, reveals that ZSL and CSR are two extensions of the same intension. Both of them, for instance, attempt to predict unseen classes and involve two spaces, one for direct feature representation and the other for supplementary description. Yet there is no existing approach which addresses CSR from the ZSL perspective. This work, for the first time, formulates CSR as a ZSL problem, and a tailor-made ZSL method is proposed to handle CSR. Specifically, we propose a Low-rank Linear Auto-Encoder (LLAE), which challenges three cruxes, i.e., domain shift, spurious correlations and computing efficiency, in this paper. LLAE consists of two parts, a low-rank encoder maps user behavior into user attributes and a symmetric decoder reconstructs user behavior from user attributes. Extensive experiments on both ZSL and CSR tasks verify that the proposed method is a win-win formulation, i.e., not only can CSR be handled by ZSL models with a significant performance improvement compared with several conventional state-of-the-art methods, but the consideration of CSR can benefit ZSL as well.

cs.CV↗

arXiv2019

PAC-GAN: An Effective Pose Augmentation Scheme for Unsupervised Cross-View Person Re-identification

Person re-identification (person Re-Id) aims to retrieve the pedestrian images of a same person that captured by disjoint and non-overlapping cameras. Lots of researchers recently focuse on this hot issue and propose deep learning based methods to enhance the recognition rate in a supervised or unsupervised manner. However, two limitations that cannot be ignored: firstly, compared with other image retrieval benchmarks, the size of existing person Re-Id datasets are far from meeting the requirement, which cannot provide sufficient pedestrian samples for the training of deep model; secondly, the samples in existing datasets do not have sufficient human motions or postures coverage to provide more priori knowledges for learning. In this paper, we introduce a novel unsupervised pose augmentation cross-view person Re-Id scheme called PAC-GAN to overcome these limitations. We firstly present the formal definition of cross-view pose augmentation and then propose the framework of PAC-GAN that is a novel conditional generative adversarial network (CGAN) based approach to improve the performance of unsupervised corss-view person Re-Id. Specifically, The pose generation model in PAC-GAN called CPG-Net is to generate enough quantity of pose-rich samples from original image and skeleton samples. The pose augmentation dataset is produced by combining the synthesized pose-rich samples with the original samples, which is fed into the corss-view person Re-Id model named Cross-GAN. Besides, we use weight-sharing strategy in the CPG-Net to improve the quality of new generated samples. To the best of our knowledge, we are the first try to enhance the unsupervised cross-view person Re-Id by pose augmentation, and the results of extensive experiments show that the proposed scheme can combat the state-of-the-arts.

cs.CV↗

arXiv2019

ZAIGA: Zhaoshan Long-baseline Atom Interferometer Gravitation Antenna

The Zhaoshan long-baseline Atom Interferometer Gravitation Antenna (ZAIGA) is a new type of underground laser-linked interferometer facility, and is currently under construction. It is in the 200-meter-on-average underground of a mountain named Zhaoshan which is about 80 km southeast to Wuhan. ZAIGA will be equipped with long-baseline atom interferometers, high-precision atom clocks, and large-scale gyros. ZAIGA facility will take an equilateral triangle configuration with two 1-km-apart atom interferometers in each arm, a 300-meter vertical tunnel with atom fountain and atom clocks mounted, and a tracking-and-ranging 1-km-arm-length prototype with lattice optical clocks linked by locked lasers. The ZAIGA facility will be used for experimental research on gravitation and related problems including gravitational wave detection, high-precision test of the equivalence principle of micro-particles, clock based gravitational red-shift measurement, rotation measurement and gravito-magnetic effect.

physics.atom-ph↗

arXiv2019

Direction-aware Spatial Context Features for Shadow Detection and Removal

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a Euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.

cs.CV↗

arXiv2019

Carbon-Chain Molecules in Molecular Outflows and Lupus I Region--New Producing Region and New Forming Mechanism

Using the new equipment of the Shanghai Tian Ma Radio Telescope, we have searched for carbon-chain molecules (CCMs) towards five outflow sources and six Lupus I starless dust cores, including one region known to be characterized by warm carbon-chain chemistry (WCCC), Lupus I-1 (IRAS 15398-3359), and one TMC-1 like cloud, Lupus I-6 (Lupus-1A). Lines of HC3N J=2-1, HC5N J=6-5, HC7N J=14-13, 15-14, 16-15 and C3S J=3-2 were detected in all the targets except in the outflow source L1660 and the starless dust core Lupus I-3/4. The column densities of nitrogen-bearing species range from 10$^{12}$ to 10$^{14}$ cm$^{-2}$ and those of C$_3$S are about 10$^{12}$ cm$^{-2}$. Two outflow sources, I20582+7724 and L1221, could be identified as new carbon-chain--producing regions. Four of the Lupus I dust cores are newly identified as early quiescent and dark carbon-chain--producing regions similar to Lup I-6, which together with the WCCC source, Lup I-1, indicate that carbon-chain-producing regions are popular in Lupus I which can be regard as a Taurus like molecular cloud complex in our Galaxy. The column densities of C3S are larger than those of HC7N in the three outflow sources I20582, L1221 and L1251A. Shocked carbon-chain chemistry (SCCC) is proposed to explain the abnormal high abundances of C3S compared with those of nitrogen-bearing CCMs. Gas-grain chemical models support the idea that shocks can fuel the environment of those sources with enough $S^+$ thus driving the generation of S-bearing CCMs.

astro-ph.SR↗

arXiv2019

Exploring Auxiliary Context: Discrete Semantic Transfer Hashing for Scalable Image Retrieval

Unsupervised hashing can desirably support scalable content-based image retrieval (SCBIR) for its appealing advantages of semantic label independence, memory and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as \emph{Discrete Semantic Transfer Hashing} (DSTH). The key idea is to \emph{directly} augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Further, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit--uncorrelation constraint and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark datasets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.

cs.IR↗

arXiv2019

Adaptive Collaborative Similarity Learning for Unsupervised Multi-view Feature Selection

In this paper, we investigate the research problem of unsupervised multi-view feature selection. Conventional solutions first simply combine multiple pre-constructed view-specific similarity structures into a collaborative similarity structure, and then perform the subsequent feature selection. These two processes are separate and independent. The collaborative similarity structure remains fixed during feature selection. Further, the simple undirected view combination may adversely reduce the reliability of the ultimate similarity structure for feature selection, as the view-specific similarity structures generally involve noises and outlying entries. To alleviate these problems, we propose an adaptive collaborative similarity learning (ACSL) for multi-view feature selection. We propose to dynamically learn the collaborative similarity structure, and further integrate it with the ultimate feature selection into a unified framework. Moreover, a reasonable rank constraint is devised to adaptively learn an ideal collaborative similarity structure with proper similarity combination weights and desirable neighbor assignment, both of which could positively facilitate the feature selection. An effective solution guaranteed with the proved convergence is derived to iteratively tackle the formulated optimization problem. Experiments demonstrate the superiority of the proposed approach.

cs.IR↗

arXiv2019

Discrete Optimal Graph Clustering

Graph based clustering is one of the major clustering methods. Most of it work in three separate steps: similarity graph construction, clustering label relaxing and label discretization with k-means. Such common practice has three disadvantages: 1) the predefined similarity graph is often fixed and may not be optimal for the subsequent clustering. 2) the relaxing process of cluster labels may cause significant information loss. 3) label discretization may deviate from the real clustering result since k-means is sensitive to the initialization of cluster centroids. To tackle these problems, in this paper, we propose an effective discrete optimal graph clustering (DOGC) framework. A structured similarity graph that is theoretically optimal for clustering performance is adaptively learned with a guidance of reasonable rank constraint. Besides, to avoid the information loss, we explicitly enforce a discrete transformation on the intermediate continuous label, which derives a tractable optimization problem with discrete solution. Further, to compensate the unreliability of the learned labels and enhance the clustering accuracy, we design an adaptive robust module that learns prediction function for the unseen data based on the learned discrete cluster labels. Finally, an iterative optimization strategy guaranteed with convergence is developed to directly solve the clustering results. Extensive experiments conducted on both real and synthetic datasets demonstrate the superiority of our proposed methods compared with several state-of-the-art clustering approaches.

cs.LG↗

arXiv2019

Leveraging the Invariant Side of Generative Zero-Shot Learning

Conventional zero-shot learning (ZSL) methods generally learn an embedding, e.g., visual-semantic mapping, to handle the unseen visual samples via an indirect manner. In this paper, we take the advantage of generative adversarial networks (GANs) and propose a novel method, named leveraging invariant side GAN (LisGAN), which can directly generate the unseen features from random noises which are conditioned by the semantic descriptions. Specifically, we train a conditional Wasserstein GANs in which the generator synthesizes fake unseen features from noises and the discriminator distinguishes the fake from real via a minimax game. Considering that one semantic description can correspond to various synthesized visual samples, and the semantic description, figuratively, is the soul of the generated features, we introduce soul samples as the invariant side of generative zero-shot learning in this paper. A soul sample is the meta-representation of one class. It visualizes the most semantically-meaningful aspects of each sample in the same category. We regularize that each generated sample (the varying side of generative ZSL) should be close to at least one soul sample (the invariant side) which has the same class label with it. At the zero-shot recognition stage, we propose to use two classifiers, which are deployed in a cascade way, to achieve a coarse-to-fine result. Experiments on five popular benchmarks verify that our proposed approach can outperform state-of-the-art methods with significant improvements.

cs.CV↗

arXiv2019

The JCMT BISTRO Survey: The Magnetic Field In The Starless Core $ρ$ Ophiuchus C

We report 850~$μ$m dust polarization observations of a low-mass ($\sim$12 $M_{\odot}$) starless core in the $ρ$ Ophiuchus cloud, Ophiuchus C, made with the POL-2 instrument on the James Clerk Maxwell Telescope (JCMT) as part of the JCMT B-fields In STar-forming Region Observations (BISTRO) survey. We detect an ordered magnetic field projected on the plane of sky in the starless core. The magnetic field across the $\sim$0.1~pc core shows a predominant northeast-southwest orientation centering between $\sim$40$^\circ$ to $\sim$100$^\circ$, indicating that the field in the core is well aligned with the magnetic field in lower-density regions of the cloud probed by near-infrared observations and also the cloud-scale magnetic field traced by Planck observations. The polarization percentage ($P$) decreases with an increasing total intensity ($I$) with a power-law index of $-$1.03 $\pm$ 0.05. We estimate the plane-of-sky field strength ($B_{\mathrm{pos}}$) using modified Davis-Chandrasekhar-Fermi (DCF) methods based on structure function (SF), auto-correlation (ACF), and unsharp masking (UM) analyses. We find that the estimates from the SF, ACF, and UM methods yield strengths of 103 $\pm$ 46 $μ$G, 136 $\pm$ 69 $μ$G, and 213 $\pm$ 115 $μ$G, respectively. Our calculations suggest that the Ophiuchus C core is near magnetically critical or slightly magnetically supercritical (i.e. unstable to collapse). The total magnetic energy calculated from the SF method is comparable to the turbulent energy in Ophiuchus C, while the ACF method and the UM method only set upper limits for the total magnetic energy because of large uncertainties.

astro-ph.GA↗

arXiv2019

Recovering the spectral and spatial information of an object behind a scattering media

Light passing through scattering media will be strongly scattered and diffused into complex speckle pattern, which contains almost all the spatial information and spectral information of the objects. Although various methods have been proposed to recover the spatial information of the hidden objects, it is still a challenge to simultaneously obtain their spectral information. Here, we present an effective approach to realize spectral imaging through scattering media by combining the spectra retrieval and the speckle-correlation. Compared to the traditional imaging spectrometer, our approach is more flexible in the choice of core element. In this paper, we have demonstrated employing the frosted glass as the core element to achieve spectral imaging. Obtaining the spectral information and spatial information are demonstrated via numerical simulations. Experiment results further demonstrate the performance of our scheme in spectral imaging through scattering media. The spectral imaging based on scattering media is well suited for new type spectral imaging applications.

physics.optics↗

arXiv2018

MMALFM: Explainable Recommendation by Leveraging Reviews and Images

Although the latent factor model achieves good accuracy in rating prediction, it suffers from many problems including cold-start, non-transparency, and suboptimal results for individual user-item pairs. In this paper, we exploit textual reviews and item images together with ratings to tackle these limitations. Specifically, we first apply a proposed multi-modal aspect-aware topic model (MATM) on text reviews and item images to model users' preferences and items' features from different aspects, and also estimate the aspect importance of a user towards an item. Then the aspect importance is integrated into a novel aspect-aware latent factor model (ALFM), which learns user's and item's latent factors based on ratings. In particular, ALFM introduces a weight matrix to associate those latent factors with the same set of aspects in MATM, such that the latent factors could be used to estimate aspect ratings. Finally, the overall rating is computed via a linear combination of the aspect ratings, which are weighted by the corresponding aspect importance. To this end, our model could alleviate the data sparsity problem and gain good interpretability for recommendation. Besides, every aspect rating is weighted by its aspect importance, which is dependent on the targeted user's preferences and the targeted item's features. Therefore, it is expected that the proposed method can model a user's preferences on an item more accurately for each user-item pair. Comprehensive experimental studies have been conducted on the Yelp 2017 Challenge dataset and Amazon product datasets to demonstrate the effectiveness of our method.

cs.IR↗

arXiv2018

Explicit Interaction Model towards Text Classification

Text classification is one of the fundamental tasks in natural language processing. Recently, deep neural networks have achieved promising performance in the text classification task compared to shallow models. Despite of the significance of deep models, they ignore the fine-grained (matching signals between words and classes) classification clues since their classifications mainly rely on the text-level representations. To address this problem, we introduce the interaction mechanism to incorporate word-level matching signals into the text classification task. In particular, we design a novel framework, EXplicit interAction Model (dubbed as EXAM), equipped with the interaction mechanism. We justified the proposed approach on several benchmark datasets including both multi-label and multi-class text classification tasks. Extensive experimental results demonstrate the superiority of the proposed method. As a byproduct, we have released the codes and parameter settings to facilitate other researches.

cs.CL↗

arXiv2018

CNN-VWII: An Efficient Approach for Large-Scale Video Retrieval by Image Queries

This paper aims to solve the problem of large-scale video retrieval by a query image. Firstly, we define the problem of top-$k$ image to video query. Then, we combine the merits of convolutional neural networks(CNN for short) and Bag of Visual Word(BoVW for short) module to design a model for video frames information extraction and representation. In order to meet the requirements of large-scale video retrieval, we proposed a visual weighted inverted index(VWII for short) and related algorithm to improve the efficiency and accuracy of retrieval process. Comprehensive experiments show that our proposed technique achieves substantial improvements (up to an order of magnitude speed up) over the state-of-the-art techniques with similar accuracy.

cs.MM↗

arXiv2018

SVS-JOIN: Efficient Spatial Visual Similarity Join over Multimedia Data

In the big data era, massive amount of multimedia data with geo-tags has been generated and collected by mobile smart devices equipped with mobile communications module and position sensor module. This trend has put forward higher request on large-scale of geo-multimedia data retrieval. Spatial similarity join is one of the important problem in the area of spatial database. Previous works focused on textual document with geo-tags, rather than geo-multimedia data such as geo-images. In this paper, we study a novel search problem named spatial visual similarity join (SVS-JOIN for short), which aims to find similar geo-image pairs in both the aspects of geo-location and visual content. We propose the definition of SVS-JOIN at the first time and present how to measure geographical similarity and visual similarity. Then we introduce a baseline inspired by the method for textual similarity join and a extension named SVS-JOIN$_G$ which applies spatial grid strategy to improve the efficiency. To further improve the performance of search, we develop a novel approach called SVS-JOIN$_Q$ which utilizes a quadtree and a global inverted index. Experimental evaluations on real geo-image datasets demonstrate that our solution has a really high performance.

cs.MM↗