arXiv subjects

Zhao Yang

Publications and source records attributed to Zhao Yang.

At least 91 records · Page 5Linked to original sources

arXiv2022

When to Go, and When to Explore: The Benefit of Post-Exploration in Intrinsic Motivation

Go-Explore achieved breakthrough performance on challenging reinforcement learning (RL) tasks with sparse rewards. The key insight of Go-Explore was that successful exploration requires an agent to first return to an interesting state ('Go'), and only then explore into unknown terrain ('Explore'). We refer to such exploration after a goal is reached as 'post-exploration'. In this paper we present a systematic study of post-exploration, answering open questions that the Go-Explore paper did not answer yet. First, we study the isolated potential of post-exploration, by turning it on and off within the same algorithm. Subsequently, we introduce new methodology to adaptively decide when to post-explore and for how long to post-explore. Experiments on a range of MiniGrid environments show that post-exploration indeed boosts performance (with a bigger impact than tuning regular exploration parameters), and this effect is further enhanced by adaptively deciding when and for how long to post-explore. In short, our work identifies adaptive post-exploration as a promising direction for RL exploration research.

cs.LG↗

arXiv2022

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

Referring image segmentation is a fundamental vision-language task that aims to segment out an object referred to by a natural language expression from an image. One of the key challenges behind this task is leveraging the referring expression for highlighting relevant positions in the image. A paradigm for tackling this problem is to leverage a powerful vision-language ("cross-modal") decoder to fuse features independently extracted from a vision encoder and a language encoder. Recent methods have made remarkable advancements in this paradigm by exploiting Transformers as cross-modal decoders, concurrent to the Transformer's overwhelming success in many other vision-language tasks. Adopting a different approach in this work, we show that significantly better cross-modal alignments can be achieved through the early fusion of linguistic and visual features in intermediate layers of a vision Transformer encoder network. By conducting cross-modal feature fusion in the visual feature encoding stage, we can leverage the well-proven correlation modeling power of a Transformer encoder for excavating helpful multi-modal context. This way, accurate segmentation results are readily harvested with a light-weight mask predictor. Without bells and whistles, our method surpasses the previous state-of-the-art methods on RefCOCO, RefCOCO+, and G-Ref by large margins.

cs.CV↗

arXiv2022

On the Effectiveness of Pinyin-Character Dual-Decoding for End-to-End Mandarin Chinese ASR

End-to-end automatic speech recognition (ASR) has achieved promising results. However, most existing end-to-end ASR methods neglect the use of specific language characteristics. For Mandarin Chinese ASR tasks, there exist mutual promotion relationship between Pinyin and Character where Chinese characters can be romanized by Pinyin. Based on the above intuition, we first investigate types of end-to-end encoder-decoder based models in the single-input dual-output (SIDO) multi-task framework, after which a novel asynchronous decoding with fuzzy Pinyin sampling method is proposed according to the one-to-one correspondence characteristics between Pinyin and Character. Furthermore, we proposed a two-stage training strategy to make training more stable and converge faster. The results on the test sets of AISHELL-1 dataset show that the proposed enhanced dual-decoder model without a language model is improved by a big margin compared to strong baseline models.

cs.CL↗

arXiv2021

Seq-Masks: Bridging the gap between appearance and gait modeling for video-based person re-identification

ideo-based person re-identification (Re-ID) aims to match person images in video sequences captured by disjoint surveillance cameras. Traditional video-based person Re-ID methods focus on exploring appearance information, thus, vulnerable against illumination changes, scene noises, camera parameters, and especially clothes/carrying variations. Gait recognition provides an implicit biometric solution to alleviate the above headache. Nonetheless, it experiences severe performance degeneration as camera view varies. In an attempt to address these problems, in this paper, we propose a framework that utilizes the sequence masks (SeqMasks) in the video to integrate appearance information and gait modeling in a close fashion. Specifically, to sufficiently validate the effectiveness of our method, we build a novel dataset named MaskMARS based on MARS. Comprehensive experiments on our proposed large wild video Re-ID dataset MaskMARS evidenced our extraordinary performance and generalization capability. Validations on the gait recognition metric CASIA-B dataset further demonstrated the capability of our hybrid model.

cs.CV↗

arXiv2021

An alternative proof of modulation instability of Stokes waves in deep water

We generalize the periodic Evans function approach recently used to study the spectral stability of Stokes wave and gravity-capillary (including Wilton ripples) in water of finite depth to study spectral stability of Stokes waves in water of infinite depth. We prove waves of sufficiently small amplitude are always low-frequency unstable regardless of the wave number and gravity, giving an alternative proof for the Benjamin-Feir modulational instability in the infinite depth case. Here, the first proof for the infinite depth case is recently obtained by Nguyen and Strauss. We also study the spectral stability at non-zero resonant frequencies and find no additional instability.

math.AP↗

arXiv2021

Transfer Learning and Curriculum Learning in Sokoban

Transfer learning can speed up training in machine learning and is regularly used in classification tasks. It reuses prior knowledge from other tasks to pre-train networks for new tasks. In reinforcement learning, learning actions for a behavior policy that can be applied to new environments is still a challenge, especially for tasks that involve much planning. Sokoban is a challenging puzzle game. It has been used widely as a benchmark in planning-based reinforcement learning. In this paper, we show how prior knowledge improves learning in Sokoban tasks. We find that reusing feature representations learned previously can accelerate learning new, more complex, instances. In effect, we show how curriculum learning, from simple to complex tasks, works in Sokoban. Furthermore, feature representations learned in simpler instances are more general, and thus lead to positive transfers towards more complex tasks, but not vice versa. We have also studied which part of the knowledge is most important for transfer to succeed, and identify which layers should be used for pre-training.

cs.AI↗

arXiv2021

Potential-based Reward Shaping in Sokoban

Learning to solve sparse-reward reinforcement learning problems is difficult, due to the lack of guidance towards the goal. But in some problems, prior knowledge can be used to augment the learning process. Reward shaping is a way to incorporate prior knowledge into the original reward function in order to speed up the learning. While previous work has investigated the use of expert knowledge to generate potential functions, in this work, we study whether we can use a search algorithm(A*) to automatically generate a potential function for reward shaping in Sokoban, a well-known planning task. The results showed that learning with shaped reward function is faster than learning from scratch. Our results indicate that distance functions could be a suitable function for Sokoban. This work demonstrates the possibility of solving multiple instances with the help of reward shaping. The result can be compressed into a single policy, which can be seen as the first phrase towards training a general policy that is able to solve unseen instances.

cs.LG↗

arXiv2021

Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback

The delayed feedback problem is one of the imperative challenges in online advertising, which is caused by the highly diversified feedback delay of a conversion varying from a few minutes to several days. It is hard to design an appropriate online learning system under these non-identical delay for different types of ads and users. In this paper, we propose to tackle the delayed feedback problem in online advertising by "Following the Prophet" (FTP for short). The key insight is that, if the feedback came instantly for all the logged samples, we could get a model without delayed feedback, namely the "prophet". Although the prophet cannot be obtained during online learning, we show that we could predict the prophet's predictions by an aggregation policy on top of a set of multi-task predictions, where each task captures the feedback patterns of different periods. We propose the objective and optimization approach for the policy, and use the logged data to imitate the prophet. Extensive experiments on three real-world advertising datasets show that our method outperforms the previous state-of-the-art baselines.

cs.LG↗

arXiv2021

Explore User Neighborhood for Real-time E-commerce Recommendation

Recommender systems play a vital role in modern online services, such as Amazon and Taobao. Traditional personalized methods, which focus on user-item (UI) relations, have been widely applied in industrial settings, owing to their efficiency and effectiveness. Despite their success, we argue that these approaches ignore local information hidden in similar users. To tackle this problem, user-based methods exploit similar user relations to make recommendations in a local perspective. Nevertheless, traditional user-based methods, like userKNN and matrix factorization, are intractable to be deployed in the real-time applications since such transductive models have to be recomputed or retrained with any new interaction. To overcome this challenge, we propose a framework called self-complementary collaborative filtering~(SCCF) which can make recommendations with both global and local information in real time. On the one hand, it utilizes UI relations and user neighborhood to capture both global and local information. On the other hand, it can identify similar users for each user in real time by inferring user representations on the fly with an inductive model. The proposed framework can be seamlessly incorporated into existing inductive UI approach and benefit from user neighborhood with little additional computation. It is also the first attempt to apply user-based methods in real-time settings. The effectiveness and efficiency of SCCF are demonstrated through extensive offline experiments on four public datasets, as well as a large scale online A/B test in Taobao.

cs.IR↗

arXiv2020

Two-Grid Deflated Krylov Methods for Linear Equations

An approach is given for solving large linear systems that combines Krylov methods with use of two different grid levels. Eigenvectors are computed on the coarse grid and used to deflate eigenvalues on the fine grid. GMRES-type methods are first used on both the coarse and fine grids. Then another approach is given that has a restarted BiCGStab (or IDR) method on the fine grid. While BiCGStab is generally considered to be a non-restarted method, it works well in this context with deflating and restarting. Tests show this new approach can be very efficient for difficult linear equations problems.

math.NA↗

arXiv2020

Stability of strong detonation waves for Majda's model with general ignition functions

For strong detonation waves of the inviscid Majda model, spectral stability was established by Jung and Yao for waves with step-type ignition functions, by a proof based largely on explicit knowledge of wave profiles. In the present work, we extend their stability results to strong detonation waves with more general ignition functions where explicit profiles are unknown. Our proof is based on reduction to a generalized Sturm-Liouville problem, similar to that used by Sukhtayev, Yang, and Zumbrun to study spectral stability of hydraulic shock profiles of the Saint-Venant equations.

math.AP↗

arXiv2019

Quantum Causal Influence

We introduce a framework to study the emergence of time and causal structure in quantum many-body systems. In doing so, we consider quantum states which encode spacetime dynamics, and develop information theoretic tools to extract the causal relationships between putative spacetime subsystems. Our analysis reveals a quantum generalization of the thermodynamic arrow of time and begins to explore the roles of entanglement, scrambling and quantum error correction in the emergence of spacetime. For instance, exotic causal relationships can arise due to dynamically induced quantum error correction in spacetime: there can exist a spatial region in the past which does not causally influence any small spatial regions in the future, but yet it causally influences the union of several small spatial regions in the future. We provide examples of quantum causal influence in Hamiltonian evolution, quantum error correction codes, quantum teleportation, holographic tensor networks, the final state projection model of black holes, and many other systems. We find that the quantum causal influence provides a unifying perspective on spacetime correlations in these seemingly distinct settings. In addition, we prove a variety of general structural results and discuss the relation of quantum causal influence to spacetime quantum entropies.

hep-th↗

arXiv2019

Anchor Diffusion for Unsupervised Video Object Segmentation

Unsupervised video object segmentation has often been tackled by methods based on recurrent neural networks and optical flow. Despite their complexity, these kinds of approaches tend to favour short-term temporal dependencies and are thus prone to accumulating inaccuracies, which cause drift over time. Moreover, simple (static) image segmentation models, alone, can perform competitively against these methods, which further suggests that the way temporal dependencies are modelled should be reconsidered. Motivated by these observations, in this paper we explore simple yet effective strategies to model long-term temporal dependencies. Inspired by the non-local operators of [70], we introduce a technique to establish dense correspondences between pixel embeddings of a reference "anchor" frame and the current one. This allows the learning of pairwise dependencies at arbitrarily long distances without conditioning on intermediate frames. Without online supervision, our approach can suppress the background and precisely segment the foreground object even in challenging scenarios, while maintaining consistent performance over time. With a mean IoU of $81.7\%$, our method ranks first on the DAVIS-2016 leaderboard of unsupervised methods, while still being competitive against state-of-the-art online semi-supervised approaches. We further evaluate our method on the FBMS dataset and the ViSal video saliency dataset, showing results competitive with the state of the art.

cs.CV↗

arXiv2019

Hetero-Center Loss for Cross-Modality Person Re-Identification

Cross-modality person re-identification is a challenging problem which retrieves a given pedestrian image in RGB modality among all the gallery images in infrared modality. The task can address the limitation of RGB-based person Re-ID in dark environments. Existing researches mainly focus on enlarging inter-class differences of feature to solve the problem. However, few studies investigate improving intra-class cross-modality similarity, which is important for this issue. In this paper, we propose a novel loss function, called Hetero-Center loss (HC loss) to reduce the intra-class cross-modality variations. Specifically, HC loss can supervise the network learning the cross-modality invariant information by constraining the intra-class center distance between two heterogenous modalities. With the joint supervision of Cross-Entropy (CE) loss and HC loss, the network is trained to achieve two vital objectives, inter-class discrepancy and intra-class cross-modality similarity as much as possible. Besides, we propose a simple and high-performance network architecture to learn local feature representations for cross-modality person re-identification, which can be a baseline for future research. Extensive experiments indicate the effectiveness of the proposed methods, which outperform state-of-the-art methods by a wide margin.

cs.CV↗

arXiv2019

Stability of hydraulic shock profiles

We establish nonlinear $H^2\cap L^1 \to H^2$ stability with sharp rates of decay in $L^p$, $p\geq 2$, of general hydraulic shock profiles, with or without subshocks, of the inviscid Saint-Venant equations of shallow water flow, under the assumption of Evans-Lopatinsky stability of the associated eigenvalue problem. We verify this assumption numerically for all profiles, giving in particular the first nonlinear stability results for shock profiles with subshocks of a hyperbolic relaxation system.

math.AP↗

arXiv2019

Res-embedding for Deep Learning Based Click-Through Rate Prediction Modeling

Recently, click-through rate (CTR) prediction models have evolved from shallow methods to deep neural networks. Most deep CTR models follow an Embedding\&MLP paradigm, that is, first mapping discrete id features, e.g. user visited items, into low dimensional vectors with an embedding module, then learn a multi-layer perception (MLP) to fit the target. In this way, embedding module performs as the representative learning and plays a key role in the model performance. However, in many real-world applications, deep CTR model often suffers from poor generalization performance, which is mostly due to the learning of embedding parameters. In this paper, we model user behavior using an interest delay model, study carefully the embedding mechanism, and obtain two important results: (i) We theoretically prove that small aggregation radius of embedding vectors of items which belongs to a same user interest domain will result in good generalization performance of deep CTR model. (ii) Following our theoretical analysis, we design a new embedding structure named res-embedding. In res-embedding module, embedding vector of each item is the sum of two components: (i) a central embedding vector calculated from an item-based interest graph (ii) a residual embedding vector with its scale to be relatively small. Empirical evaluation on several public datasets demonstrates the effectiveness of the proposed res-embedding structure, which brings significant improvement on the model performance.

stat.ML↗

arXiv2018

Spectral stability of hydraulic shock profiles

By reduction to a generalized Sturm Liouville problem, we establish spectral stability of hydraulic shock profiles of the Saint-Venant equations for inclined shallow-water flow, over the full parameter range of their existence, for both smooth-type profiles and discontinuous-type profiles containing subshocks. Together with work of Mascia-Zumbrun and Yang-Zumbrun, this yields linear and nonlinear $H^2\cap L^1 \to H^2$ stability with sharp rates of decay in $L^p$, $p\geq 2$, the first complete stability results for large-amplitude shock profiles of a hyperbolic relaxation system.

math.AP↗

arXiv2018

Spectral Stability of Inviscid Roll Waves

We carry out a systematic analytical and numerical study of spectral stability of discontinuous roll wave solutions of the inviscid Saint Venant equations, based on a periodic Evans-Lopatinski determinant analogous to the periodic Evans function of Gardner in the (smooth) viscous case, obtaining a complete spectral stability diagram useful in hydraulic engineering and related applications. In particular, we obtain an explicit low-frequency stability boundary, which, moreover, matches closely with its (numerically-determined) counterpart in the viscous case. This is seen to be related to but not implied by the associated formal first-order Whitham modulation equations.

math.AP↗