arXiv subjects

Yuchen Liu

Publications and source records attributed to Yuchen Liu.

At least 217 records · Page 12Linked to original sources

arXiv2022

K-stability and birational models of moduli of quartic K3 surfaces

We show that the K-moduli spaces of log Fano pairs $(\mathbb{P}^3, cS)$ where $S$ is a quartic surface interpolate between the GIT moduli space of quartic surfaces and the Baily-Borel compactification of moduli of quartic K3 surfaces as $c$ varies in the interval $(0,1)$. We completely describe the wall crossings of these K-moduli spaces. As the main application, we verify Laza-O'Grady's prediction on the Hassett-Keel-Looijenga program for quartic K3 surfaces. We also obtain the K-moduli compactification of quartic double solids, and classify all Gorenstein canonical Fano degenerations of $\mathbb{P}^3$.

math.AG↗

arXiv2022

CCATMos: Convolutional Context-aware Transformer Network for Non-intrusive Speech Quality Assessment

Speech quality assessment has been a critical component in many voice communication related applications such as telephony and online conferencing. Traditional intrusive speech quality assessment requires the clean reference of the degraded utterance to provide an accurate quality measurement. This requirement limits the usability of these methods in real-world scenarios. On the other hand, non-intrusive subjective measurement is the ``golden standard" in evaluating speech quality as human listeners can intrinsically evaluate the quality of any degraded speech with ease. In this paper, we propose a novel end-to-end model structure called Convolutional Context-Aware Transformer (CCAT) network to predict the mean opinion score (MOS) of human raters. We evaluate our model on three MOS-annotated datasets spanning multiple languages and distortion types and submit our results to the ConferencingSpeech 2022 Challenge. Our experiments show that CCAT provides promising MOS predictions compared to current state-of-art non-intrusive speech assessment models with average Pearson correlation coefficient (PCC) increasing from 0.530 to 0.697 and average RMSE decreasing from 0.768 to 0.570 compared to the baseline model on the challenge evaluation test set.

eess.AS↗

arXiv2022

Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation

End-to-end Speech Translation (ST) aims at translating the source language speech into target language text without generating the intermediate transcriptions. However, the training of end-to-end methods relies on parallel ST data, which are difficult and expensive to obtain. Fortunately, the supervised data for automatic speech recognition (ASR) and machine translation (MT) are usually more accessible, making zero-shot speech translation a potential direction. Existing zero-shot methods fail to align the two modalities of speech and text into a shared semantic space, resulting in much worse performance compared to the supervised ST methods. In order to enable zero-shot ST, we propose a novel Discrete Cross-Modal Alignment (DCMA) method that employs a shared discrete vocabulary space to accommodate and match both modalities of speech and text. Specifically, we introduce a vector quantization module to discretize the continuous representations of speech and text into a finite set of virtual tokens, and use ASR data to map corresponding speech and text to the same virtual token in a shared codebook. This way, source language speech can be embedded in the same semantic space as the source language text, which can be then transformed into target language text with an MT module. Experiments on multiple language pairs demonstrate that our zero-shot ST method significantly improves the SOTA, and even performers on par with the strong supervised ST baselines.

cs.CL↗

arXiv2022

Scalable Model-based Policy Optimization for Decentralized Networked Systems

Reinforcement learning algorithms require a large amount of samples; this often limits their real-world applications on even simple tasks. Such a challenge is more outstanding in multi-agent tasks, as each step of operation is more costly requiring communications or shifting or resources. This work aims to improve data efficiency of multi-agent control by model-based learning. We consider networked systems where agents are cooperative and communicate only locally with their neighbors, and propose the decentralized model-based policy optimization framework (DMPO). In our method, each agent learns a dynamic model to predict future states and broadcast their predictions by communication, and then the policies are trained under the model rollouts. To alleviate the bias of model-generated data, we restrain the model usage for generating myopic rollouts, thus reducing the compounding error of model generation. To pertain the independence of policy update, we introduce extended value function and theoretically prove that the resulting policy gradient is a close approximation to true policy gradients. We evaluate our algorithm on several benchmarks for intelligent transportation systems, which are connected autonomous vehicle control tasks (Flow and CACC) and adaptive traffic signal control (ATSC). Empirically results show that our method achieves superior data efficiency and matches the performance of model-free methods using true models.

cs.LG↗

arXiv2022

Multi-Task Learning Framework for Emotion Recognition in-the-wild

This paper presents our system for the Multi-Task Learning (MTL) Challenge in the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. We explore the research problems of this challenge from three aspects: 1) For obtaining efficient and robust visual feature representations, we propose MAE-based unsupervised representation learning and IResNet/DenseNet-based supervised representation learning methods; 2) Considering the importance of temporal information in videos, we explore three types of sequential encoders to capture the temporal information, including the encoder based on transformer, the encoder based on LSTM, and the encoder based on GRU; 3) For modeling the correlation between these different tasks (i.e., valence, arousal, expression, and AU) for multi-task affective analysis, we first explore the dependency between these different tasks and propose three multi-task learning frameworks to model the correlations effectively. Our system achieves the performance of $1.7607$ on the validation dataset and $1.4361$ on the test dataset, ranking first in the MTL Challenge. The code is available at https://github.com/AIM3-RUC/ABAW4.

cs.CV↗

arXiv2022

3D-FM GAN: Towards 3D-Controllable Face Manipulation

3D-controllable portrait synthesis has significantly advanced, thanks to breakthroughs in generative adversarial networks (GANs). However, it is still challenging to manipulate existing face images with precise 3D control. While concatenating GAN inversion and a 3D-aware, noise-to-image GAN is a straight-forward solution, it is inefficient and may lead to noticeable drop in editing quality. To fill this gap, we propose 3D-FM GAN, a novel conditional GAN framework designed specifically for 3D-controllable face manipulation, and does not require any tuning after the end-to-end learning phase. By carefully encoding both the input face image and a physically-based rendering of 3D edits into a StyleGAN's latent spaces, our image generator provides high-quality, identity-preserved, 3D-controllable face manipulation. To effectively learn such novel framework, we develop two essential training strategies and a novel multiplicative co-modulation architecture that improves significantly upon naive schemes. With extensive evaluations, we show that our method outperforms the prior arts on various tasks, with better editability, stronger identity preservation, and higher photo-realism. In addition, we demonstrate a better generalizability of our design on large pose editing and out-of-domain images.

cs.CV↗

arXiv2022

Court Judgement Labeling Using Topic Modeling and Syntactic Parsing

In regions that practice common law, relevant historical cases are essential references for sentencing. To help legal practitioners find previous judgement easier, this paper aims to label each court judgement by some tags. These tags are legally important to summarize the judgement and can guide the user to similar judgements. We introduce a heuristic system to solve the problem, which starts from Aspect-driven Topic Modeling and uses Dependency Parsing and Constituency Parsing for phrase generation. We also construct a legal term tree for Hong Kong and implemented a sentence simplification module to support the system. Finally, we propose a similar document recommendation algorithm based on the generated tags. It enables users to find similar documents based on a few selected aspects rather than the whole passage. Experiment results show that this system is the best approach for this specific task. It is better than simple term extraction method in terms of summarizing the document, and the recommendation algorithm is more effective than full-text comparison approaches. We believe that the system has huge potential in law as well as in other areas.

cs.IR↗

arXiv2022

Evolving Transferable Neural Pruning Functions

Structural design of neural networks is crucial for the success of deep learning. While most prior works in evolutionary learning aim at directly searching the structure of a network, few attempts have been made on another promising track, channel pruning, which recently has made major headway in designing efficient deep learning models. In fact, prior pruning methods adopt human-made pruning functions to score a channel's importance for channel pruning, which requires domain knowledge and could be sub-optimal. To this end, we pioneer the use of genetic programming (GP) to discover strong pruning metrics automatically. Specifically, we craft a novel design space to express high-quality and transferable pruning functions, which ensures an end-to-end evolution process where no manual modification is needed on the evolved functions for their transferability after evolution. Unlike prior methods, our approach can provide both compact pruned networks for efficient inference and novel closed-form pruning metrics which are mathematically explainable and thus generalizable to different pruning tasks. While the evolution is conducted on small datasets, our functions shows promising results when applied to more challenging datasets, different from those used in the evolution process. For example, on ILSVRC-2012, an evolved function achieves state-of-the-art pruning results.

cs.CV↗

arXiv2022

Impact of the turnover in the high-z galaxy luminosity function on the 21-cm signal during Cosmic Dawn and Epoch of Reionization

The shape of the faint-end of the high-z galaxy luminosity function (LF) informs early star formation and reionization physics during the Cosmic Dawn and Epoch of Reionization. Until recently, based on the strong gravitational lensing cluster deep surveys, the Hubble Frontier Fields (HFF) has found a potential turnover in the ultraviolet (UV) LF at z$\sim$6. In this paper, we analyze the contribution of extremely faint galaxies with the magnitude larger than the turnover magnitude in LF to cosmic reionization. We apply the measurement from HFF to our suppressed star formation efficiency model, including three free parameters: halo mass threshold $M_t$, curvature parameter $β$ and a UV conversion factor $l_{\rm UV}$. According to our fit of 68\% confidence level, the high-redshift star formation in haloes smaller than $ M_t=1.82^{+2.86}_{-1.08}\times10^{10} \rm M_{\odot}$ is found to be dampened. The turnover magnitude $\rm \gtrsim -13.99-2.45$, correspondingly the halo mass $\lesssim(4.57+20.03)\times10^{9} \rm M_{\odot}$. We find that the absorption trough in the global 21-cm signal is sensitive to our SFE model parameters. Together with ($β$, $l_{\rm UV}$) = ($2.17^{+2.42}_{-1.72}$, $9.33^{+0.43}_{-0.42} \rm ~erg~yr ~s^{-1}M_{\odot}^{-1})$, the trough locates at $\sim$ $134^{+10}_{-17}$ $\rm MHz$ with an amplitude of $\sim$ $-237^{-6}_{+7}$ $\rm mK$, compared to (106\rm MHz, -212\rm mK) in the absence of turnover. Besides, we find that the star formation of faint galaxies has also an impact on the 21-cm power spectra. The best fitting peak power decreases by $\sim4\%$ and shifts towards smaller scales from $0.88 h \rm Mpc^{-1}$ to $0.91 h \rm Mpc^{-1}$. According to our calculation, such impact is distinguishable with the forthcoming Square Kilometre Array.

astro-ph.GA↗

arXiv2022

SdAE: Self-distillated Masked Autoencoder

With the development of generative-based self-supervised learning (SSL) approaches like BeiT and MAE, how to learn good representations by masking random patches of the input image and reconstructing the missing information has grown in concern. However, BeiT and PeCo need a "pre-pretraining" stage to produce discrete codebooks for masked patches representing. MAE does not require a pre-training codebook process, but setting pixels as reconstruction targets may introduce an optimization gap between pre-training and downstream tasks that good reconstruction quality may not always lead to the high descriptive capability for the model. Considering the above issues, in this paper, we propose a simple Self-distillated masked AutoEncoder network, namely SdAE. SdAE consists of a student branch using an encoder-decoder structure to reconstruct the missing information, and a teacher branch producing latent representation of masked tokens. We also analyze how to build good views for the teacher branch to produce latent representation from the perspective of information bottleneck. After that, we propose a multi-fold masking strategy to provide multiple masked views with balanced information for boosting the performance, which can also reduce the computational complexity. Our approach generalizes well: with only 300 epochs pre-training, a vanilla ViT-Base model achieves an 84.1% fine-tuning accuracy on ImageNet-1k classification, 48.6 mIOU on ADE20K segmentation, and 48.9 mAP on COCO detection, which surpasses other methods by a considerable margin. Code is available at https://github.com/AbrahamYabo/SdAE.

cs.CV↗

arXiv2022

M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database

The emotional state of a speaker can be influenced by many different factors in dialogues, such as dialogue scene, dialogue topic, and interlocutor stimulus. The currently available data resources to support such multimodal affective analysis in dialogues are however limited in scale and diversity. In this work, we propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset, M3ED, which contains 990 dyadic emotional dialogues from 56 different TV series, a total of 9,082 turns and 24,449 utterances. M3 ED is annotated with 7 emotion categories (happy, surprise, sad, disgust, anger, fear, and neutral) at utterance level, and encompasses acoustic, visual, and textual modalities. To the best of our knowledge, M3ED is the first multimodal emotional dialogue dataset in Chinese. It is valuable for cross-culture emotion analysis and recognition. We apply several state-of-the-art methods on the M3ED dataset to verify the validity and quality of the dataset. We also propose a general Multimodal Dialogue-aware Interaction framework, MDI, to model the dialogue context for emotion recognition, which achieves comparable performance to the state-of-the-art methods on the M3ED. The full dataset and codes are available.

cs.CL↗

arXiv2022

Subspace Nonnegative Matrix Factorization for Feature Representation

Traditional nonnegative matrix factorization (NMF) learns a new feature representation on the whole data space, which means treating all features equally. However, a subspace is often sufficient for accurate representation in practical applications, and redundant features can be invalid or even harmful. For example, if a camera has some sensors destroyed, then the corresponding pixels in the photos from this camera are not helpful to identify the content, which means only the subspace consisting of remaining pixels is worthy of attention. This paper proposes a new NMF method by introducing adaptive weights to identify key features in the original space so that only a subspace involves generating the new representation. Two strategies are proposed to achieve this: the fuzzier weighted technique and entropy regularized weighted technique, both of which result in an iterative solution with a simple form. Experimental results on several real-world datasets demonstrated that the proposed methods can generate a more accurate feature representation than existing methods. The code developed in this study is available at https://github.com/WNMF1/FWNMF-ERWNMF.

cs.CV↗

arXiv2022

Multi-modal Emotion Estimation for in-the-wild Videos

In this paper, we briefly introduce our submission to the Valence-Arousal Estimation Challenge of the 3rd Affective Behavior Analysis in-the-wild (ABAW) competition. Our method utilizes the multi-modal information, i.e., the visual and audio information, and employs a temporal encoder to model the temporal context in the videos. Besides, a smooth processor is applied to get more reasonable predictions, and a model ensemble strategy is used to improve the performance of our proposed method. The experiment results show that our method achieves 65.55% ccc for valence and 70.88% ccc for arousal on the validation set of the Aff-Wild2 dataset, which prove the effectiveness of our proposed method.

cs.CV↗

arXiv2022

A Deep Reinforcement Learning Environment for Particle Robot Navigation and Object Manipulation

Particle robots are novel biologically-inspired robotic systems where locomotion can be achieved collectively and robustly, but not independently. While its control is currently limited to a hand-crafted policy for basic locomotion tasks, such a multi-robot system could be potentially controlled via Deep Reinforcement Learning (DRL) for different tasks more efficiently. However, the particle robot system presents a new set of challenges for DRL differing from existing swarm robotics systems: the low degrees of freedom of each robot and the increased necessity of coordination between robots. We present a 2D particle robot simulator using the OpenAI Gym interface and Pymunk as the physics engine, and introduce new tasks and challenges to research the underexplored applications of DRL in the particle robot system. Moreover, we use Stable-baselines3 to provide a set of benchmarks for the tasks. Current baseline DRL algorithms show signs of achieving the tasks but are yet unable to reach the performance of the hand-crafted policy. Further development of DRL algorithms is necessary in order to accomplish the proposed tasks.

cs.RO↗

arXiv2022

Emotions in Online Content Diffusion

Social media-transmitted online information, which is associated with emotional expressions, shapes our thoughts and actions. In this study, we incorporate social network theories and analyses and use a computational approach to investigate how emotional expressions, particularly \textit{negative discrete emotional expressions} (i.e., anxiety, sadness, anger, and disgust), lead to differential diffusion of online content in social media networks. We rigorously quantify diffusion cascades' structural properties (i.e., size, depth, maximum breadth, and structural virality) and analyze the individual characteristics (i.e., age, gender, and network degree) and social ties (i.e., strong and weak) involved in the cascading process. In our sample, more than six million unique individuals transmitted 387,486 randomly selected articles in a massive-scale online social network, WeChat. We detect the expression of discrete emotions embedded in these articles, using a newly generated domain-specific and up-to-date emotion lexicon. We apply a partial-linear instrumental variable approach with a double machine learning framework to causally identify the impact of the negative discrete emotions on online content diffusion. We find that articles with more expressions of anxiety spread to a larger number of individuals and diffuse more deeply, broadly, and virally. Expressions of anger and sadness, however, reduce cascades' size and maximum breadth. We further show that the articles with different degrees of negative emotional expressions tend to spread differently based on individual characteristics and social ties. Our results shed light on content marketing and regulation, utilizing negative emotional expressions.

econ.GN↗

arXiv2022

Towards the Maximum Traffic Demand and Throughput Supported by Relay-Assisted mmWave Backhaul Networks

This paper investigates the throughput performance issue of the relay-assisted mmWave backhaul network. The maximum traffic demand of small-cell base stations (BSs) and the maximum throughput at the macro-cell BS have been found in a tree-style backhaul network through linear programming under different network settings, which concern both the number of radio chains available on BSs and the interference relationship between logical links in the backhaul network. A novel interference model for the relay-assisted mmWave backhaul network in the dense urban environment is proposed, which demonstrates the limited interference footprint of mmWave directional communications. Moreover, a scheduling algorithm is developed to find the optimal scheduling for tree-style mmWave backhaul networks. Extensive numerical analysis and simulations are conducted to show and validate the network throughput performance and the scheduling algorithm.

cs.NI↗

arXiv2022

An efficient distributed scheduling algorithm for relay-assisted mmWave backhaul networks

In this paper, a novel distributed scheduling algorithm is proposed, which aims to efficiently schedule both the uplink and downlink backhaul traffic in the relay-assisted mmWave backhaul network with a tree topology. The handshaking of control messages, calculation of local schedules, and the determination of final valid schedule are all discussed. Simulation results show that the performance of the distributed algorithm can reach very close to the maximum traffic demand of the backhaul network, and it can also adapt to the dynamic traffic with sharp traffic demand change of small-cell BSs quickly and accurately.

cs.NI↗

arXiv2022

Finite generation for valuations computing stability thresholds and applications to K-stability

We prove that on any log Fano pair of dimension $n$ whose stability threshold is less than $\frac{n+1}{n}$, any valuation computing the stability threshold has a finitely generated associated graded ring. Together with earlier works, this implies: (a) a log Fano pair is uniformly K-stable (resp. reduced uniformly K-stable) if and only if it is K-stable (resp. K-polystable); (b) the K-moduli spaces are proper and projective; and combining with the previously known equivalence between the existence of Kähler-Einstein metric and reduced uniform K-stability proved by the variational approach, (c) the Yau-Tian-Donaldson conjecture holds for general (possibly singular) log Fano pairs.

math.AG↗