arXiv subjects

Liang Chen

Publications and source records attributed to Liang Chen.

At least 325 records · Page 18Linked to original sources

arXiv2023

Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularization

Generating persona consistent dialogue response is important for developing an intelligent conversational agent. Recent works typically fine-tune large-scale pre-trained models on this task by concatenating persona texts and dialogue history as a single input sequence to generate the target response. While simple and effective, our analysis shows that this popular practice is seriously affected by order sensitivity where different input orders of persona sentences significantly impact the quality and consistency of generated response, resulting in severe performance fluctuations (i.e., 29.4% on GPT2 and 83.2% on BART). To mitigate the order sensitivity problem, we propose a model-agnostic framework, ORder Insensitive Generation (ORIG), which enables dialogue models to learn robust representation under different persona orders and improve the consistency of response generation. Experiments on the Persona-Chat dataset justify the effectiveness and superiority of our method with two dominant pre-trained models (GPT2 and BART).

cs.CL↗

arXiv2023

Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks

Recent years have seen a surge in research on dynamic graph representation learning, which aims to model temporal graphs that are dynamic and evolving constantly over time. However, current work typically models graph dynamics with recurrent neural networks (RNNs), making them suffer seriously from computation and memory overheads on large temporal graphs. So far, scalability of dynamic graph representation learning on large temporal graphs remains one of the major challenges. In this paper, we present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. We explore a new direction in that we can capture the evolving dynamics of temporal graphs with spiking neural networks (SNNs) instead of RNNs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations and enable spike-based propagation in an efficient way. Experiments on three large real-world temporal graph datasets demonstrate that SpikeNet outperforms strong baselines on the temporal node classification task with lower computational costs. Particularly, SpikeNet generalizes to a large temporal graph (2.7M nodes and 13.9M edges) with significantly fewer parameters and computation overheads.Our code is publicly available at \url{https://github.com/EdisonLeeeee/SpikeNet}.

cs.NE↗

arXiv2023

Less Can Be More: Unsupervised Graph Pruning for Large-scale Dynamic Graphs

The prevalence of large-scale graphs poses great challenges in time and storage for training and deploying graph neural networks (GNNs). Several recent works have explored solutions for pruning the large original graph into a small and highly-informative one, such that training and inference on the pruned and large graphs have comparable performance. Although empirically effective, current researches focus on static or non-temporal graphs, which are not directly applicable to dynamic scenarios. In addition, they require labels as ground truth to learn the informative structure, limiting their applicability to new problem domains where labels are hard to obtain. To solve the dilemma, we propose and study the problem of unsupervised graph pruning on dynamic graphs. We approach the problem by our proposed STEP, a self-supervised temporal pruning framework that learns to remove potentially redundant edges from input dynamic graphs. From a technical and industrial viewpoint, our method overcomes the trade-offs between the performance and the time & memory overheads. Our results on three real-world datasets demonstrate the advantages on improving the efficacy, robustness, and efficiency of GNNs on dynamic node classification tasks. Most notably, STEP is able to prune more than 50% of edges on a million-scale industrial graph Alipay (7M nodes, 21M edges) while approximating up to 98% of the original performance. Code is available at https://github.com/EdisonLeeeee/STEP.

cs.LG↗

arXiv2023

Transporting cold atoms towards a GaN-on-sapphire chip via an optical conveyor belt

Trapped atoms on photonic structures inspire many novel quantum devices for quantum information processing and quantum sensing. Here, we have demonstrated a hybrid photonic-atom chip platform based on a GaN-on-sapphire chip and the transport of an ensemble of atoms from free space towards the chip with an optical conveyor belt. The maximum transport efficiency of atoms is about 50% with a transport distance of 500 $\mathrm{μm}$. Our results open up a new route toward the efficiently loading of cold atoms into the evanescent-field trap formed by the photonic integrated circuits, which promises strong and controllable interactions between single atoms and single photons.

physics.atom-ph↗

arXiv2023

Recent Advances in Reliable Deep Graph Learning: Inherent Noise, Distribution Shift, and Adversarial Attack

Deep graph learning (DGL) has achieved remarkable progress in both business and scientific areas ranging from finance and e-commerce to drug and advanced material discovery. Despite the progress, applying DGL to real-world applications faces a series of reliability threats including inherent noise, distribution shift, and adversarial attacks. This survey aims to provide a comprehensive review of recent advances for improving the reliability of DGL algorithms against the above threats. In contrast to prior related surveys which mainly focus on adversarial attacks and defense, our survey covers more reliability-related aspects of DGL, i.e., inherent noise and distribution shift. Additionally, we discuss the relationships among above aspects and highlight some important issues to be explored in future research.

cs.LG↗

arXiv2023

Estimation of Characteristics-based Quantile Factor Models

This paper studies the estimation of characteristic-based quantile factor models where the factor loadings are unknown functions of observed individual characteristics while the idiosyncratic error terms are subject to conditional quantile restrictions. We propose a three-stage estimation procedure that is easily implementable in practice and has nice properties. The convergence rates, the limiting distributions of the estimated factors and loading functions, and a consistent selection criterion for the number of factors at each quantile are derived under general conditions. The proposed estimation methodology is shown to work satisfactorily when: (i) the idiosyncratic errors have heavy tails, (ii) the time dimension of the panel dataset is not large, and (iii) the number of factors exceeds the number of characteristics. Finite sample simulations and an empirical application aimed at estimating the loading functions of the daily returns of a large panel of S\&P500 index securities help illustrate these properties.

econ.EM↗

arXiv2023

Common Correlated Effects Estimation of Nonlinear Panel Data Models

This paper focuses on estimating the coefficients and average partial effects of observed regressors in nonlinear panel data models with interactive fixed effects, using the common correlated effects (CCE) framework. The proposed two-step estimation method involves applying principal component analysis to estimate latent factors based on cross-sectional averages of the regressors in the first step, and jointly estimating the coefficients of the regressors and factor loadings in the second step. The asymptotic distributions of the proposed estimators are derived under general conditions, assuming that the number of time-series observations is comparable to the number of cross-sectional observations. To correct for asymptotic biases of the estimators, we introduce both analytical and split-panel jackknife methods, and confirm their good performance in finite samples using Monte Carlo simulations. An empirical application utilizes the proposed method to study the arbitrage behaviour of nonfinancial firms across different security markets.

econ.EM↗

arXiv2023

Curriculum Modeling the Dependence among Targets with Multi-task Learning for Financial Marketing

Multi-task learning for various real-world applications usually involves tasks with logical sequential dependence. For example, in online marketing, the cascade behavior pattern of $impression \rightarrow click \rightarrow conversion$ is usually modeled as multiple tasks in a multi-task manner, where the sequential dependence between tasks is simply connected with an explicitly defined function or implicitly transferred information in current works. These methods alleviate the data sparsity problem for long-path sequential tasks as the positive feedback becomes sparser along with the task sequence. However, the error accumulation and negative transfer will be a severe problem for downstream tasks. Especially, at the beginning stage of training, the optimization for parameters of former tasks is not converged yet, and thus the information transferred to downstream tasks is negative. In this paper, we propose a prior information merged model (\textbf{PIMM}), which explicitly models the logical dependence among tasks with a novel prior information merged (\textbf{PIM}) module for multiple sequential dependence task learning in a curriculum manner. Specifically, the PIM randomly selects the true label information or the prior task prediction with a soft sampling strategy to transfer to the downstream task during the training. Following an easy-to-difficult curriculum paradigm, we dynamically adjust the sampling probability to ensure that the downstream task will get the effective information along with the training. The offline experimental results on both public and product datasets verify that PIMM outperforms state-of-the-art baselines. Moreover, we deploy the PIMM in a large-scale FinTech platform, and the online experiments also demonstrate the effectiveness of PIMM.

cs.IR↗

arXiv2023

Improved Test-Time Adaptation for Domain Generalization

The main challenge in domain generalization (DG) is to handle the distribution shift problem that lies between the training and test data. Recent studies suggest that test-time training (TTT), which adapts the learned model with test data, might be a promising solution to the problem. Generally, a TTT strategy hinges its performance on two main factors: selecting an appropriate auxiliary TTT task for updating and identifying reliable parameters to update during the test phase. Both previous arts and our experiments indicate that TTT may not improve but be detrimental to the learned model if those two factors are not properly considered. This work addresses those two factors by proposing an Improved Test-Time Adaptation (ITTA) method. First, instead of heuristically defining an auxiliary objective, we propose a learnable consistency loss for the TTT task, which contains learnable parameters that can be adjusted toward better alignment between our TTT task and the main prediction task. Second, we introduce additional adaptive parameters for the trained model, and we suggest only updating the adaptive parameters during the test phase. Through extensive experiments, we show that the proposed two strategies are beneficial for the learned model (see Figure 1), and ITTA could achieve superior performance to the current state-of-the-art methods on several DG benchmarks. Code is available at https://github.com/liangchen527/ITTA.

cs.CV↗

arXiv2023

Manipulating Federated Recommender Systems: Poisoning with Synthetic Users and Its Countermeasures

Federated Recommender Systems (FedRecs) are considered privacy-preserving techniques to collaboratively learn a recommendation model without sharing user data. Since all participants can directly influence the systems by uploading gradients, FedRecs are vulnerable to poisoning attacks of malicious clients. However, most existing poisoning attacks on FedRecs are either based on some prior knowledge or with less effectiveness. To reveal the real vulnerability of FedRecs, in this paper, we present a new poisoning attack method to manipulate target items' ranks and exposure rates effectively in the top-$K$ recommendation without relying on any prior knowledge. Specifically, our attack manipulates target items' exposure rate by a group of synthetic malicious users who upload poisoned gradients considering target items' alternative products. We conduct extensive experiments with two widely used FedRecs (Fed-NCF and Fed-LightGCN) on two real-world recommendation datasets. The experimental results show that our attack can significantly improve the exposure rate of unpopular target items with extremely fewer malicious users and fewer global epochs than state-of-the-art attacks. In addition to disclosing the security hole, we design a novel countermeasure for poisoning attacks on FedRecs. Specifically, we propose a hierarchical gradient clipping with sparsified updating to defend against existing poisoning attacks. The empirical results demonstrate that the proposed defending mechanism improves the robustness of FedRecs.

cs.IR↗

arXiv2023

Cross-View Hierarchy Network for Stereo Image Super-Resolution

Stereo image super-resolution aims to improve the quality of high-resolution stereo image pairs by exploiting complementary information across views. To attain superior performance, many methods have prioritized designing complex modules to fuse similar information across views, yet overlooking the importance of intra-view information for high-resolution reconstruction. It also leads to problems of wrong texture in recovered images. To address this issue, we explore the interdependencies between various hierarchies from intra-view and propose a novel method, named Cross-View-Hierarchy Network for Stereo Image Super-Resolution (CVHSSR). Specifically, we design a cross-hierarchy information mining block (CHIMB) that leverages channel attention and large kernel convolution attention to extract both global and local features from the intra-view, enabling the efficient restoration of accurate texture details. Additionally, a cross-view interaction module (CVIM) is proposed to fuse similar features from different views by utilizing cross-view attention mechanisms, effectively adapting to the binocular scene. Extensive experiments demonstrate the effectiveness of our method. CVHSSR achieves the best stereo image super-resolution performance than other state-of-the-art methods while using fewer parameters. The source code and pre-trained models are available at https://github.com/AlexZou14/CVHSSR.

cs.CV↗

arXiv2023

MR Elastography with Optimization-Based Phase Unwrapping and Traveling Wave Expansion-based Neural Network (TWENN)

Magnetic Resonance Elastography (MRE) can characterize biomechanical properties of soft tissue for disease diagnosis and treatment planning. However, complicated wavefields acquired from MRE coupled with noise pose challenges for accurate displacement extraction and modulus estimation. Here we propose a pipeline for processing MRE images using optimization-based displacement extraction and Traveling Wave Expansion-based Neural Network (TWENN) modulus estimation. Phase unwrapping and displacement extraction were achieved by optimization of an objective function with Dual Data Consistency (Dual-DC). A complex-valued neural network using displacement covariance as input has been constructed for the estimation of complex wavenumbers. A model of traveling wave expansion is used to generate training datasets with different levels of noise for the network. The complex shear modulus map is obtained by a fusion of multifrequency and multidirectional data. Validation using images of brain and liver simulation demonstrates the practical value of the proposed pipeline, which can estimate the biomechanical properties with minimum root-mean-square-errors compared with state-of-the-art methods. Applications of the proposed method for processing MRE images of phantom, brain, and liver show clear anatomical features and that the pipeline is robust to noise and has a good generalization capability.

eess.IV↗

arXiv2023

Radio emission of tidal disruption events from wind-cloud interaction

Winds can be launched in tidal disruption event (TDE). It has been proposed that the winds can interact with the cloud surrounding the black hole, produce bow shocks, accelerate electrons, and produce radio emission. We restudy the wind-cloud interaction model. We employ the properties of winds found by the radiation hydrodynamic simulations of super-Eddington circularized accretion flow in TDEs. We can calculate the peak radio emission frequency, the luminosity at the peak frequency, and their time-evolution based on the TDEs wind-cloud interaction model. We find that the model predicted peak radio emission frequency, the luminosity at peak frequency, and their time evolution can be well consistent with those in TDEs AT2019dsg and ASASSN-14li. This indicates that in these two radio TDEs, the wind-cloud interaction mechanism may be responsible for the radio emission.

astro-ph.HE↗

arXiv2023

Neighborhood Consensus Contrastive Learning for Backward-Compatible Representation

In object re-identification (ReID), the development of deep learning techniques often involves model updates and deployment. It is unbearable to re-embedding and re-index with the system suspended when deploying new models. Therefore, backward-compatible representation is proposed to enable "new" features to be compared with "old" features directly, which means that the database is active when there are both "new" and "old" features in it. Thus we can scroll-refresh the database or even do nothing on the database to update. The existing backward-compatible methods either require a strong overlap between old and new training data or simply conduct constraints at the instance level. Thus they are difficult in handling complicated cluster structures and are limited in eliminating the impact of outliers in old embeddings, resulting in a risk of damaging the discriminative capability of new features. In this work, we propose a Neighborhood Consensus Contrastive Learning (NCCL) method. With no assumptions about the new training data, we estimate the sub-cluster structures of old embeddings. A new embedding is constrained with multiple old embeddings in both embedding space and discrimination space at the sub-class level. The effect of outliers diminished, as the multiple samples serve as "mean teachers". Besides, we also propose a scheme to filter the old embeddings with low credibility, further improving the compatibility robustness. Our method ensures backward compatibility without impairing the accuracy of the new model. And it can even improve the new model's accuracy in most scenarios.

cs.CV↗

arXiv2023

Locally Spontaneous Dynamic Oxygen Migration on Biphenylene: A DFT Study

The dynamic oxygen migration at the interface of carbon allotropes dominated by the periodic hexagonal rings, including graphene and carbon nanotube, has opened up a new avenue to realize dynamic covalent materials. However, for the carbon materials with hybrid carbon rings, such as biphenylene, whether the dynamic oxygen migration at its interface can still be found remains unknown. Using both density functional theory calculations and machine-learning-based molecular dynamics (MLMD) simulations, we found that the oxygen migration departing away from the four-membered carbon (C4) ring is hindered, and the oxygen atom prefers to spontaneously migrate toward/around the C4 ring. This locally spontaneous dynamic oxygen migration on the biphenylene is attributed to the high barrier of about 1.5 eV for the former process and relatively low barrier of about 0.3 eV for the latter one, originating from the enhanced activity of C-O bond near/around the C4 ring due to the hybrid carbon rings structure. Moreover, the locally spontaneous dynamic oxygen migration is further confirmed by MLMD simulations. This work sheds light on the potential of biphenylene as a catalyst for spatial-controlled energy conversion and provides the guidance for realizing the dynamic covalent interface at other carbon-based or two-dimensional materials.

cond-mat.mtrl-sci↗

arXiv2023

Solution of Volume Integral and Hydrodynamic Equations to Analyze Electromagnetic Scattering from Composite Nanostructures

A coupled system of volume integral and hydrodynamic equations is solved to analyze electromagnetic scattering from nanostructures consisting of metallic and dielectric parts. In the metallic part, the hydrodynamic equation relates the free electron polarization current to the electric flux and effectively "updates" the constitutive relation to enable the modeling of nonlocality. In the metallic and the dielectric parts, the volume integral equation relates the electric flux and the free electron polarization current to the scattered electric field. Unknown electric flux and free electron polarization current are expanded using Schaubert-Wilton-Glisson basis functions. Inserting these expansions into the coupled system of the volume integral and hydrodynamic equations and using Galerkin testing yield a matrix system in unknown expansion coefficients. An efficient two-level iterative solver is proposed to solve this matrix system. This approach "inverts" the discretized hydrodynamic equation for the coefficients of the free electron polarization current and substitutes the result in the discretized volume integral equation. Outer iterations solve this reduced matrix system while the inner iterations invert the discretized hydrodynamic equation at every iteration of the outer iterations. Numerical experiments are carried out to demonstrate the accuracy, the efficiency, and the applicability of the proposed method.

physics.comp-ph↗

arXiv2023

Tunable correlation effects of magnetic impurities by the cubic Rashba spin-orbit couplings

We theoretically study the influence of the $k$-cubic Rashba spin-orbit coupling (SOC) on the correlation effects of magnetic impurities by combining the variational method and the Hirsch-Fye quantum Monte Carlo (HFQMC) simulations. Markedly different from the normal $k$-linear Rashba SOC, even a small cubic Rashba term can greatly alter the band structure and induce a Van Hove singularity in a wide range of energy, thus the single impurity local moment becomes largely tunable. The cubic Rashba SOC adopted in this work breaks the rotational symmetry, but the host material is still invariant under the operations $\mathcal{R}^z(π)$, $\mathcal{IR}^z(π/2)$, $\mathcal{M}_{xz}$, $\mathcal{M}_{yz}$, where $\mathcal{R}^z(θ)$ is the rotation of angle $θ$ about the $z$-axis, $\mathcal{I}$ is the inversion operator and $\mathcal{M}_{xz}$ ($\mathcal{M}_{yz}$) is the mirror reflection about the $x$-$z$ ($y$-$z$) principal plane. Saliently, various components of spin-spin correlation between the single magnetic impurity and the conduction electrons show three- or six-fold rotational symmetry. This unique feature is due to the triple winding of the spins with a $2π$ rotation of $\mathbf{k}$, which is a hallmark of the cubic Rashba effect and can possibly be an identifier to distinguish the cubic Rashba SOC from the normal $k$-linear Rashba term in experiments. Although the cubic Rashba term drastically alters the electronic properties of the host, we find that the spatial decay rate of the spin-spin correlation function remains essentially unchanged. Moreover, the carrier-mediated Ruderman-Kittel-Kasuya-Yosida interactions between two magnetic impurities show twisted features, the ferromagnetic diagonal terms dominate when two magnetic impurities are very close, but the off-diagonal terms become important at long distances.

cond-mat.str-el↗

arXiv2023

Self-Sampling Training and Evaluation for the Accuracy-Bias Tradeoff in Recommendation

Research on debiased recommendation has shown promising results. However, some issues still need to be handled for its application in industrial recommendation. For example, most of the existing methods require some specific data, architectures and training methods. In this paper, we first argue through an online study that arbitrarily removing all the biases in industrial recommendation may not consistently yield a desired performance improvement. For the situation that a randomized dataset is not available, we propose a novel self-sampling training and evaluation (SSTE) framework to achieve the accuracy-bias tradeoff in recommendation, i.e., eliminate the harmful biases and preserve the beneficial ones. Specifically, SSTE uses a self-sampling module to generate some subsets with different degrees of bias from the original training and validation data. A self-training module infers the beneficial biases and learns better tradeoff based on these subsets, and a self-evaluation module aims to use these subsets to construct more plausible references to reflect the optimized model. Finally, we conduct extensive offline experiments on two datasets to verify the effectiveness of our SSTE. Moreover, we deploy our SSTE in homepage recommendation of a famous financial management product called Tencent Licaitong, and find very promising results in an online A/B test.

cs.IR↗