arXiv subjects

Liang Wang

Publications and source records attributed to Liang Wang.

At least 415 records · Page 23Linked to original sources

arXiv2023

Knowledge Augmented Relation Inference for Group Activity Recognition

Most existing group activity recognition methods construct spatial-temporal relations merely based on visual representation. Some methods introduce extra knowledge, such as action labels, to build semantic relations and use them to refine the visual presentation. However, the knowledge they explored just stay at the semantic-level, which is insufficient for pursing notable accuracy. In this paper, we propose to exploit knowledge concretization for the group activity recognition, and develop a novel Knowledge Augmented Relation Inference framework that can effectively use the concretized knowledge to improve the individual representations. Specifically, the framework consists of a Visual Representation Module to extract individual appearance features, a Knowledge Augmented Semantic Relation Module explore semantic representations of individual actions, and a Knowledge-Semantic-Visual Interaction Module aims to integrate visual and semantic information by the knowledge. Benefiting from these modules, the proposed framework can utilize knowledge to enhance the relation inference process and the individual representations, thus improving the performance of group activity recognition. Experimental results on two public datasets show that the proposed framework achieves competitive performance compared with state-of-the-art methods.

cs.CV↗

arXiv2023

Securing IoT Communication using Physical Sensor Data -- Graph Layer Security with Federated Multi-Agent Deep Reinforcement Learning

Internet-of-Things (IoT) devices are often used to transmit physical sensor data over digital wireless channels. Traditional Physical Layer Security (PLS)-based cryptography approaches rely on accurate channel estimation and information exchange for key generation, which irrevocably ties key quality with digital channel estimation quality. Recently, we proposed a new concept called Graph Layer Security (GLS), where digital keys are derived from physical sensor readings. The sensor readings between legitimate users are correlated through a common background infrastructure environment (e.g., a common water distribution network or electric grid). The challenge for GLS has been how to achieve distributed key generation. This paper presents a Federated multi-agent Deep reinforcement learning-assisted Distributed Key generation scheme (FD2K), which fully exploits the common features of physical dynamics to establish secret key between legitimate users. We present for the first time initial experimental results of GLS with federated learning, achieving considerable security performance in terms of key agreement rate (KAR), and key randomness.

cs.AI↗

arXiv2023

ATA-Cache: Contention Mitigation for GPU Shared L1 Cache with Aggregated Tag Array

GPU shared L1 cache is a promising architecture while still suffering from high resource contentions. We present a GPU shared L1 cache architecture with an aggregated tag array that minimizes the L1 cache contentions and takes full advantage of inter-core locality. The key idea is to decouple and aggregate the tag arrays of multiple L1 caches so that the cache requests can be compared with all tag arrays in parallel to probe the replicated data in other caches. The GPU caches are only accessed by other GPU cores when replicated data exists, filtering out unnecessary cache accesses that cause high resource contentions. The experimental results show that GPU IPC can be improved by 12% on average for applications with a high inter-core locality.

cs.AR↗

arXiv2023

How Effective is Multiple-Vantage-Point Domain Control Validation?

Multiple-vantage-point domain control validation (multiVA) is an emerging defense for mitigating BGP hijacking attacks against certificate authorities. While the adoption of multiVA is on the rise, little work has quantified its effectiveness against BGP hijacks in the wild. We bridge the gap by presenting the first analysis framework that measures the security of a multiVA deployment under real-world network configurations (e.g., DNS and RPKI). Our framework accurately models the attack surface of multiVA by 1) considering the attacks on DNS nameservers involved in domain validation, 2) considering deployed practical security techniques such as RPKI, 3) performing fine-grained internet-scale analysis to compute multiVA resilience (i.e., how difficult it is to launch a BGP hijack against a domain and get a bogus certificate under multiVA). We use our framework to perform a rigorous security analysis of the multiVA deployment of Let's Encrypt, using a dataset that consists of about 1 million certificates and 31 billion DNS queries collected over four months. Our analysis shows while DNS does enlarge the attack surface of multiVA, the of Let's Encrypt's multiVA deployment still offers an 88% median resilience against BGP hijacks, a notable improvement over 76% offered by single-vantage-point validation. RPKI, even in its current state of partial deployment, effectively mitigates BGP attacks and improves the security of the deployment by 15% as compared to the case without considering RPKI. Exploring 11,000 different multiVA configurations, we find that Let's Encrypt's deployment can be further enhanced to achieve a resilience of over 99% by using a full quorum policy with only two additional vantage points in different public clouds.

cs.CR↗

arXiv2023

Sketch Less Face Image Retrieval: A New Challenge

In some specific scenarios, face sketch was used to identify a person. However, drawing a complete face sketch often needs skills and takes time, which hinder its widespread applicability in the practice. In this study, we proposed a new task named sketch less face image retrieval (SLFIR), in which the retrieval was carried out at each stroke and aim to retrieve the target face photo using a partial sketch with as few strokes as possible (see Fig.1). Firstly, we developed a method to generate the data of sketch with drawing process, and opened such dataset; Secondly, we proposed a two-stage method as the baseline for SLFIR that (1) A triplet network, was first adopt to learn the joint embedding space shared between the complete sketch and its target face photo; (2) Regarding the sketch drawing episode as a sequence, we designed a LSTM module to optimize the representation of the incomplete face sketch. Experiments indicate that the new framework can finish the retrieval using a partial or pool drawing sketch.

cs.CV↗

arXiv2023

Inferring synchronizability of networked heterogenous oscillators with machine learning

In the study of network synchronization, an outstanding question of both theoretical and practical significance is how to allocate a given set of heterogenous oscillators on a complex network in order for improving the synchronization performance. Whereas methods have been proposed to address this question in literature, the methods are based on accurate models describing the system dynamics, which, however, are normally unavailable in realistic situations. Here we show that this question can be addressed by the model-free technique of feed-forward neural network (FNN) in machine learning. Specifically, we measure the synchronization performance of a number of allocation schemes and use the measured data to train a machine. It is found that the trained machine is able to not only infer the synchronization performance of any new allocation scheme, but also find from a huge amount of candidates the optimal allocation scheme for synchronization.

nlin.AO↗

arXiv2023

Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition

Skeleton-based action recognition is widely used in varied areas, e.g., surveillance and human-machine interaction. Existing models are mainly learned in a supervised manner, thus heavily depending on large-scale labeled data which could be infeasible when labels are prohibitively expensive. In this paper, we propose a novel Contrast-Reconstruction Representation Learning network (CRRL) that simultaneously captures postures and motion dynamics for unsupervised skeleton-based action recognition. It mainly consists of three parts: Sequence Reconstructor, Contrastive Motion Learner, and Information Fuser. The Sequence Reconstructor learns representation from skeleton coordinate sequence via reconstruction, thus the learned representation tends to focus on trivial postural coordinates and be hesitant in motion learning. To enhance the learning of motions, the Contrastive Motion Learner performs contrastive learning between the representations learned from coordinate sequence and additional velocity sequence, respectively. Finally, in the Information Fuser, we explore varied strategies to combine the Sequence Reconstructor and Contrastive Motion Learner, and propose to capture postures and motions simultaneously via a knowledge-distillation based fusion strategy that transfers the motion learning from the Contrastive Motion Learner to the Sequence Reconstructor. Experimental results on several benchmarks, i.e., NTU RGB+D 60, NTU RGB+D 120, CMU mocap, and NW-UCLA, demonstrate the promise of the proposed CRRL method by far outperforming state-of-the-art approaches.

cs.CV↗

arXiv2023

Hybrid Contrastive Constraints for Multi-Scenario Ad Ranking

Multi-scenario ad ranking aims at leveraging the data from multiple domains or channels for training a unified ranking model to improve the performance at each individual scenario. Although the research on this task has made important progress, it still lacks the consideration of cross-scenario relations, thus leading to limitation in learning capability and difficulty in interrelation modeling. In this paper, we propose a Hybrid Contrastive Constrained approach (HC^2) for multi-scenario ad ranking. To enhance the modeling of data interrelation, we elaborately design a hybrid contrastive learning approach to capture commonalities and differences among multiple scenarios. The core of our approach consists of two elaborated contrastive losses, namely generalized and individual contrastive loss, which aim at capturing common knowledge and scenario-specific knowledge, respectively. To adapt contrastive learning to the complex multi-scenario setting, we propose a series of important improvements. For generalized contrastive loss, we enhance contrastive learning by extending the contrastive samples (label-aware and diffusion noise enhanced contrastive samples) and reweighting the contrastive samples (reciprocal similarity weighting). For individual contrastive loss, we use the strategies of dropout-based augmentation and {cross-scenario encoding} for generating meaningful positive and negative contrastive samples, respectively. Extensive experiments on both offline evaluation and online test have demonstrated the effectiveness of the proposed HC$^2$ by comparing it with a number of competitive baselines.

cs.IR↗

arXiv2023

Modeling Adaptive Fine-grained Task Relatedness for Joint CTR-CVR Estimation

In modern advertising and recommender systems, multi-task learning (MTL) paradigm has been widely employed to jointly predict diverse user feedbacks (e.g. click and purchase). While, existing MTL approaches are either rigid to adapt to different scenarios, or only capture coarse-grained task relatedness, thus making it difficult to effectively transfer knowledge across tasks. To address these issues, in this paper, we propose an Adaptive Fine-grained Task Relatedness modeling approach, AdaFTR, for joint CTR-CVR estimation. Our approach is developed based on a parameter-sharing MTL architecture, and introduces a novel adaptive inter-task representation alignment method based on contrastive learning.Given an instance, the inter-task representations of the same instance are considered as positive, while the representations of another random instance are considered as negative. Furthermore, we explicitly model fine-grained task relatedness as the contrast strength (i.e. the temperature coefficient in InfoNCE loss) at the instance level. For this purpose, we build a relatedness prediction network, so that it can predict the contrast strength for inter-task representations of an instance. In this way, we can adaptively set the temperature for contrastive learning in a fine-grained way (i.e. instance level), so as to better capture task relatedness. Both offline evaluation with public e-commerce datasets and online test in a real advertising system at Alibaba have demonstrated the effectiveness of our approach.

cs.IR↗

arXiv2023

Free Lunch for Domain Adversarial Training: Environment Label Smoothing

A fundamental challenge for machine learning models is how to generalize learned models for out-of-distribution (OOD) data. Among various approaches, exploiting invariant features by Domain Adversarial Training (DAT) received widespread attention. Despite its success, we observe training instability from DAT, mostly due to over-confident domain discriminator and environment label noise. To address this issue, we proposed Environment Label Smoothing (ELS), which encourages the discriminator to output soft probability, which thus reduces the confidence of the discriminator and alleviates the impact of noisy environment labels. We demonstrate, both experimentally and theoretically, that ELS can improve training stability, local convergence, and robustness to noisy environment labels. By incorporating ELS with DAT methods, we are able to yield state-of-art results on a wide range of domain generalization/adaptation tasks, particularly when the environment labels are highly noisy.

cs.LG↗

arXiv2023

Sharp endpoint $L_p$ estimates of quantum Schrödinger groups

In this article, we establish sharp endpoint $L_p$ estimates of Schrödinger groups on general measure spaces which may not be equipped with good metrics but admit submarkovian semigroups satisfying purely algebraic assumptions. One of the key ingredients of our proof is to introduce and investigate a new noncommutative high-cancellation BMO space by constructing an abstract form of P-metric codifying some sort of underlying metric and position. This provides the first form of Schrödinger group theory on arbitrary von Neumann algebras and can be applied to many models, including Schrödinger groups associated with non-negative self-adjoint operators satisfying purely Gaussian upper bounds on doubling metric spaces, standard Schrödinger groups on quantum Euclidean spaces, matrix algebras and group von Neumann algebras with finite dimensional cocycles.

math.AP↗

arXiv2023

Breathing cluster in complex neuron-astrocyte networks

Brain activities are featured by spatially distributed neural clusters of coherent firings and a spontaneous switching of the clusters between the synchrony and asynchrony states. Evidences from {\it in vivo} experiments suggest that astrocytes, a type of glial cell regarded previously as providing only structural and metabolic supports to neurons, participate actively in brain functions and play a crucial role in regulating the neural firing activities, yet the mechanism remains unknown. Introducing astrocyte as a reservoir of the glutamate released from neuron synapses, here we propose the model of complex neuron-astrocyte network and employ it to explore the roles of astrocyte in regulating the synchronization behaviors of networked neurons. It is found that a fraction of neurons on the network can be synchronized as a cluster, while the remaining neurons are kept as desynchronized. Moreover, during the course of network evolution, the cluster is switching between the synchrony and asynchrony states intermittently, henceforth the phenomenon of ``breathing cluster". By the method of symmetry-based analysis, we conduct a theoretical investigation on the stability of the cluster and the mechanism generating the breathing activities. It is revealed that the contents of the cluster are determined by the network symmetry and the breathing activities are due to the interplay between the neural network and the astrocyte. The breathing phenomenon is demonstrated in network models of different structures and neural dynamics. The studies give insights into the cellular mechanism of astrocytes in regulating neural activities, and shed lights onto the spontaneous state switching of the neocortex.

q-bio.NC↗

arXiv2023

Numerical Study of Magnetic Island Coalescence Using Magnetohydrodynamics With Adaptively Embedded Particle-In-Cell Model

Collisionless magnetic reconnection typically requires kinetic treatments that are, in general, computationally expensive compared to fluid-based models. In this study, we use the magnetohydrodynamics with adaptively embedded particle-in-cell (MHD-AEPIC) model to study the interaction of two magnetic flux ropes. This innovative model embeds one or more adaptive PIC regions into a global MHD simulation domain such that the kinetic treatment is only applied in regions where kinetic physics is prominent. We compare the simulation results among three cases: 1) MHD with adaptively embedded PIC regions, 2) MHD with statically (or fixed) embedded PIC regions, and 3) a full PIC simulation. The comparison yields good agreement when analyzing their reconnection rates and magnetic island separations, as well as the ion pressure tensor elements and ion agyrotropy. In order to reach a good agreement among the three cases, large adaptive PIC regions are needed within the MHD domain, which indicates that the magnetic island coalescence problem is highly kinetic in nature where the coupling between the macro-scale MHD and micro-scale kinetic physics is important.

physics.plasm-ph↗

arXiv2023

Edge Preserving Implicit Surface Representation of Point Clouds

Learning implicit surface directly from raw data recently has become a very attractive representation method for 3D reconstruction tasks due to its excellent performance. However, as the raw data quality deteriorates, the implicit functions often lead to unsatisfactory reconstruction results. To this end, we propose a novel edge-preserving implicit surface reconstruction method, which mainly consists of a differentiable Laplican regularizer and a dynamic edge sampling strategy. Among them, the differential Laplican regularizer can effectively alleviate the implicit surface unsmoothness caused by the point cloud quality deteriorates; Meanwhile, in order to reduce the excessive smoothing at the edge regions of implicit suface, we proposed a dynamic edge extract strategy for sampling near the sharp edge of point cloud, which can effectively avoid the Laplacian regularizer from smoothing all regions. Finally, we combine them with a simple regularization term for robust implicit surface reconstruction. Compared with the state-of-the-art methods, experimental results show that our method significantly improves the quality of 3D reconstruction results. Moreover, we demonstrate through several experiments that our method can be conveniently and effectively applied to some point cloud analysis tasks, including point cloud edge feature extraction, normal estimation,etc.

cs.CV↗

arXiv2022

In-situ animal behavior classification using knowledge distillation and fixed-point quantization

We explore the use of knowledge distillation (KD) for learning compact and accurate models that enable classification of animal behavior from accelerometry data on wearable devices. To this end, we take a deep and complex convolutional neural network, known as residual neural network (ResNet), as the teacher model. ResNet is specifically designed for multivariate time-series classification. We use ResNet to distill the knowledge of animal behavior classification datasets into soft labels, which consist of the predicted pseudo-probabilities of every class for each datapoint. We then use the soft labels to train our significantly less complex student models, which are based on the gated recurrent unit (GRU) and multilayer perceptron (MLP). The evaluation results using two real-world animal behavior classification datasets show that the classification accuracy of the student GRU-MLP models improves appreciably through KD, approaching that of the teacher ResNet model. To further reduce the computational and memory requirements of performing inference using the student models trained via KD, we utilize dynamic fixed-point quantization (DQ) through an appropriate modification of the computational graph of the considered models. We implement both unquantized and quantized versions of the developed KD-based models on the embedded systems of our purpose-built collar and ear tag devices to classify animal behavior in situ and in real time. Our evaluations corroborate the effectiveness of KD and DQ in improving the accuracy and efficiency of in-situ animal behavior classification.

cs.LG↗

arXiv2022

Learning Domain Invariant Representations for Generalizable Person Re-Identification

Generalizable person Re-Identification (ReID) has attracted growing attention in recent computer vision community. In this work, we construct a structural causal model among identity labels, identity-specific factors (clothes/shoes color etc), and domain-specific factors (background, viewpoints etc). According to the causal analysis, we propose a novel Domain Invariant Representation Learning for generalizable person Re-Identification (DIR-ReID) framework. Specifically, we first propose to disentangle the identity-specific and domain-specific feature spaces, based on which we propose an effective algorithmic implementation for backdoor adjustment, essentially serving as a causal intervention towards the SCM. Extensive experiments have been conducted, showing that DIR-ReID outperforms state-of-the-art methods on large-scale domain generalization ReID benchmarks.

cs.CV↗

arXiv2022

Matrix Profile XXVII: A Novel Distance Measure for Comparing Long Time Series

The most useful data mining primitives are distance measures. With an effective distance measure, it is possible to perform classification, clustering, anomaly detection, segmentation, etc. For single-event time series Euclidean Distance and Dynamic Time Warping distance are known to be extremely effective. However, for time series containing cyclical behaviors, the semantic meaningfulness of such comparisons is less clear. For example, on two separate days the telemetry from an athlete workout routine might be very similar. The second day may change the order in of performing push-ups and squats, adding repetitions of pull-ups, or completely omitting dumbbell curls. Any of these minor changes would defeat existing time series distance measures. Some bag-of-features methods have been proposed to address this problem, but we argue that in many cases, similarity is intimately tied to the shapes of subsequences within these longer time series. In such cases, summative features will lack discrimination ability. In this work we introduce PRCIS, which stands for Pattern Representation Comparison in Series. PRCIS is a distance measure for long time series, which exploits recent progress in our ability to summarize time series with dictionaries. We will demonstrate the utility of our ideas on diverse tasks and datasets.

cs.LG↗

arXiv2022

Quantized Wasserstein Procrustes Alignment of Word Embedding Spaces

Optimal Transport (OT) provides a useful geometric framework to estimate the permutation matrix under unsupervised cross-lingual word embedding (CLWE) models that pose the alignment task as a Wasserstein-Procrustes problem. However, linear programming algorithms and approximate OT solvers via Sinkhorn for computing the permutation matrix come with a significant computational burden since they scale cubically and quadratically, respectively, in the input size. This makes it slow and infeasible to compute OT distances exactly for a larger input size, resulting in a poor approximation quality of the permutation matrix and subsequently a less robust learned transfer function or mapper. This paper proposes an unsupervised projection-based CLWE model called quantized Wasserstein Procrustes (qWP). qWP relies on a quantization step of both the source and target monolingual embedding space to estimate the permutation matrix given a cheap sampling procedure. This approach substantially improves the approximation quality of empirical OT solvers given fixed computational cost. We demonstrate that qWP achieves state-of-the-art results on the Bilingual lexicon Induction (BLI) task.

cs.CL↗