arXiv subjects

Yu Cheng

Publications and source records attributed to Yu Cheng.

At least 379 records · Page 21Linked to original sources

arXiv2022

Flavor Specific $U(1)_{B_q-L_μ}$ Gauge Model for Muon $g-2$ and $b \to s \bar μμ$ Anomalies

The muon $(g-2)_μ$ and $b\to s \bar μμ$ induced $B$ anomalies as hints of new physics beyond the standard model (SM) have attracted much attention. These two anomalies indicate that there may exist new interaction specifically related to muon. A lot of theoretical ideas have been proposed to explain these anomalies. Gauged flavor specific $U(1)_{B_q-L_μ}$ is among the promising ones. The new gauge boson $Z'$ from $U(1)_{B_q-L_μ}$ interacts with muon and provides necessary ingredient to solve the $(g-2)_μ$ anomaly. The $Z'$-quark coupling can generate flavor changing interactions after diagonalization of quark mass matrix between weak eigen-state and mass eigen-state basis. We revisit challenges for such models attempting to explain the $(g-2)_μ$ and $B$ anomalies separately or simultaneously. We find although for $U(1)_{B_q-L_μ}$ models there is still parameter space to provide solutions for separately explaining the $(g-2)_μ$ and $B$ anomalies, there exists no parameter space for such models to solve both the anomalies simultaneously, after taking into account existing constraints from $τ\to μγ$, $τ\to 3 μ$, neutrino trident and $B_s - \bar B_s$ data. Among them leptonic processes restrict $Z^\prime$ mass to be less than a few hundred MeV if required to solve the $(g-2)_μ$ anomaly, which causes conflict between data from $B_s - \bar B_s$, $D^0 - \bar D^0$ mixing and also hadron decays with $Z^\prime$ in the final states. The effects of $U(1)_Y$ and $U(1)_{B_q-L_μ}$ kinetic mixing on these anomalies are also studied. We find that neither can these effects do much to bring the two anomalies together to be solved simultaneously.

hep-ph↗

arXiv2022

ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest

We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs. ZOOMER is designed for tackling two challenges presented by the massive user data at Taobao: low training/serving efficiency due to the huge scale of the graphs, and low recommendation quality due to the information overload which distracts the recommendation model from specific user intentions. ZOOMER achieves this by introducing a key concept, Region of Interests (ROI) in GNNs for recommendations, i.e., a neighborhood region in the graph with significant relevance to a strong user intention. ZOOMER narrows the focus from the whole graph and "zooms in" on the more relevant ROIs, thereby reducing the training/serving cost and mitigating the information overload at the same time. With carefully designed mechanisms, ZOOMER identifies the interest expressed by each recommendation request, constructs an ROI subgraph by sampling with respect to the interest, and guides the GNN to reweigh different parts of the ROI towards the interest by a multi-level attention module. Deployed as a large-scale distributed system, ZOOMER supports graphs with billions of nodes for training and thousands of requests per second for serving. ZOOMER achieves up to 14x speedup when downsizing sampling scales with comparable (even better) AUC performance than baseline methods. Besides, both the offline evaluation and online A/B test demonstrate the effectiveness of ZOOMER.

cs.IR↗

arXiv2022

A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models

Large pre-trained vision-language (VL) models can learn a new task with a handful of examples and generalize to a new task without fine-tuning. However, these VL models are hard to deploy for real-world applications due to their impractically huge sizes and slow inference speed. To solve this limitation, we study prompt-based low-resource learning of VL tasks with our proposed method, FewVLM, relatively smaller than recent few-shot learners. For FewVLM, we pre-train a sequence-to-sequence transformer model with prefix language modeling (PrefixLM) and masked language modeling (MaskedLM). Furthermore, we analyze the effect of diverse prompts for few-shot tasks. Experimental results on VQA show that FewVLM with prompt-based learning outperforms Frozen which is 31x larger than FewVLM by 18.2% point and achieves comparable results to a 246x larger model, PICa. In our analysis, we observe that (1) prompts significantly affect zero-shot performance but marginally affect few-shot performance, (2) models with noisy prompts learn as quickly as hand-crafted prompts given larger training data, and (3) MaskedLM helps VQA tasks while PrefixLM boosts captioning performance. Our code is publicly available at \url{https://github.com/woojeongjin/FewVLM}

cs.CV↗

arXiv2022

The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy

Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolutional networks. However, it is questionable whether such potential has been fully unleashed in practice, as the learned ViTs often suffer from over-smoothening, yielding likely redundant models. Recent works made preliminary attempts to identify and alleviate such redundancy, e.g., via regularizing embedding similarity or re-injecting convolution-like structures. However, a "head-to-toe assessment" regarding the extent of redundancy in ViTs, and how much we could gain by thoroughly mitigating such, has been absent for this field. This paper, for the first time, systematically studies the ubiquitous existence of redundancy at all three levels: patch embedding, attention map, and weight space. In view of them, we advocate a principle of diversity for training ViTs, by presenting corresponding regularizers that encourage the representation diversity and coverage at each of those levels, that enabling capturing more discriminative information. Extensive experiments on ImageNet with a number of ViT backbones validate the effectiveness of our proposals, largely eliminating the observed ViT redundancy and significantly boosting the model generalization. For example, our diversified DeiT obtains 0.70%~1.76% accuracy boosts on ImageNet with highly reduced similarity. Our codes are fully available in https://github.com/VITA-Group/Diverse-ViT.

cs.LG↗

arXiv2022

A SBP-SAT FDTD Subgridding Method Using Staggered Yee's Grids Without Modifying Field Components

A summation-by-parts simultaneous approximation term (SBP-SAT) finite-difference time-domain (FDTD) subgridding method is proposed to model geometrically fine structures in this paper. Compared with our previous work, the proposed SBP-SAT FDTD method uses the staggered Yee's grid without adding or modifying any field components through field extrapolation on the boundaries to make the discrete operators satisfy the SBP property. The accuracy of extrapolation keeps consistency with that of the second-order finite-difference scheme near the boundaries. In addition, the SATs are used to weakly enforce the tangential boundary conditions between multiple mesh blocks with different mesh sizes. With carefully designed interpolation matrices and selected free parameters of the SATs, no dissipation occurs in the whole computational domain. Therefore, its long-time stability is theoretically guaranteed. Three numerical examples are carried out to validate its effectiveness. Results show that the proposed SBP-SAT FDTD subgridding method is stable, accurate, efficient, and easy to implement based on existing FDTD codes with only a few modifications.

cs.CE↗

arXiv2022

A Deep Reinforcement Learning based Approach for NOMA-based Random Access Network with Truncated Channel Inversion Power Control

As a main use case of 5G and Beyond wireless network, the ever-increasing machine type communications (MTC) devices pose critical challenges over MTC network in recent years. It is imperative to support massive MTC devices with limited resources. To this end, Non-orthogonal multiple access (NOMA) based random access network has been deemed as a prospective candidate for MTC network. In this paper, we propose a deep reinforcement learning (RL) based approach for NOMA-based random access network with truncated channel inversion power control. Specifically, each MTC device randomly selects a pre-defined power level with a certain probability for data transmission. Devices are using channel inversion power control yet subject to the upper bound of the transmission power. Due to the stochastic feature of the channel fading and the limited transmission power, devices with different achievable power levels have been categorized as different types of devices. In order to achieve high throughput with considering the fairness between all devices, two objective functions are formulated. One is to maximize the minimum long-term expected throughput of all MTC devices, the other is to maximize the geometric mean of the long-term expected throughput for all MTC devices. A Policy based deep reinforcement learning approach is further applied to tune the transmission probabilities of each device to solve the formulated optimization problems. Extensive simulations are conducted to show the merits of our proposed approach.

cs.NI↗

arXiv2022

Widening the $U(1)_{L_μ-L_τ}$ $Z^\prime$ mass range for resolving the muon $g-2$ anomaly

Exchanging a $Z^\prime$ gauge boson is a favored mechanism to solve the muon $(g-2)_μ$ anomaly. Among such models the $Z^\prime$ from $U(1)_{L_μ- L_τ}$ gauge group has been extensively studied. In this model the same interaction addressing $(g-2)_μ$, leads to an enhanced muon neutrino trident (MNT) process $ν_μN \to ν_μμ\bar μN$ constraining the $Z^\prime$ mass to be less than a few hundred MeV. Many other $Z^\prime$ models face the same problem. It has long been realized that the coupling of $Z^\prime$ in the model can admit $(\bar μγ^μτ+ \bar ν_μγ^μL ν_τ)Z^\prime_μ$ interaction which does not contribute to the MNT process. It can solve $(g-2)_μ$ anomaly for a much wider $Z^\prime$ mass range. However this new interaction induces $τ\to μ\barν_μν_τ$ which rules out it as a solution to $(g-2)_μ$ anomaly. Here we propose a mechanism by introducing type-II seesaw $SU(2)_L$ triplet scalars to evade constraints from all known data to allow a wide $Z^\prime$ mass range to solve the $(g-2)_μ$ anomaly. This mechanism opens a new window for $Z^\prime$ physics.

hep-ph↗

arXiv2022

Unsupervised Temporal Video Grounding with Deep Semantic Clustering

Temporal video grounding (TVG) aims to localize a target segment in a video according to a given sentence query. Though respectable works have made decent achievements in this task, they severely rely on abundant video-query paired data, which is expensive and time-consuming to collect in real-world scenarios. In this paper, we explore whether a video grounding model can be learned without any paired annotations. To the best of our knowledge, this paper is the first work trying to address TVG in an unsupervised setting. Considering there is no paired supervision, we propose a novel Deep Semantic Clustering Network (DSCNet) to leverage all semantic information from the whole query set to compose the possible activity in each video for grounding. Specifically, we first develop a language semantic mining module, which extracts implicit semantic features from the whole query set. Then, these language semantic features serve as the guidance to compose the activity in video via a video-based semantic aggregation module. Finally, we utilize a foreground attention branch to filter out the redundant background activities and refine the grounding results. To validate the effectiveness of our DSCNet, we conduct experiments on both ActivityNet Captions and Charades-STA datasets. The results demonstrate that DSCNet achieves competitive performance, and even outperforms most weakly-supervised approaches.

cs.CV↗

arXiv2022

Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

Large-scale pre-trained language models have achieved tremendous success across a wide range of natural language understanding (NLU) tasks, even surpassing human performance. However, recent studies reveal that the robustness of these models can be challenged by carefully crafted textual adversarial examples. While several individual datasets have been proposed to evaluate model robustness, a principled and comprehensive benchmark is still missing. In this paper, we present Adversarial GLUE (AdvGLUE), a new multi-task benchmark to quantitatively and thoroughly explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks. In particular, we systematically apply 14 textual adversarial attack methods to GLUE tasks to construct AdvGLUE, which is further validated by humans for reliable annotations. Our findings are summarized as follows. (i) Most existing adversarial attack algorithms are prone to generating invalid or ambiguous adversarial examples, with around 90% of them either changing the original semantic meanings or misleading human annotators as well. Therefore, we perform a careful filtering process to curate a high-quality benchmark. (ii) All the language models and robust training methods we tested perform poorly on AdvGLUE, with scores lagging far behind the benign accuracy. We hope our work will motivate the development of new adversarial attacks that are more stealthy and semantic-preserving, as well as new robust language models against sophisticated adversarial attacks. AdvGLUE is available at https://adversarialglue.github.io.

cs.CL↗

arXiv2022

Memory-Guided Semantic Learning Network for Temporal Sentence Grounding

Temporal sentence grounding (TSG) is crucial and fundamental for video understanding. Although the existing methods train well-designed deep networks with a large amount of data, we find that they can easily forget the rarely appeared cases in the training stage due to the off-balance data distribution, which influences the model generalization and leads to undesirable performance. To tackle this issue, we propose a memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG tasks. Specifically, MGSL-Net consists of three main parts: a cross-modal inter-action module, a memory augmentation module, and a heterogeneous attention module. We first align the given video-query pair by a cross-modal graph convolutional network, and then utilize a memory module to record the cross-modal shared semantic features in the domain-specific persistent memory. During training, the memory slots are dynamically associated with both common and rare cases, alleviating the forgetting issue. In testing, the rare cases can thus be enhanced by retrieving the stored memories, resulting in better generalization. At last, the heterogeneous attention module is utilized to integrate the enhanced multi-modal features in both video and query domains. Experimental results on three benchmarks show the superiority of our method on both effectiveness and efficiency, which substantially improves the accuracy not only on the entire dataset but also on rare cases.

cs.CV↗

arXiv2021

Efficient Robust Training via Backward Smoothing

Adversarial training is so far the most effective strategy in defending against adversarial examples. However, it suffers from high computational costs due to the iterative adversarial attacks in each training step. Recent studies show that it is possible to achieve fast Adversarial Training by performing a single-step attack with random initialization. However, such an approach still lags behind state-of-the-art adversarial training algorithms on both stability and model robustness. In this work, we develop a new understanding towards Fast Adversarial Training, by viewing random initialization as performing randomized smoothing for better optimization of the inner maximization problem. Following this new perspective, we also propose a new initialization strategy, backward smoothing, to further improve the stability and model robustness over single-step robust training methods. Experiments on multiple benchmarks demonstrate that our method achieves similar model robustness as the original TRADES method while using much less training time ($\sim$3x improvement with the same training schedule).

cs.LG↗

arXiv2021

Surveys of Clumps, Cores, and Condensations in the Cygnus X: II. Radio Properties of the Massive Dense Cores

We have carried out a high-sensitivity and high-resolution radio continuum study towards a sample of 47 massive dense cores (MDCs) in the Cygnus X star-forming complex using the Karl G. Jansky Very Large Array, aiming to detect and characterize the radio emission associated with star-forming activities down to ~0.01 pc scales. We have detected 64 radio sources within or closely around the full width at half-maximum (FWHM) of the MDCs, of which 37 are reported for the first time. The majority of the detected radio sources are associated with dust condensations embedded within the MDCs, and they are mostly weak and compact. We are able to build spectral energy distributions for 8 sources. Two of them indicate non-thermal emission and the other six indicate thermal free-free emission. We have determined that most of the radio sources are ionized jets or winds originating from massive young stellar objects, whereas only a few sources are likely to be ultra-compact HII regions. Further quantitative analyses indicate that the radio luminosity of the detected radio sources increases along the evolution path of the MDCs.

astro-ph.GA↗

arXiv2021

Playing Lottery Tickets with Vision and Language

Large-scale pre-training has recently revolutionized vision-and-language (VL) research. Models such as LXMERT and UNITER have significantly lifted the state of the art over a wide range of VL tasks. However, the large number of parameters in such models hinders their application in practice. In parallel, work on the lottery ticket hypothesis (LTH) has shown that deep neural networks contain small matching subnetworks that can achieve on par or even better performance than the dense networks when trained in isolation. In this work, we perform the first empirical study to assess whether such trainable subnetworks also exist in pre-trained VL models. We use UNITER as the main testbed (also test on LXMERT and ViLT), and consolidate 7 representative VL tasks for experiments, including visual question answering, visual commonsense reasoning, visual entailment, referring expression comprehension, image-text retrieval, GQA, and NLVR$^2$. Through comprehensive analysis, we summarize our main findings as follows. ($i$) It is difficult to find subnetworks that strictly match the performance of the full model. However, we can find "relaxed" winning tickets at 50%-70% sparsity that maintain 99% of the full accuracy. ($ii$) Subnetworks found by task-specific pruning transfer reasonably well to the other tasks, while those found on the pre-training tasks at 60%/70% sparsity transfer universally, matching 98%/96% of the full accuracy on average over all the tasks. ($iii$) Besides UNITER, other models such as LXMERT and ViLT can also play lottery tickets. However, the highest sparsity we can achieve for ViLT is far lower than LXMERT and UNITER (30% vs. 70%). ($iv$) LTH also remains relevant when using other training methods (e.g., adversarial training).

cs.CV↗

arXiv2021

The Elastic Lottery Ticket Hypothesis

Lottery Ticket Hypothesis (LTH) raises keen attention to identifying sparse trainable subnetworks, or winning tickets, which can be trained in isolation to achieve similar or even better performance compared to the full models. Despite many efforts being made, the most effective method to identify such winning tickets is still Iterative Magnitude-based Pruning (IMP), which is computationally expensive and has to be run thoroughly for every different network. A natural question that comes in is: can we "transform" the winning ticket found in one network to another with a different architecture, yielding a winning ticket for the latter at the beginning, without re-doing the expensive IMP? Answering this question is not only practically relevant for efficient "once-for-all" winning ticket finding, but also theoretically appealing for uncovering inherently scalable sparse patterns in networks. We conduct extensive experiments on CIFAR-10 and ImageNet, and propose a variety of strategies to tweak the winning tickets found from different networks of the same model family (e.g., ResNets). Based on these results, we articulate the Elastic Lottery Ticket Hypothesis (E-LTH): by mindfully replicating (or dropping) and re-ordering layers for one network, its corresponding winning ticket could be stretched (or squeezed) into a subnetwork for another deeper (or shallower) network from the same family, whose performance is nearly the same competitive as the latter's winning ticket directly found by IMP. We have also extensively compared E-LTH with pruning-at-initialization and dynamic sparse training methods, as well as discussed the generalizability of E-LTH to different model families, layer types, and across datasets. Code is available at https://github.com/VITA-Group/ElasticLTH.

cs.CV↗

arXiv2021

Collider search of light dark matter model with dark sector decay

We explore the possibility that the dark matter relic density is not produced by thermal mechanism directly, but by the decay of other heavier dark sector particles which on the other hand can be produced by the thermal freeze-out mechanism. Using a concrete model with a light dark matter from dark sector decay, we study the collider signature of the dark sector particles in association with Higgs production processes. We find that the future lepton colliders can be a better place to probe the signature of this kind of light dark matter model than the hadron collider such as LHC. Meanwhile, it is found that a Higgs factory with center of mass energy 250 GeV has a better potential to resolve the signature of this kind of light dark matter model than the Higgs factory with center of mass energy 350 GeV.

hep-ph↗

arXiv2021

Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective

Training generative adversarial networks (GANs) with limited real image data generally results in deteriorated performance and collapsed models. To conquer this challenge, we are inspired by the latest observation, that one can discover independently trainable and highly sparse subnetworks (a.k.a., lottery tickets) from GANs. Treating this as an inductive prior, we suggest a brand-new angle towards data-efficient GAN training: by first identifying the lottery ticket from the original GAN using the small training set of real images; and then focusing on training that sparse subnetwork by re-using the same set. We find our coordinated framework to offer orthogonal gains to existing real image data augmentation methods, and we additionally present a new feature-level augmentation that can be applied together with them. Comprehensive experiments endorse the effectiveness of our proposed framework, across various GAN architectures (SNGAN, BigGAN, and StyleGAN-V2) and diverse datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet, ImageNet, and multiple few-shot generation datasets). Codes are available at: https://github.com/VITA-Group/Ultra-Data-Efficient-GAN-Training.

cs.LG↗

arXiv2021

Chasing Sparsity in Vision Transformers: An End-to-End Exploration

Vision transformers (ViTs) have recently received explosive popularity, but their enormous model sizes and training costs remain daunting. Conventional post-training pruning often incurs higher training budgets. In contrast, this paper aims to trim down both the training memory overhead and the inference complexity, without sacrificing the achievable accuracy. We carry out the first-of-its-kind comprehensive exploration, on taking a unified approach of integrating sparsity in ViTs "from end to end". Specifically, instead of training full ViTs, we dynamically extract and train sparse subnetworks, while sticking to a fixed small parameter budget. Our approach jointly optimizes model parameters and explores connectivity throughout training, ending up with one sparse network as the final output. The approach is seamlessly extended from unstructured to structured sparsity, the latter by considering to guide the prune-and-grow of self-attention heads inside ViTs. We further co-explore data and architecture sparsity for additional efficiency gains by plugging in a novel learnable token selector to adaptively determine the currently most vital patches. Extensive results on ImageNet with diverse ViT backbones validate the effectiveness of our proposals which obtain significantly reduced computational cost and almost unimpaired generalization. Perhaps most surprisingly, we find that the proposed sparse (co-)training can sometimes improve the ViT accuracy rather than compromising it, making sparsity a tantalizing "free lunch". For example, our sparsified DeiT-Small at (5%, 50%) sparsity for (data, architecture), improves 0.28% top-1 accuracy, and meanwhile enjoys 49.32% FLOPs and 4.40% running time savings. Our codes are available at https://github.com/VITA-Group/SViTE.

cs.CV↗

arXiv2021

A Stable FDTD Subgridding Scheme with SBP-SAT for Transient Electromagnetic Analysis

We proposed a provably stable FDTD subgridding method for accurate and efficient transient electromagnetic analysis. In the proposed method, several field components are properly added to the boundaries of Yee's grid to make sure that the discrete operators meet the summation-by-parts (SBP) property. Then, by incorporating the simultaneous approximation terms (SATs) into the finite-difference time-domain (FDTD) method, the proposed FDTD subgridding method mimics the energy estimate of the continuous Maxwell's equations at the semi-discrete level to guarantee its stability. Further, to couple multiple mesh blocks with different mesh sizes, the interpolation matrices are also derived. The proposed FDTD subgridding method is accurate, efficient, easy to implement and be integrated into the existing FDTD codes with only simple modifications. At last, three numerical examples with fine structures are carried out to validate the effectiveness of the proposed method.

cs.CE↗