arXiv subjects

Peng Sun

Publications and source records attributed to Peng Sun.

At least 199 records · Page 11Linked to original sources

arXiv2022

Pion and Kaon Distribution Amplitudes from Lattice QCD

We present the state-of-the-art lattice QCD calculation of the pion and kaon light-cone distribution amplitudes (DAs) using large-momentum effective theory. The calculation is done at three lattice spacings $a\approx\{0.06,0.09,0.12\}$ fm and physical pion and kaon masses, with the meson momenta $P_z = \{1.29,1.72,2.15\}$ GeV. The result is non-perturbatively renormalized in a recently proposed hybrid scheme with self renormalization, and extrapolated to the continuum as well as the infinite momentum limit. We find a significant deviation of the pion and kaon DAs from the asymptotic form, and a large $SU(3)$ flavor breaking effect in the kaon DA.

hep-lat↗

arXiv2022

A Simulation Platform for Multi-tenant Machine Learning Services on Thousands of GPUs

Multi-tenant machine learning services have become emerging data-intensive workloads in data centers with heavy usage of GPU resources. Due to the large scale, many tuning parameters and heavy resource usage, it is usually impractical to evaluate and benchmark those machine learning services on real clusters. In this demonstration, we present AnalySIM, a cluster simulator that allows efficient design explorations for multi-tenant machine learning services. Specifically, by trace-driven cluster workload simulation, AnalySIM can easily test and analyze various scheduling policies in a number of performance metrics such as GPU resource utilization. AnalySIM simulates the cluster computational resource based on both physical topology and logical partition. The tool has been used in SenseTime to understand the impact of different scheduling policies with the trace from a real production cluster of over 1000 GPUs. We find that preemption and migration are able to significantly reduce average job completion time and mitigate the resource fragmentation problem.

cs.DC↗

arXiv2021

Efficient Channel Estimation for RIS-Aided MIMO Communications with Unitary Approximate Message Passing

Reconfigurable intelligent surface (RIS) is very promising for wireless networks to achieve high energy efficiency, extended coverage, improved capacity, massive connectivity, etc. To unleash the full potentials of RIS-aided communications, acquiring accurate channel state information is crucial, which however is very challenging. For RIS-aided multiple-input and multiple-output (MIMO) communications, the existing channel estimation methods have computational complexity growing rapidly with the number of RIS units $N$ (e.g., in the order of $N^2$ or $N^3$) and/or have special requirements on the matrices involved (e.g., the matrices need to be sparse for algorithm convergence to achieve satisfactory performance), which hinder their applications. In this work, instead of using the conventional signal model in the literature, we derive a new signal model obtained through proper vectorization and reduction operations. Then, leveraging the unitary approximate message passing (UAMP), we develop a more efficient channel estimator that has complexity linear with $N$ and does not have special requirements on the relevant matrices, thanks to the robustness of UAMP. These facilitate the applications of the proposed algorithm to a general RIS-aided MIMO system with a larger $N$. Moreover, extensive numerical results show that the proposed estimator delivers much better performance and/or requires significantly less number of training symbols, thereby leading to notable reductions in both training overhead and latency.

cs.IT↗

arXiv2021

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language models can further exploit their enormous potential. A unified framework named ERNIE 3.0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters. ERNIE 3.0 outperformed the state-of-the-art models on various NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle platform. Furthermore, we design a self-supervised adversarial loss and a controllable language modeling loss to make ERNIE 3.0 Titan generate credible and controllable texts. To reduce the computation overhead and carbon emission, we propose an online distillation framework for ERNIE 3.0 Titan, where the teacher model will teach students and train itself simultaneously. ERNIE 3.0 Titan is the largest Chinese dense pre-trained model so far. Empirical results show that the ERNIE 3.0 Titan outperforms the state-of-the-art models on 68 NLP datasets.

cs.CL↗

arXiv2021

Near Threshold Heavy Quarkonium Photoproduction at Large Momentum Transfer

Perturbative QCD is applied to investigate the near threshold heavy quarkonium photoproduction at large momentum transfer. We take into account the contributions from the leading three-quark Fock states of the nucleon. The dominant contribution comes from the three-quark Fock state with one unit quark orbital angular momentum (OAM) whereas that from zero quark OAM is suppressed at the threshold. From our analysis, we also show that there is no direct connection between the near threshold heavy quarkonium photoproduction and the gluonic gravitational form factors of the nucleon. Based on the comparison between our result and recent GlueX data of $J/ψ$ photoproduction, we make predictions for $ψ'$ and $Υ$ (1S,2S) states which can be tested in future experiments.

hep-ph↗

arXiv2021

Deep Learning Based OFDM Channel Estimation Using Frequency-Time Division and Attention Mechanism

In this paper, we propose a frequency-time division network (FreqTimeNet) to improve the performance of deep learning (DL) based OFDM channel estimation. This FreqTimeNet is designed based on the orthogonality between the frequency domain and the time domain. In FreqTimeNet, the input is processed by parallel frequency blocks and parallel time blocks sequentially. By introducing the attention mechanism using the SNR information, an attention based FreqTimeNet (AttenFreqTimeNet) is proposed. Using 3rd Generation Partnership Project (3GPP) channel models, the mean square error (MSE) performance of FreqTimeNet and AttenFreqTimeNet under different scenarios is evaluated. A method for constructing mixed training data is proposed, which could address the generalization problem in DL. It is observed that AttenFreqTimeNet outperforms FreqTimeNet, and FreqTimeNet outperforms other DL networks with reasonable complexity.

cs.IT↗

arXiv2021

Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters

Modern GPU datacenters are critical for delivering Deep Learning (DL) models and services in both the research community and industry. When operating a datacenter, optimization of resource scheduling and management can bring significant financial benefits. Achieving this goal requires a deep understanding of the job features and user behaviors. We present a comprehensive study about the characteristics of DL jobs and resource management. First, we perform a large-scale analysis of real-world job traces from SenseTime. We uncover some interesting conclusions from the perspectives of clusters, jobs and users, which can facilitate the cluster system designs. Second, we introduce a general-purpose framework, which manages resources based on historical data. As case studies, we design: a Quasi-Shortest-Service-First scheduling service, which can minimize the cluster-wide average job completion time by up to 6.5x; and a Cluster Energy Saving service, which improves overall cluster utilization by up to 13%.

cs.DC↗

arXiv2021

RI/MOM renormalization of the quasi-PDF in lattice regularization

We analyze the lattice spacing dependence for the pion unpolarized matrix element of a quark bilinear operator with Wilson link (quasi-PDF operator) in the rest frame, using 13 lattice spacings ranging from 0.032 fm to 0.121 fm. We compare results for three different fermion actions with or without good chiral symmetry on dynamical gauge ensembles from three collaborations. This investigation is motivated by the fact that the gauge link generates an $1/a$ divergence, the cancelation of which in many ratios can be numerically tricky. Indeed, our results show that this cancelation deteriorates with decreasing lattice spacing, and that the RI/MOM method leaves a linearly divergent residue for quasi-PDFs. We also show that in the Landau gauge the interaction between the Wilson link and the external state results in a linear divergence which depends on the discretized fermion action.

hep-lat↗

arXiv2021

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities. Despite their success, these large-scale models are trained on plain texts without introducing knowledge such as linguistic knowledge and world knowledge. In addition, most large-scale models are trained in an auto-regressive way. As a result, this kind of traditional fine-tuning approach demonstrates relatively weak performance when solving downstream language understanding tasks. In order to solve the above problems, we propose a unified framework named ERNIE 3.0 for pre-training large-scale knowledge enhanced models. It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning. We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph. Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing the human performance by +0.8% (90.6% vs. 89.8%).

cs.CL↗

arXiv2021

Towards Distraction-Robust Active Visual Tracking

In active visual tracking, it is notoriously difficult when distracting objects appear, as distractors often mislead the tracker by occluding the target or bringing a confusing appearance. To address this issue, we propose a mixed cooperative-competitive multi-agent game, where a target and multiple distractors form a collaborative team to play against a tracker and make it fail to follow. Through learning in our game, diverse distracting behaviors of the distractors naturally emerge, thereby exposing the tracker's weakness, which helps enhance the distraction-robustness of the tracker. For effective learning, we then present a bunch of practical methods, including a reward function for distractors, a cross-modal teacher-student learning strategy, and a recurrent attention mechanism for the tracker. The experimental results show that our tracker performs desired distraction-robust active visual tracking and can be well generalized to unseen environments. We also show that the multi-agent game can be used to adversarially test the robustness of trackers.

cs.CV↗

arXiv2021

A Demonstration of Hadron Mass Origin from QCD Trace Anomaly

Quantum chromodynamics (QCD) claims that the major source of the nucleon invariant mass is not the Higgs mechanism but the trace anomaly in QCD energy momentum tensor. Although experimental and theoretical results support such conclusion, a direct demonstration is still absent. We present the first Lattice QCD calculation of the quark and gluon trace anomaly contributions to the hadron masses, using the overlap fermion on the 2+1 flavor dynamical Domain wall quark ensemble at $m_π=340$ MeV and lattice spacing $a=$0.1105 fm. The result shows that the gluon trace anomaly contributes to most of the nucleon mass, and the contribution in the pion state is smaller than that in others nearly by a factor $\sim$10 since the gluon trace anomaly density inside pion is different from the other hadrons and the magnitude is much smaller. The gluon trace anomaly coefficient $β/g^3=-0.056(6)$ we obtained is consistent with its regularization independent leading order value $(-11+\frac{2N_f}{3})/(4π)^2$ perfectly.

hep-lat↗

arXiv2021

ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems

MLOps is about taking experimental ML models to production, i.e., serving the models to actual users. Unfortunately, existing ML serving systems do not adequately handle the dynamic environments in which online data diverges from offline training data, resulting in tedious model updating and deployment works. This paper implements a lightweight MLOps plugin, termed ModelCI-e (continuous integration and evolution), to address the issue. Specifically, it embraces continual learning (CL) and ML deployment techniques, providing end-to-end supports for model updating and validation without serving engine customization. ModelCI-e includes 1) a model factory that allows CL researchers to prototype and benchmark CL models with ease, 2) a CL backend to automate and orchestrate the model updating efficiently, and 3) a web interface for an ML team to manage CL service collaboratively. Our preliminary results demonstrate the usability of ModelCI-e, and indicate that eliminating the interference between model updating and inference workloads is crucial for higher system efficiency.

cs.DC↗

arXiv2021

Message Passing Based Structured Sparse Signal Recovery for Estimation of OTFS Channels with Fractional Doppler Shifts

The orthogonal time frequency space (OTFS) modulation has emerged as a promising modulation scheme for high mobility wireless communications. To enable efficient OTFS detection in the delay-Doppler (DD) domain, the DD domain channels need to be acquired accurately. To achieve the low latency requirement in future wireless communications, the time duration of the OTFS block should be small, therefore fractional Doppler shifts have to be considered to avoid significant modelling errors due to the assumption of integer Doppler shifts. However, there lack investigations on the estimation of OTFS channels with fractional Doppler shifts in the literature. In this work, we develop a high performing channel estimator for OTFS with the bi-orthogonal waveform or the rectangular waveform. Instead of estimating the DD domain channel directly, we estimate the channel gains and (fractional) Doppler shifts that parameterize the DD domain channel. The estimation is formulated as a structured signal recovery problem with a Bayesian treatment. Based on a factor graph representation of the problem, an efficient message passing algorithm is developed to recover the structured sparse signal (thereby the OTFS channel). The Cramer-Rao Lower Bound (CRLB) for the estimation is developed and the effectiveness of the algorithm is demonstrated through simulations.

cs.IT↗

arXiv2021

Towards Efficient Compressive Data Collection in the Internet of Things

It is of paramount importance to achieve efficient data collection in the Internet of Things (IoT). Due to the inherent structural properties (e.g., sparsity) existing in many signals of interest, compressive sensing (CS) technology has been extensively used for data collection in IoT to improve both accuracy and energy efficiency. Apart from the existing works which leverage CS as a channel coding scheme to deal with data loss during transmission, some recent results have started to employ CS as a source coding strategy. The frequently used projection matrices in these CS-based source coding schemes include dense random matrices (e.g., Gaussian matrices or Bernoulli matrices) and structured matrices (e.g., Toeplitz matrices). However, these matrices are either difficult to be implemented on resource-constrained IoT sensor nodes or have limited applicability. To address these issues, in this paper, we design a novel simple and efficient projection matrix, named sparse Gaussian matrix, which is easy and resource-saving to be implemented in practical IoT applications. We conduct both theoretical analysis and experimental evaluation of the designed sparse Gaussian matrix. The results demonstrate that employing the designed projection matrix to perform CS-based source coding could significantly save time and memory cost while ensuring satisfactory signal recovery performance.

cs.IT↗

arXiv2021

Gluons in charmoniumlike states

The mass components of charmoniumlike states are investigated through the decomposition of QCD energy-momentum tensor (EMT) on lattice. The quark mass contribution $\langle H_m\rangle$ and the momentum fraction $\langle x\rangle$ of valence charm quark and antiquark are calculated for conventional $1S,1P,1D$ charmonia and the exotic $1^{-+}$ charmoniumlike state, based on the $N_f=2+1$ gauge configurations generated by the RBC/UKQCD collaboration. It is found that $\langle H_m\rangle$ is close to each other and around 2.0 to 2.2 GeV for these states, which implies that the mass splittings among these states come almost from the gluon contribution of QCD trace anomaly. The $\langle x\rangle$ of the $1^{-+}$ state is only around 0.55, while that in conventional charmonia is around 0.7 to 0.8. This difference manifests that the proportion of light quarks and gluons in the $1^{-+}$ charmoniumlike state is significantly larger than conventional states.

hep-lat↗

arXiv2021

TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game

StarCraft, one of the most difficult esport games with long-standing history of professional tournaments, has attracted generations of players and fans, and also, intense attentions in artificial intelligence research. Recently, Google's DeepMind announced AlphaStar, a grandmaster level AI in StarCraft II that can play with humans using comparable action space and operations. In this paper, we introduce a new AI agent, named TStarBot-X, that is trained under orders of less computations and can play competitively with expert human players. TStarBot-X takes advantage of important techniques introduced in AlphaStar, and also benefits from substantial innovations including new league training methods, novel multi-agent roles, rule-guided policy search, stabilized policy improvement, lightweight neural network architecture, and importance sampling in imitation learning, etc. We show that with orders of less computation scale, a faithful reimplementation of AlphaStar's methods can not succeed and the proposed techniques are necessary to ensure TStarBot-X's competitive performance. We reveal all technical details that are complementary to those mentioned in AlphaStar, showing the most sensitive parts in league training, reinforcement learning and imitation learning that affect the performance of the agents. Most importantly, this is an open-sourced study that all codes and resources (including the trained model parameters) are publicly accessible via \url{https://github.com/tencent-ailab/tleague_projpage}. We expect this study could be beneficial for both academic and industrial future research in solving complex problems like StarCraft, and also, might provide a sparring partner for all StarCraft II players and other AI agents.

cs.AI↗

arXiv2021

Soft Gluon Resummation in Double Heavy Quarkonium Production at LHC

In this paper, the soft gluon resummation effect in double heavy quarkonium production at the LHC is studied. By applying the transverse momentum dependent factorization formalism, the large logarithms introduced by the small total transverse momentum of heavy quarkonium pair final state system, are resummed to all orders in the expansion of the strong interaction coupling at the Next-to-Leading Logarithm accuracy. We also compare our result with the LHC data. We find that the distribution shape predicted by resummation calculation is consistent with experimental data very well. Since this process is mainly initiated by gluon fusion, it supplies us an important channel to study the gluon parton property in hadrons.

hep-ph↗

arXiv2021

FedCom: A Byzantine-Robust Local Model Aggregation Rule Using Data Commitment for Federated Learning

Federated learning (FL) is a promising privacy-preserving distributed machine learning methodology that allows multiple clients (i.e., workers) to collaboratively train statistical models without disclosing private training data. Due to the characteristics of data remaining localized and the uninspected on-device training process, there may exist Byzantine workers launching data poisoning and model poisoning attacks, which would seriously deteriorate model performance or prevent the model from convergence. Most of the existing Byzantine-robust FL schemes are either ineffective against several advanced poisoning attacks or need to centralize a public validation dataset, which is intractable in FL. Moreover, to the best of our knowledge, none of the existing Byzantine-robust distributed learning methods could well exert its power in Non-Independent and Identically distributed (Non-IID) data among clients. To address these issues, we propose FedCom, a novel Byzantine-robust federated learning framework by incorporating the idea of commitment from cryptography, which could achieve both data poisoning and model poisoning tolerant FL under practical Non-IID data partitions. Specifically, in FedCom, each client is first required to make a commitment to its local training data distribution. Then, we identify poisoned datasets by comparing the Wasserstein distance among commitments submitted by different clients. Furthermore, we distinguish abnormal local model updates from benign ones by testing each local model's behavior on its corresponding data commitment. We conduct an extensive performance evaluation of FedCom. The results demonstrate its effectiveness and superior performance compared to the state-of-the-art Byzantine-robust schemes in defending against typical data poisoning and model poisoning attacks under practical Non-IID data distributions.

cs.LG↗