arXiv subjects

Tao Yang

Publications and source records attributed to Tao Yang.

At least 253 records · Page 14Linked to original sources

arXiv2022

Space-borne atom interferometric gravitational wave detections. Part III. Eccentricity on dark sirens

Eccentricity of the inspiraling compact binaries can greatly improve the distance inference and source localization of dark sirens. In this paper, we continue the research for the space-borne atom interferometric gravitational-wave detector AEDGE and investigate the effects of eccentricity on the dark sirens observed by AEDGE in the mid-band. We simulate five types of typical compact binaries with component mass ranging from $1-100~M_{\odot}$. The largest improvement for both distance inference and localization can be as much as 1.5--3 orders of magnitude. We then construct the catalogs of dark sirens observed by AEDGE in five years. We find eccentricity is crucial to the detection of golden binary black holes (BBH) whose host galaxy can be uniquely identified. With only 5--10 golden dark BBHs one can obtain a 2 percent precision measurement of $H_0$ which is sufficient to arbitrate the Hubble tension. Regardless of eccentricity, AEDGE can also observe tens of golden binary neutron stars (BNS) and neutron star--black hole binaries (NSBH) with unique host galaxies. These golden dark sirens can serve as early warnings for the follow-up observations of gravitational waves in the high frequency band as well as the search of their electromagnetic counterparts. Our results show eccentricity is a crucial factor in the detection, data analysis, and application of GWs with the atom interferometers in the mid-band.

gr-qc↗

arXiv2022

Visual Concepts Tokenization

Obtaining the human-like perception ability of abstracting visual concepts from concrete pixels has always been a fundamental and important target in machine learning research fields such as disentangled representation learning and scene decomposition. Towards this goal, we propose an unsupervised transformer-based Visual Concepts Tokenization framework, dubbed VCT, to perceive an image into a set of disentangled visual concept tokens, with each concept token responding to one type of independent visual concept. Particularly, to obtain these concept tokens, we only use cross-attention to extract visual information from the image tokens layer by layer without self-attention between concept tokens, preventing information leakage across concept tokens. We further propose a Concept Disentangling Loss to facilitate that different concept tokens represent independent visual concepts. The cross-attention and disentangling loss play the role of induction and mutual exclusion for the concept tokens, respectively. Extensive experiments on several popular datasets verify the effectiveness of VCT on the tasks of disentangled representation learning and scene decomposition. VCT achieves the state of the art results by a large margin.

cs.CV↗

arXiv2022

AD-DROP: Attribution-Driven Dropout for Robust Language Model Fine-Tuning

Fine-tuning large pre-trained language models on downstream tasks is apt to suffer from overfitting when limited training data is available. While dropout proves to be an effective antidote by randomly dropping a proportion of units, existing research has not examined its effect on the self-attention mechanism. In this paper, we investigate this problem through self-attention attribution and find that dropping attention positions with low attribution scores can accelerate training and increase the risk of overfitting. Motivated by this observation, we propose Attribution-Driven Dropout (AD-DROP), which randomly discards some high-attribution positions to encourage the model to make predictions by relying more on low-attribution positions to reduce overfitting. We also develop a cross-tuning strategy to alternate fine-tuning and AD-DROP to avoid dropping high-attribution positions excessively. Extensive experiments on various benchmarks show that AD-DROP yields consistent improvements over baselines. Analysis further confirms that AD-DROP serves as a strategic regularizer to prevent overfitting during fine-tuning.

cs.CL↗

arXiv2022

Communication Compression for Distributed Nonconvex Optimization

This paper considers distributed nonconvex optimization with the cost functions being distributed over agents. Noting that information compression is a key tool to reduce the heavy communication load for distributed algorithms as agents iteratively communicate with neighbors, we propose three distributed primal--dual algorithms with compressed communication. The first two algorithms are applicable to a general class of compressors with bounded relative compression error and the third algorithm is suitable for two general classes of compressors with bounded absolute compression error. We show that the proposed distributed algorithms with compressed communication have comparable convergence properties as state-of-the-art algorithms with exact communication. Specifically, we show that they can find first-order stationary points with sublinear convergence rate $\mathcal{O}(1/T)$ when each local cost function is smooth, where $T$ is the total number of iterations, and find global optima with linear convergence rate under an additional condition that the global cost function satisfies the Polyak--Łojasiewicz condition. Numerical simulations are provided to illustrate the effectiveness of the theoretical results.

math.OC↗

arXiv2022

Doubly-Irregular Repeat-Accumulate Codes over Integer Rings for Multi-user Communications

Structured codes based on lattices were shown to provide enlarged capacity for multi-user communication networks. In this paper, we study capacity-approaching irregular repeat accumulate (IRA) codes over integer rings $\mathbb{Z}_{2^{m}}$ for $2^m$-PAM signaling, $m=1,2,\cdots$. Such codes feature the property that the integer sum of $K$ codewords belongs to the extended codebook (or lattice) w.r.t. the base code. With it, \emph{% structured binning} can be utilized and the gains promised in lattice based network information theory can be materialized in practice. In designing IRA ring codes, we first analyze the effect of zero-divisors of integer ring on the iterative belief-propagation (BP) decoding, and show the invalidity of symmetric Gaussian approximation. Then we propose a doubly IRA (D-IRA) ring code structure, consisting of \emph{irregular multiplier distribution} and \emph{irregular node-degree distribution}, that can restore the symmetry and optimize the BP decoding threshold. For point-to-point AWGN channel with $% 2^m $-PAM inputs, D-IRA ring codes perform as low as 0.29 dB to the capacity limits, outperforming existing bit-interleaved coded-modulation (BICM) and IRA modulation codes over GF($2^m$). We then proceed to design D-IRA ring codes for two important multi-user communication setups, namely compute-forward (CF) and dirty paper coding (DPC), with $2^m$-PAM signaling. With it, a physical-layer network coding scheme yields a gap to the CF limit by 0.24 dB, and a simple linear DPC scheme exhibits a gap to the capacity by 0.91 dB.

cs.IT↗

arXiv2022

Leakage current simulations of Low Gain Avalanche Diode with improved Radiation Damage Modeling

We report precise TCAD simulations of IHEP-IME-v1 Low Gain Avalanche Diode (LGAD) calibrated by secondary ion mass spectroscopy (SIMS). Our setup allows us to evaluate the leakage current, capacitance, and breakdown voltage of LGAD, which agree with measurements' results before irradiation. And we propose an improved LGAD Radiation Damage Model (LRDM) which combines local acceptor removal with global deep energy levels. The LRDM is applied to the IHEP-IME-v1 LGAD and able to predict the leakage current well at -30 $^{\circ}$C after an irradiation fluence of $ Φ_{eq}=2.5 \times 10^{15} ~n_{eq}/cm^{2}$. The charge collection efficiency (CCE) is under development.

physics.ins-det↗

arXiv2022

Flickering Buoyant Diffusion Flames in Weakly Rotatory Flows

Flickering buoyant diffusion methane flames in weakly rotatory flows were computationally and theoretically investigated. The prominent computational finding is that the flicker frequency nonlinearly increases with the rotational intensity number R (up to 0.24), which measures the relative importance of the rotational speed compared with the methane jet speed. This finding is consistent with the previous experimental observations that flame flicker is enhanced by rotatory flows within a certain extent. Based on the vortex-dynamical understanding of flickering flames that the flame flicker is caused by the periodic shedding of buoyancy-induced toroidal vortices, we formulated a scaling theory for flickering buoyant diffusion flames in weakly rotatory flows. The theory predicts that, with respect to the flicker frequency f_0 at R=0, the increase of the flicker frequency f at nonzero R obeys the scaling relation f-f_0~R^2, which agrees very well with the present computational results. In physics, the externally rotatory flow enhances the radial pressure gradient around the flame, and the significant baroclinic effect contributes an additional source for the growth of toroidal vortices so that their periodic shedding is faster.

physics.flu-dyn↗

arXiv2022

High-Order Leader-Follower Tracking Control under Limited Information Availability

Limited information availability represents a fundamental challenge for control of multi-agent systems, since an agent often lacks sensing capabilities to measure certain states of its own and can exchange data only with its neighbors. The challenge becomes even greater when agents are governed by high-order dynamics. The present work is motivated to conduct control design for linear and nonlinear high-order leader-follower multi-agent systems in a context where only the first state of an agent is measured. To address this open challenge, we develop novel distributed observers to enable followers to reconstruct unmeasured or unknown quantities about themselves and the leader and on such a basis, build observer-based tracking control approaches. We analyze the convergence properties of the proposed approaches and validate their performance through simulation.

eess.SY↗

arXiv2022

Simulation of the 4H-SiC Low Gain Avalanche Diode

Silicon Carbide device (4H-SiC) has potential radiation hardness, high saturated carrier velocity and low temperature sensitivity theoretically. The Silicon Low Gain Avalanche Diode (LGAD) has been verified to have excellent time performance. Therefore, the 4H-SiC LGAD is introduced in this work for application to detect the Minimum Ionization Particles (MIPs). We provide guidance to determine the thickness and doping level of the gain layer after an analytical analysis. The gain layer thickness $d_{gain}=0.5~μm$ is adopted in our design. We design two different types of 4H-SiC LGAD which have two types electric field, and the corresponding leakage current, capacitance and gain are simulated by TCAD tools. Through analysis of the simulation results, the advantages and disadvantages are discussed for two types of 4H-SiC LGAD.

physics.ins-det↗

arXiv2022

A Flexible Diffusion Model

Diffusion (score-based) generative models have been widely used for modeling various types of complex data, including images, audios, and point clouds. Recently, the deep connection between forward-backward stochastic differential equations (SDEs) and diffusion-based models has been revealed, and several new variants of SDEs are proposed (e.g., sub-VP, critically-damped Langevin) along this line. Despite the empirical success of the hand-crafted fixed forward SDEs, a great quantity of proper forward SDEs remain unexplored. In this work, we propose a general framework for parameterizing the diffusion model, especially the spatial part of the forward SDE. An abstract formalism is introduced with theoretical guarantees, and its connection with previous diffusion models is leveraged. We demonstrate the theoretical advantage of our method from an optimization perspective. Numerical experiments on synthetic datasets, MINIST and CIFAR10 are also presented to validate the effectiveness of our framework.

cs.LG↗

arXiv2022

Learning to Rank Rationales for Explainable Recommendation

State-of-the-art recommender system (RS) mostly rely on complex deep neural network (DNN) model structure, which makes it difficult to provide explanations along with RS decisions. Previous researchers have proved that providing explanations along with recommended items can help users make informed decisions and improve their trust towards the uninterpretable blackbox system. In model-agnostic explainable recommendation, system designers deploy a separate explanation model to take as input from the decision model, and generate explanations to meet the goal of persuasiveness. In this work, we explore the task of ranking textual rationales (supporting evidences) for model-agnostic explainable recommendation. Most of existing rationales ranking algorithms only utilize the rationale IDs and interaction matrices to build latent factor representations; and the semantic information within the textual rationales are not learned effectively. We argue that such design is suboptimal as the important semantic information within the textual rationales may be used to better profile user preferences and item features. Seeing this gap, we propose a model named Semantic-Enhanced Bayesian Personalized Explanation Ranking (SE-BPER) to effectively combine the interaction information and semantic information. SE-BPER first initializes the latent factor representations with contextualized embeddings generated by transformer model, then optimizes them with the interaction data. Extensive experiments show that such methodology improves the rationales ranking performance while simplifying the model training process (fewer hyperparameters and faster convergence). We conclude that the optimal way to combine semantic and interaction information remains an open question in the task of rationales ranking.

cs.IR↗

arXiv2022

Design and testing of LGAD sensor with shallow carbon implantation

The low gain avalanche detectors (LGADs) are thin sensors with fast charge collection which in combination with internal gain deliver an outstanding time resolution of about 30 ps. High collision rates and consequent large particle rates crossing the detectors at the upgraded Large Hadron Collider (LHC) in 2028 will lead to radiation damage and deteriorated performance of the LGADs. The main consequence of radiation damage is loss of gain layer doping (acceptor removal) which requires an increase of bias voltage to compensate for the loss of charge collection efficiency and consequently time resolution. The Institute of High Energy Physics (IHEP), Chinese Academy of Sciences (CAS) has developed a process based on the Institute of Microelectronics (IME), CAS capability to enrich the gain layer with carbon to reduce the acceptor removal effect by radiation. After 1 MeV neutron equivalent fluence of 2.5$\times$10$^{15}$ n$_{eq}$/cm$^{2}$, which is the maximum fluence to which sensors will be exposed at ATLAS High Granularity Timing Detector (HGTD), the IHEP-IME second version (IHEP-IMEv2) 50 $μ$m LGAD sensors already deliver adequate charge collection > 4 fC and time resolution < 50 ps at voltages < 400 V. The operation voltages of these 50 $μ$m devices are well below those at which single event burnout may occur.

physics.ins-det↗

arXiv2022

Quench dynamics of Bose-Einstein condensates in boxlike traps

We investigate the nonequilibrium dynamics of two-dimensional Bose-Einstein condensates in boxlike traps with power-law potential boundaries by quenching the interatomic interactions. For both concave and convex potentials, we show that ring dark solitons can be excited during the quench dynamics. The modulation strength of the quench and the steepness of the boundary are two main factors affecting the evolution of the system. Five dynamic regimes are identified concerning the number of ring dark solitons excited in the condensate. For the situation without ring dark soliton excitations, the condensate undergoes damped radius oscillation. As far as the appearance of ring dark solitons, interesting structures arise from their decay. For the concave potential, the excitation patterns show a nested structure of vortex-antivortex pairs. For the convex potential, on the other hand, the dynamic excitation patterns display richer structures that have multiple transport behaviors.

nlin.PS↗

arXiv2022

Test-time Batch Normalization

Deep neural networks often suffer the data distribution shift between training and testing, and the batch statistics are observed to reflect the shift. In this paper, targeting of alleviating distribution shift in test time, we revisit the batch normalization (BN) in the training process and reveals two key insights benefiting test-time optimization: $(i)$ preserving the same gradient backpropagation form as training, and $(ii)$ using dataset-level statistics for robust optimization and inference. Based on the two insights, we propose a novel test-time BN layer design, GpreBN, which is optimized during testing by minimizing Entropy loss. We verify the effectiveness of our method on two typical settings with distribution shift, i.e., domain generalization and robustness tasks. Our GpreBN significantly improves the test-time performance and achieves the state of the art results.

cs.LG↗

arXiv2022

NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results

This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29.00dB on DIV2K validation set. IMDN is set as the baseline for efficiency measurement. The challenge had 3 tracks including the main track (runtime), sub-track one (model complexity), and sub-track two (overall performance). In the main track, the practical runtime performance of the submissions was evaluated. The rank of the teams were determined directly by the absolute value of the average runtime on the validation set and test set. In sub-track one, the number of parameters and FLOPs were considered. And the individual rankings of the two metrics were summed up to determine a final ranking in this track. In sub-track two, all of the five metrics mentioned in the description of the challenge including runtime, parameter count, FLOPs, activations, and memory consumption were considered. Similar to sub-track one, the rankings of five metrics were summed up to determine a final ranking. The challenge had 303 registered participants, and 43 teams made valid submissions. They gauge the state-of-the-art in efficient single image super-resolution.

cs.CV↗

arXiv2022

Dual Skipping Guidance for Document Retrieval with Learned Sparse Representations

This paper proposes a dual skipping guidance scheme with hybrid scoring to accelerate document retrieval that uses learned sparse representations while still delivering a good relevance. This scheme uses both lexical BM25 and learned neural term weights to bound and compose the rank score of a candidate document separately for skipping and final ranking, and maintains two top-k thresholds during inverted index traversal. This paper evaluates time efficiency and ranking relevance of the proposed scheme in searching MS MARCO TREC datasets.

cs.IR↗

arXiv2022

Time Resolution of the 4H-SiC PIN Detector

We address the determination of the time resolution for the $\rm 100~μm$ 4H-SiC PIN detectors fabricated by Nanjing University (NJU). The time response to $\rm β$ particles from a $\rm ^{90}$Sr source is investigated for the detection of the minimum ionizing particles (MIPs). We study the influence of different reverse voltages, which correspond to different carrier velocities and device sizes, and how this correlates with the detector capacitance. We determine a time resolution $\rm (94\pm1)~ps$ for $\rm 100~μm$ 4H-SiC PIN detector. A fast simulation software, termed RASER (RAdiation SEmiconductoR), is developed, and validated by comparing the waveform obtained from simulated and measured data. The simulated time resolution is $\rm (73\pm 1)~ps$ after considering the intrinsic leading contributions of the detector to time resolution.

physics.ins-det↗

arXiv2022

Compact Token Representations with Contextual Quantization for Efficient Document Re-ranking

Transformer based re-ranking models can achieve high search relevance through context-aware soft matching of query tokens with document tokens. To alleviate runtime complexity of such inference, previous work has adopted a late interaction architecture with pre-computed contextual token representations at the cost of a large online storage. This paper proposes contextual quantization of token embeddings by decoupling document-specific and document-independent ranking contributions during codebook-based compression. This allows effective online decompression and embedding composition for better search relevance. This paper presents an evaluation of the above compact token representation model in terms of relevance and space efficiency.

cs.IR↗