arXiv subjects

Yue Zhao

Publications and source records attributed to Yue Zhao.

At least 343 records · Page 19Linked to original sources

arXiv2022

Probing Early Universe Supercooled Phase Transitions with Gravitational Wave Data

We investigate the reach of the LIGO/Virgo/KAGRA detectors in the search for signatures of first-order phase transitions in the early Universe. Utilising data from the first three observing runs, we derive constraints on the parameters of the underlying gravitational-wave background, focusing on transitions characterised by strong supercooling. As an application of our analysis, we determine bounds on the parameter space of two representative particle physics models. We also comment on the expected reach of third-generation detectors in probing supercooled phase transitions.

hep-ph↗

arXiv2022

Birefringence Tomography for Axion Cloud

An axion cloud surrounding a supermassive black hole can be naturally produced through the superradiance process. Its existence can be examined by the axion induced birefringence effect. It predicts an oscillation of the electric vector position angle of linearly polarized radiations. Stringent constraints of the existence of the axion in a particular mass window has been obtained based on the recent Event Horizon Telescope measurement on M87$^\star$. The future Very-Long-Baseline Interferometry (VLBI) observations will be able to measure the vicinity of many supermassive black holes, thus it opens the possibility to search for the existence of axions in a wide mass regime. In this paper, we study how different black hole properties and accretion flows influence the signatures of the axion induced birefringence. We include the impacts of black hole inclination angles, spins, magnetic fields, plasma velocity distributions, the thickness of the accretion flows. We pay special attention to characterize the washout effects induced by the finite thickness of the accretion flows and the lensed photons. Based on this study, we give prospects on how to optimize the axion search using future VLBI observations, such as the next-generation Event Horizon Telescope, to further increase the sensitivity.

hep-ph↗

arXiv2022

Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks

Deep learning models have achieved excellent recognition results on large-scale video benchmarks. However, they perform poorly when applied to videos with rare scenes or objects, primarily due to the bias of existing video datasets. We tackle this problem from two different angles: algorithm and dataset. From the perspective of algorithms, we propose Spatial-aware Multi-Aspect Debiasing (SMAD), which incorporates both explicit debiasing with multi-aspect adversarial training and implicit debiasing with the spatial actionness reweighting module, to learn a more generic representation invariant to non-action aspects. To neutralize the intrinsic dataset bias, we propose OmniDebias to leverage web data for joint training selectively, which can achieve higher performance with far fewer web data. To verify the effectiveness, we establish evaluation protocols and perform extensive experiments on both re-distributed splits of existing datasets and a new evaluation dataset focusing on the action with rare scenes. We also show that the debiased representation can generalize better when transferred to other datasets and tasks.

cs.CV↗

arXiv2022

Real-time Online Video Detection with Temporal Smoothing Transformers

Streaming video recognition reasons about objects and their actions in every frame of a video. A good streaming recognition model captures both long-term dynamics and short-term changes of video. Unfortunately, in most existing methods, the computational complexity grows linearly or quadratically with the length of the considered dynamics. This issue is particularly pronounced in transformer-based architectures. To address this issue, we reformulate the cross-attention in a video transformer through the lens of kernel and apply two kinds of temporal smoothing kernel: A box kernel or a Laplace kernel. The resulting streaming attention reuses much of the computation from frame to frame, and only requires a constant time update each frame. Based on this idea, we build TeSTra, a Temporal Smoothing Transformer, that takes in arbitrarily long inputs with constant caching and computing overhead. Specifically, it runs $6\times$ faster than equivalent sliding-window based transformers with 2,048 frames in a streaming setting. Furthermore, thanks to the increased temporal span, TeSTra achieves state-of-the-art results on THUMOS'14 and EPIC-Kitchen-100, two standard online action detection and action anticipation datasets. A real-time version of TeSTra outperforms all but one prior approaches on the THUMOS'14 dataset.

cs.CV↗

arXiv2022

TOD: GPU-accelerated Outlier Detection via Tensor Operations

Outlier detection (OD) is a key learning task for finding rare and deviant data samples, with many time-critical applications such as fraud detection and intrusion detection. In this work, we propose TOD, the first tensor-based system for efficient and scalable outlier detection on distributed multi-GPU machines. A key idea behind TOD is decomposing complex OD applications into a small collection of basic tensor algebra operators. This decomposition enables TOD to accelerate OD computations by leveraging recent advances in deep learning infrastructure in both hardware and software. Moreover, to deploy memory-intensive OD applications on modern GPUs with limited on-device memory, we introduce two key techniques. First, provable quantization speeds up OD computations and reduces its memory footprint by automatically performing specific floating-point operations in lower precision while provably guaranteeing no accuracy loss. Second, to exploit the aggregated compute resources and memory capacity of multiple GPUs, we introduce automatic batching, which decomposes OD computations into small batches for both sequential execution on a single GPU and parallel execution on multiple GPUs. TOD supports a diverse set of OD algorithms. Extensive evaluation on 11 real and 3 synthetic OD datasets shows that TOD is on average 10.9x faster than the leading CPU-based OD system PyOD (with a maximum speedup of 38.9x), and can handle much larger datasets than existing GPU-based OD systems. In addition, TOD allows easy integration of new OD operators, enabling fast prototyping of emerging and yet-to-discovered OD algorithms.

cs.LG↗

arXiv2022

ADBench: Anomaly Detection Benchmark

Given a long list of anomaly detection algorithms developed in the last few decades, how do they perform with regard to (i) varying levels of supervision, (ii) different types of anomalies, and (iii) noisy and corrupted data? In this work, we answer these key questions by conducting (to our best knowledge) the most comprehensive anomaly detection benchmark with 30 algorithms on 57 benchmark datasets, named ADBench. Our extensive experiments (98,436 in total) identify meaningful insights into the role of supervision and anomaly types, and unlock future directions for researchers in algorithm selection and design. With ADBench, researchers can easily conduct comprehensive and fair evaluations for newly proposed methods on the datasets (including our contributed ones from natural language and computer vision domains) against the existing baselines. To foster accessibility and reproducibility, we fully open-source ADBench and the corresponding results.

cs.LG↗

arXiv2022

ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

Outlier detection refers to the identification of data points that deviate from a general data distribution. Existing unsupervised approaches often suffer from high computational cost, complex hyperparameter tuning, and limited interpretability, especially when working with large, high-dimensional datasets. To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution. In a nutshell, ECOD first estimates the underlying distribution of the input data in a nonparametric fashion by computing the empirical cumulative distribution per dimension of the data. ECOD then uses these empirical distributions to estimate tail probabilities per dimension for each data point. Finally, ECOD computes an outlier score of each data point by aggregating estimated tail probabilities across dimensions. Our contributions are as follows: (1) we propose a novel outlier detection method called ECOD, which is both parameter-free and easy to interpret; (2) we perform extensive experiments on 30 benchmark datasets, where we find that ECOD outperforms 11 state-of-the-art baselines in terms of accuracy, efficiency, and scalability; and (3) we release an easy-to-use and scalable (with distributed support) Python implementation for accessibility and reproducibility.

cs.LG↗

arXiv2022

Federated Learning with Non-IID Data

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

cs.LG↗

arXiv2022

BA-Net: Bridge Attention for Deep Convolutional Neural Networks

In recent years, channel attention mechanism has been widely investigated due to its great potential in improving the performance of deep convolutional neural networks (CNNs) in many vision tasks. However, in most of the existing methods, only the output of the adjacent convolution layer is fed into the attention layer for calculating the channel weights. Information from other convolution layers has been ignored. With these observations, a simple strategy, named Bridge Attention Net (BA-Net), is proposed in this paper for better performance with channel attention mechanisms. The core idea of this design is to bridge the outputs of the previous convolution layers through skip connections for channel weights generation. Based on our experiment and theory analysis, we find that features from previous layers also contribute to the weights significantly. The Comprehensive evaluation demonstrates that the proposed approach achieves state-of-the-art(SOTA) performance compared with the existing methods in accuracy and speed. which shows that Bridge Attention provides a new perspective on the design of neural network architectures with great potential in improving performance. The code is available at https://github.com/zhaoy376/Bridge-Attention.

cs.CV↗

arXiv2022

Improved random batch Ewald method in molecular dynamics simulations

The random batch Ewald (RBE) is an efficient and accurate method for molecular dynamics (MD) simulations of physical systems at the nano-/micro- scale. The method shows great potential to solve the computational bottleneck of long-range interactions, motivating a necessity to accelerating short-range components of the non-bonded interactions for a further speedup of MD simulations. In this work, we present an improved RBE method for the non-bonding interactions by introducing the random batch idea to constructing neighbor lists for the treatment of both the short-range part of the Ewald splitting and the Lennard-Jones potential. The efficiency of the novel neighbor list algorithm owes to the stochastic minibatch strategy which can significantly reduce the total number of neighbors. We obtan the error estimate and convergence by theoretical analysis and implement the improved RBE method in the LAMMPS package. Benchmark simulations are performed to demonstrate the accuracy and stability of the algorithm. Numerical tests on computer performance by conducting large-scaled MD simulations for systems including up to 0.1 billion water molecules, run on massive cluster with up to 50 thousand CPU cores, demonstrating the attractive features such as the high parallel scalability and memory-saving of the method in comparison to the existing methods.

physics.comp-ph↗

arXiv2022

Intrinsically Motivated Self-supervised Learning in Reinforcement Learning

In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abundant information in self-supervised auxiliary tasks has been disregarded, since the representation learning part and the decision-making part are separated. To sufficiently utilize information in auxiliary tasks, we present a simple yet effective idea to employ self-supervised loss as an intrinsic reward, called Intrinsically Motivated Self-Supervised learning in Reinforcement learning (IM-SSR). We formally show that the self-supervised loss can be decomposed as exploration for novel states and robustness improvement from nuisance elimination. IM-SSR can be effortlessly plugged into any reinforcement learning with self-supervised auxiliary objectives with nearly no additional cost. Combined with IM-SSR, the previous underlying algorithms achieve salient improvements on both sample efficiency and generalization in various vision-based robotics tasks from the DeepMind Control Suite, especially when the reward signal is sparse.

cs.LG↗

arXiv2022

Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis

Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectively strengthens the discriminability of hash codes. Specifically, the Deep Deterministic Information Bottleneck (Yu, Yu, and Principe 2021) for single modality is extended to the cross-modal scenario. Benefiting from this, the superfluous information is reduced, which facilitates the discriminability of hash codes. Experimental results demonstrate the superior accuracy of the proposed DSIBH compared with state-of-the-arts in cross-modal medical data retrieval tasks.

cs.LG↗

arXiv2022

A Deep Reinforcement Learning Framework for Rapid Diagnosis of Whole Slide Pathological Images

The deep neural network is a research hotspot for histopathological image analysis, which can improve the efficiency and accuracy of diagnosis for pathologists or be used for disease screening. The whole slide pathological image can reach one gigapixel and contains abundant tissue feature information, which needs to be divided into a lot of patches in the training and inference stages. This will lead to a long convergence time and large memory consumption. Furthermore, well-annotated data sets are also in short supply in the field of digital pathology. Inspired by the pathologist's clinical diagnosis process, we propose a weakly supervised deep reinforcement learning framework, which can greatly reduce the time required for network inference. We use neural network to construct the search model and decision model of reinforcement learning agent respectively. The search model predicts the next action through the image features of different magnifications in the current field of view, and the decision model is used to return the predicted probability of the current field of view image. In addition, an expert-guided model is constructed by multi-instance learning, which not only provides rewards for search model, but also guides decision model learning by the knowledge distillation method. Experimental results show that our proposed method can achieve fast inference and accurate prediction of whole slide images without any pixel-level annotations.

eess.IV↗

arXiv2022

Clues Before Answers: Generation-Enhanced Multiple-Choice QA

A trending paradigm for multiple-choice question answering (MCQA) is using a text-to-text framework. By unifying data in different tasks into a single text-to-text format, it trains a generative encoder-decoder model which is both powerful and universal. However, a side effect of twisting a generation target to fit the classification nature of MCQA is the under-utilization of the decoder and the knowledge that can be decoded. To exploit the generation capability and underlying knowledge of a pre-trained encoder-decoder model, in this paper, we propose a generation-enhanced MCQA model named GenMC. It generates a clue from the question and then leverages the clue to enhance a reader for MCQA. It outperforms text-to-text models on multiple MCQA datasets.

cs.CL↗

arXiv2022

Symmetry breaking and anomalous conductivity in a double moiré superlattice

A double moiré superlattice can be realized by stacking three layers of atomically thin two-dimensional materials with designer interlayer twisting or lattice mismatches. In this novel structure, atomic reconstruction of constituent layers could introduce significant modifications to the lattice symmetry and electronic structure at small twist angles. Here, we employ conductive atomic force microscopy (cAFM) to investigate symmetry breaking and local electrical properties in twisted trilayer graphene. We observe clear double moiré superlattices with two distinct moire periods all over the sample. At neighboring domains of the large moiré, the current exhibit either two- or six-fold rotational symmetry, indicating delicate symmetry breaking beyond the rigid model. Moreover, an anomalous current appears at the 'A-A' stacking site of the larger moiré, contradictory to previous observations on twisted bilayer graphene. Both behaviors can be understood by atomic reconstruction, and we also show that the cAFM signal of twisted graphene samples is dominated by the tip-graphene contact resistance that maps the local work function of twisted graphene and the metallic tip qualitatively. Our results unveil cAFM is an effective probe for visualizing atomic reconstruction and symmetry breaking in novel moiré superlattices, which could provide new insights for exploring and manipulating more exotic quantum states based on twisted van der Waals heterostructures.

cond-mat.mes-hall↗

arXiv2022

Gaussian Kernel Variance For an Adaptive Learning Method on Signals Over Graphs

This paper discusses a special kind of a simple yet possibly powerful algorithm, called single-kernel Gradraker (SKG), which is an adaptive learning method predicting unknown nodal values in a network using known nodal values and the network structure. We aim to find out how to configure the special kind of the model in applying the algorithm. To be more specific, we focus on SKG with a Gaussian kernel and specify how to find a suitable variance for the kernel. To do so, we introduce two variables with which we are able to set up requirements on the variance of the Gaussian kernel to achieve (near-) optimal performance and can better understand how SKG works. Our contribution is that we introduce two variables as analysis tools, illustrate how predictions will be affected under different Gaussian kernels, and provide an algorithm finding a suitable Gaussian kernel for SKG with knowledge about the training network. Simulation results on real datasets are provided.

eess.SP↗

arXiv2022

Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation

Precise segmentation of teeth from intra-oral scanner images is an essential task in computer-aided orthodontic surgical planning. The state-of-the-art deep learning-based methods often simply concatenate the raw geometric attributes (i.e., coordinates and normal vectors) of mesh cells to train a single-stream network for automatic intra-oral scanner image segmentation. However, since different raw attributes reveal completely different geometric information, the naive concatenation of different raw attributes at the (low-level) input stage may bring unnecessary confusion in describing and differentiating between mesh cells, thus hampering the learning of high-level geometric representations for the segmentation task. To address this issue, we design a two-stream graph convolutional network (i.e., TSGCN), which can effectively handle inter-view confusion between different raw attributes to more effectively fuse their complementary information and learn discriminative multi-view geometric representations. Specifically, our TSGCN adopts two input-specific graph-learning streams to extract complementary high-level geometric representations from coordinates and normal vectors, respectively. Then, these single-view representations are further fused by a self-attention module to adaptively balance the contributions of different views in learning more discriminative multi-view representations for accurate and fully automatic tooth segmentation. We have evaluated our TSGCN on a real-patient dataset of dental (mesh) models acquired by 3D intraoral scanners. Experimental results show that our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation. Github: https://github.com/ZhangLingMing1/TSGCNet.

eess.IV↗

arXiv2022

GCN-MIF: Graph Convolutional Network with Multi-Information Fusion for Low-dose CT Denoising

Being low-level radiation exposure and less harmful to health, low-dose computed tomography (LDCT) has been widely adopted in the early screening of lung cancer and COVID-19. LDCT images inevitably suffer from the degradation problem caused by complex noises. It was reported that deep learning (DL)-based LDCT denoising methods using convolutional neural network (CNN) achieved impressive denoising performance. Although most existing DL-based methods (e.g., encoder-decoder framework) can implicitly utilize non-local and contextual information via downsampling operator and 3D CNN, the explicit multi-information (i.e., local, non-local, and contextual) integration may not be explored enough. To address this issue, we propose a novel graph convolutional network-based LDCT denoising model, namely GCN-MIF, to explicitly perform multi-information fusion for denoising purpose. Concretely, by constructing intra- and inter-slice graph, the graph convolutional network is introduced to leverage the non-local and contextual relationships among pixels. The traditional CNN is adopted for the extraction of local information. Finally, the proposed GCN-MIF model fuses all the extracted local, non-local, and contextual information. Extensive experiments show the effectiveness of our proposed GCN-MIF model by quantitative and visualized results. Furthermore, a double-blind reader study on a public clinical dataset is also performed to validate the usability of denoising results in terms of the structural fidelity, the noise suppression, and the overall score. Models and code are available at https://github.com/tonyckc/GCN-MIF_demo.

eess.IV↗