arXiv subjects

Ao Zhang

Publications and source records attributed to Ao Zhang.

61 records · Page 4Linked to original sources

arXiv2021

Pre-Trained Models: Past, Present and Future

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophisticated pre-training objectives and huge model parameters, large-scale PTMs can effectively capture knowledge from massive labeled and unlabeled data. By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks, which has been extensively demonstrated via experimental verification and empirical analysis. It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch. In this paper, we take a deep look into the history of pre-training, especially its special relation with transfer learning and self-supervised learning, to reveal the crucial position of PTMs in the AI development spectrum. Further, we comprehensively review the latest breakthroughs of PTMs. These breakthroughs are driven by the surge of computational power and the increasing availability of data, towards four important directions: designing effective architectures, utilizing rich contexts, improving computational efficiency, and conducting interpretation and theoretical analysis. Finally, we discuss a series of open problems and research directions of PTMs, and hope our view can inspire and advance the future study of PTMs.

cs.AI↗

arXiv2021

RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users

Object detection using automotive radars has not been explored with deep learning models in comparison to the camera based approaches. This can be attributed to the lack of public radar datasets. In this paper, we collect a novel radar dataset that contains radar data in the form of Range-Azimuth-Doppler tensors along with the bounding boxes on the tensor for dynamic road users, category labels, and 2D bounding boxes on the Cartesian Bird-Eye-View range map. To build the dataset, we propose an instance-wise auto-annotation method. Furthermore, a novel Range-Azimuth-Doppler based multi-class object detection deep learning model is proposed. The algorithm is a one-stage anchor-based detector that generates both 3D bounding boxes and 2D bounding boxes on Range-Azimuth-Doppler and Cartesian domains, respectively. Our proposed algorithm achieves 56.3% AP with IOU of 0.3 on 3D bounding box predictions, and 51.6% with IOU of 0.5 on 2D bounding box prediction. Our dataset and the code can be found at https://github.com/ZhangAoCanada/RADDet.git.

cs.CV↗

arXiv2020

Effective Approximation for a Stochastic System with Large Potential

This letter is about effective approximation for a stochastic parabolic equation with a large potential in a periodic medium. Under a condition on the spectral properties of the associated cell problem, we prove that the solution can be approximately factorized as the product of a fast oscillating cell eigenfunction and of a solution of an effective equation.

math.AP↗

arXiv2020

Effects of substrate-surface reconstruction and orientation on spin-valley polarization in MoTe$_2$/EuO

We investigate the spin-valley polarization in MoTe$_2$ monolayer on (111) and (001) surfaces of ferromagnetic semiconductor EuO based on first-principles calculations. We consider surface reconstructions for EuO(111). We find that there is no direct chemical bonding between the reconstructed EuO(111) and the MoTe$_2$ overlayer, in contrast to the case of the ideal EuO(111). However, there is a strong hybridization between the states of MoTe$_2$ and the substrate states, which has a substantial impact on the valleys. The valley polarization due to the magnetic proximity effect is dependent on the detail of the interface structure, which is in the range of a few meV to about 40 meV. These values are at least one order of magnitude smaller than that induced by the ideal EuO(111). When the MoTe$_2$ monolayer is interfaced with EuO(001), the valley polarization is about 3.2 meV, insensitive to the interface structure. By a low-energy effective Hamiltonian model, the effective Zeeman field induced by EuO(001) is about 27 T, comparable to that for WSe$_2$/EuS obtained by experiment.

cond-mat.mtrl-sci↗

arXiv2020

DefenseVGAE: Defending against Adversarial Attacks on Graph Data via a Variational Graph Autoencoder

Graph neural networks (GNNs) achieve remarkable performance for tasks on graph data. However, recent works show they are extremely vulnerable to adversarial structural perturbations, making their outcomes unreliable. In this paper, we propose DefenseVGAE, a novel framework leveraging variational graph autoencoders(VGAEs) to defend GNNs against such attacks. DefenseVGAE is trained to reconstruct graph structure. The reconstructed adjacency matrix can reduce the effects of adversarial perturbations and boost the performance of GCNs when facing adversarial attacks. Our experiments on a number of datasets show the effectiveness of the proposed method under various threat models. Under some settings it outperforms existing defense strategies. Our code has been made publicly available at https://github.com/zhangao520/defense-vgae.

cs.LG↗

arXiv2020

Effective Wave Factorization for a Stochastic Schrödinger Equation

We study the homogenization of a stochastic Schrödinger equation with a large periodic potential in solid state physics. Denoting by $\varepsilon$ the period, the potential is scaled as $\varepsilon^{-2}$. Under a generic assumption on the spectral properties of the associated cell problem, we prove that the solution can be approximately factorized as the product of a fast oscillating cell eigenfunction and of a slowly varying solution of an effective equation. Our method is based on two-scale convergence and Bloch waves theory.

math.AP↗

arXiv2018

tau-FPL: Tolerance-Constrained Learning in Linear Time

Learning a classifier with control on the false-positive rate plays a critical role in many machine learning applications. Existing approaches either introduce prior knowledge dependent label cost or tune parameters based on traditional classifiers, which lack consistency in methodology because they do not strictly adhere to the false-positive rate constraint. In this paper, we propose a novel scoring-thresholding approach, tau-False Positive Learning (tau-FPL) to address this problem. We show the scoring problem which takes the false-positive rate tolerance into accounts can be efficiently solved in linear time, also an out-of-bootstrap thresholding method can transform the learned ranking function into a low false-positive classifier. Both theoretical analysis and experimental results show superior performance of the proposed tau-FPL over existing approaches.

cs.LG↗