arXiv Science⌕ Search

arXiv subjects

Peng Gao

Publications and source records attributed to Peng Gao.

At least 343 records · Page 19Linked to original sources
arXiv2021

Container: Context Aggregation Network

Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations. Recently, Transformers -- originally introduced in natural language processing -- have been increasingly adopted in computer vision. While early adopters continue to employ CNN backbones, the latest networks are end-to-end CNN-free Transformer solutions. A recent surprising finding shows that a simple MLP based solution without any traditional convolutional or Transformer components can produce effective visual representations. While CNNs, Transformers and MLP-Mixers may be considered as completely disparate architectures, we provide a unified view showing that they are in fact special cases of a more general method to aggregate spatial context in a neural network stack. We present the \model (CONText AggregatIon NEtwoRk), a general-purpose building block for multi-head context aggregation that can exploit long-range interactions \emph{a la} Transformers while still exploiting the inductive bias of the local convolution operation leading to faster convergence speeds, often seen in CNNs. In contrast to Transformer-based methods that do not scale well to downstream tasks that rely on larger input image resolutions, our efficient network, named \modellight, can be employed in object detection and instance segmentation networks such as DETR, RetinaNet and Mask-RCNN to obtain an impressive detection mAP of 38.9, 43.8, 45.1 and mask mAP of 41.3, providing large improvements of 6.6, 7.3, 6.9 and 6.6 pts respectively, compared to a ResNet-50 backbone with a comparable compute and parameter size. Our method also achieves promising results on self-supervised learning compared to DeiT on the DINO framework. Code is released at \url{https://github.com/allenai/container}.

cs.CV↗
arXiv2021

Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning

In previous work, we have proposed the Audio-Visual Scene-Aware Dialog (AVSD) task, collected an AVSD dataset, developed AVSD technologies, and hosted an AVSD challenge track at both the 7th and 8th Dialog System Technology Challenges (DSTC7, DSTC8). In these challenges, the best-performing systems relied heavily on human-generated descriptions of the video content, which were available in the datasets but would be unavailable in real-world applications. To promote further advancements for real-world applications, we proposed a third AVSD challenge, at DSTC10, with two modifications: 1) the human-created description is unavailable at inference time, and 2) systems must demonstrate temporal reasoning by finding evidence from the video to support each answer. This paper introduces the new task that includes temporal reasoning and our new extension of the AVSD dataset for DSTC10, for which we collected human-generated temporal reasoning data. We also introduce a baseline system built using an AV-transformer, which we released along with the new dataset. Finally, this paper introduces a new system that extends our baseline system with attentional multimodal fusion, joint student-teacher learning (JSTL), and model combination techniques, achieving state-of-the-art performances on the AVSD datasets for DSTC7, DSTC8, and DSTC10. We also propose two temporal reasoning methods for AVSD: one attention-based, and one based on a time-domain region proposal network.

cs.CL↗
arXiv2021

Multi-channel Speech Enhancement with 2-D Convolutional Time-frequency Domain Features and a Pre-trained Acoustic Model

We propose a multi-channel speech enhancement approach with a novel two-stage feature fusion method and a pre-trained acoustic model in a multi-task learning paradigm. In the first fusion stage, the time-domain and frequency-domain features are extracted separately. In the time domain, the multi-channel convolution sum (MCS) and the inter-channel convolution differences (ICDs) features are computed and then integrated with the first 2-D convolutional layer, while in the frequency domain, the log-power spectra (LPS) features from both original channels and super-directive beamforming outputs are combined with a second 2-D convolutional layer. To fully integrate the rich information of multi-channel speech, i.e. time-frequency domain features and the array geometry, we apply a third 2-D convolutional layer in the second fusion stage to obtain the final convolutional features. Furthermore, we propose to use a fixed clean acoustic model trained with the end-to-end lattice-free maximum mutual information criterion to enforce the enhanced output to have the same distribution as the clean waveform to alleviate the over-estimation problem of the enhancement task and constrain distortion. On the Task1 development dataset of ConferencingSpeech 2021 challenge, a PESQ improvement of 0.24 and 0.19 is attained compared to the official baseline and a recently proposed multi-channel separation method.

cs.SD↗
arXiv2021

Dense Contrastive Visual-Linguistic Pretraining

Inspired by the success of BERT, several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. In particular, LXMERT and UNITER adopt visual region feature regression and label classification as pretext tasks. However, they tend to suffer from the problems of noisy labels and sparse semantic annotations, based on the visual features having been pretrained on a crowdsourced dataset with limited and inconsistent semantic labeling. To overcome these issues, we propose unbiased Dense Contrastive Visual-Linguistic Pretraining (DCVLP), which replaces the region regression and classification with cross-modality region contrastive learning that requires no annotations. Two data augmentation strategies (Mask Perturbation and Intra-/Inter-Adversarial Perturbation) are developed to improve the quality of negative samples used in contrastive learning. Overall, DCVLP allows cross-modality dense region contrastive learning in a self-supervised setting independent of any object annotations. We compare our method against prior visual-linguistic pretraining frameworks to validate the superiority of dense contrastive learning on multimodal representation learning.

cs.CV↗
arXiv2021

DFTTK: Density Functional Theory ToolKit for High-throughput Lattice Dynamics Calculations

In this work, we present a software package in Python for high-throughput first-principles calculations of thermodynamic properties at finite temperatures, which we refer to as DFTTK (Density Functional Theory Tool Kit). DFTTK is based on the atomate package and integrates our experiences in the last decades on the development of theoretical methods and computational software. It includes task submissions on all major operating systems and task execution on high-performance computing environments. The distribution of the DFTTK package comes with examples of calculations of phonon density of states, heat capacity, entropy, enthalpy, and free energy under the quasi-harmonic phonon scheme for the stoichiometric phases of Al, Ni, Al3Ni, AlNi, AlNi3, Al3Ni4, and Al3Ni5, and the fcc solution phases treated using the special quasirandom structures at the compositions of Al3Ni, AlNi, and AlNi3.

cond-mat.mtrl-sci↗
arXiv2021

Atomic-Scale Probing of Heterointerface Phonon Bridges in Nitride Semiconductor

Interface phonon modes that are generated by several atomic layers at the heterointerface play a major role in the interface thermal conductance for nanoscale high-power devices such as nitride-based high-electron-mobility transistors and light emitting diodes. Here we measure the local phonon spectra across AlN/Si and AlN/Al interfaces using atomically resolved vibrational electron energy-loss spectroscopy in a scanning transmission electron microscope. At the AlN/Si interface, we observe various localized phonon modes, of which the extended and interfacial modes act as bridges to connect the bulk AlN modes and bulk Si modes, and are expected to boost the inelastic phonon transport thus substantially contribute to interface thermal conductance. In comparison, no such phonon bridge is observed at the AlN/Al interface, for which partially extended modes dominate the interface thermal conductivity. This work provides valuable insights into understanding the interfacial thermal transport in nitride semiconductors and useful guidance for thermal management via interface engineering.

cond-mat.mes-hall↗
arXiv2021

Measuring phonon dispersion at an interface

The breakdown of translational symmetry at heterointerfaces leads to the emergence of new phonon modes localized near the interface. These interface phonons play an essential role in thermal/electrical transport properties in devices especially in miniature ones wherein the interface may dominate the entire response of the device. Knowledge of phonon dispersion at interfaces is therefore highly desirable for device design and optimization. Although theoretical work has begun decades ago, experimental research is totally absent due to challenges in achieving combined spatial, momentum and spectral resolutions required to probe localized phonon modes. Here we use electron energy loss spectroscopy in an electron microscope to directly measure both the local phonon density of states and the interface phonon dispersion relation for an epitaxial cBN-diamond heterointerface. In addition to bulk phonon modes, we observe acoustic and optical phonon modes localized at the interface, and modes isolated away from the interface. These features only appear within ~ 1 nm around the interface. The experimental results can be nicely reproduced by ab initio calculations. Our findings provide insights into lattice dynamics at heterointerfaces and should be practically useful in thermal/electrical engineering.

cond-mat.mtrl-sci↗
arXiv2021

Fast Convergence of DETR with Spatially Modulated Co-Attention

The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN. However, DETR suffers from its slow convergence. Training DETR from scratch needs 500 epochs to achieve a high accuracy. To accelerate its convergence, we propose a simple yet effective scheme for improving the DETR framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. The core idea of SMCA is to conduct location-aware co-attention in DETR by constraining co-attention responses to be high near initially estimated bounding box locations. Our proposed SMCA increases DETR's convergence speed by replacing the original co-attention mechanism in the decoder while keeping other operations in DETR unchanged. Furthermore, by integrating multi-head and scale-selection attention designs into SMCA, our fully-fledged SMCA can achieve better performance compared to DETR with a dilated convolution-based backbone (45.6 mAP at 108 epochs vs. 43.3 mAP at 500 epochs). We perform extensive ablation studies on COCO dataset to validate SMCA. Code is released at https://github.com/gaopengcuhk/SMCA-DETR .

cs.CV↗
arXiv2021

Engineering of Atomic-Scale Flexoelectricity at Grain Boundaries

Flexoelectricity is a type of ubiquitous and prominent electromechanical coupling, pertaining to the response of electrical polarization to mechanical strain gradients while not restricted to the symmetry of materials. However, large elastic deformation in most solids is usually difficult to achieve and the strain gradient at minuscule is challenging to control. Here we exploit the exotic structural inhomogeneity of grain boundary to achieve a huge strain gradient (~ 1.2 nm-1) within 3 ~ 4 unit-cells, and thus obtain atomic-scale flexoelectric polarization up to ~ 38 μC/cm2 at a 24 LaAlO3 grain boundary. The nanoscale flexoelectricity also modifies the electrical activity of grain boundaries. Moreover, we prove that it is a general and feasible way to form large strain gradients at atomic scale by altering the misorientation angles of grain boundaries in different dielectric materials. Thus, engineering of grain boundaries provides an effective pathway to achieve tunable flexoelectricity and broadens the electromechanical functionalities of non-piezoelectric materials.

cond-mat.mtrl-sci↗
arXiv2021

Dynamics of Polar Skyrmion Bubbles under Electric Fields

Room-temperature polar skyrmion bubbles that are recently found in oxide superlattice, have received enormous interests for their potential applications in nanoelectronics due to the nanometer size, emergent chirality, and negative capacitance. For practical applications, the ability to controllably manipulate them by using external stimuli is prerequisite. Here, we study the dynamics of individual polar skyrmion bubbles at the nanoscale by using in situ biasing in a scanning transmission electron microscope. The reversible electric field-driven phase transition between topological and trivial polar states are demonstrated. We create, erase and monitor the shrinkage and expansion of individual polar skyrmions. We find that their transition behaviors are substantially different from that of magnetic analogue. The underlying mechanism is discussed by combing with the phase-field simulations. The controllable manipulation of nanoscale polar skyrmions allows us to tune the dielectric permittivity at atomic scale and detailed knowledge of their phase transition behaviors provides fundamentals for their applications in nanoelectronics.

cond-mat.mes-hall↗
arXiv2021

Switching magnon chirality in artificial antiferromagnet

Magnons in antiferromagnets can support both right-handed and left-handed chiralities, which shed a light on the chirality-based spintronics. Here we demonstrate the switching and reading of magnon chirality in an artificial antiferromagnet. The coexisting antiferromagnetic and ferromagnetic characteristic resonance modes are discovered, which permits a high tunability in the modulation of magnon chirality. The reading of the chirality is accomplished via the chirality-dependent spin pumping as well as spin rectification effect. Our result illustrates an ideal antiferromagnetic platform for handling magnon chirality and paves the way for chirality-based spintronics.

physics.app-ph↗
arXiv2021

Atomic-scale imaging of CH3NH3PbI3 structure and its decomposition pathway

Understanding the atomic structure and structural instability of organic-inorganic hybrid perovskites is the key to appreciate their remarkable photoelectric properties and failure mechanism. Here, using low-dose imaging technique by direct-detection electron-counting camera in transmission electron microscope, we investigate the atomic structure and decomposition pathway of CH3NH3PbI3 (MAPbI3) at the atomic scale. We successfully image the atomic structure of perovskite in real space under ultra-low electron dose condition, and observe a two-step decomposition process, i.e. initial loss of MA followed by the collapse of perovskite structure into 6H-PbI2 with their critical threshold dose also determined. Interestingly, an intermediate phase (MA0.5PbI3) with locally ordered vacancies can robustly exist before perovskite collapses, enlightening strategies for prevention and recovery of perovskite structure during degradation. Associated with structure evolution, the bandgap gradually increases from ~1.6 eV to ~2.1 eV, and it is found that both C-N and N-H bonds can be destroyed under irradiation, releasing NH3 and leaving hydrocarbons. These findings enhance our understanding of the photoelectric properties and failure mechanism of MAPbI3, providing potential strategy into material optimization.

physics.app-ph↗
arXiv2021

Scalable Transformers for Neural Machine Translation

Transformer has been widely adopted in Neural Machine Translation (NMT) because of its large capacity and parallel training of sequence generation. However, the deployment of Transformer is challenging because different scenarios require models of different complexities and scales. Naively training multiple Transformers is redundant in terms of both computation and memory. In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters. Each sub-Transformer can be easily obtained by cropping the parameters of the largest Transformer. A three-stage training scheme is proposed to tackle the difficulty of training the Scalable Transformers, which introduces additional supervisions from word-level and sequence-level self-distillation. Extensive experiments were conducted on WMT EN-De and En-Fr to validate our proposed Scalable Transformers.

cs.CL↗
arXiv2021

Sharp lower bounds for moments of $ζ'(ρ)$

We study the $2k$-th discrete moment of the derivative of the Riemann zeta-function at nontrivial zeros to establish sharp lower bounds for all real $k \geq 0$ under the Riemann hypothesis (RH).

math.NT↗
arXiv2021

Oriented Object Detection with Transformer

Object detection with Transformers (DETR) has achieved a competitive performance over traditional detectors, such as Faster R-CNN. However, the potential of DETR remains largely unexplored for the more challenging task of arbitrary-oriented object detection problem. We provide the first attempt and implement Oriented Object DEtection with TRansformer ($\bf O^2DETR$) based on an end-to-end network. The contributions of $\rm O^2DETR$ include: 1) we provide a new insight into oriented object detection, by applying Transformer to directly and efficiently localize objects without a tedious process of rotated anchors as in conventional detectors; 2) we design a simple but highly efficient encoder for Transformer by replacing the attention mechanism with depthwise separable convolution, which can significantly reduce the memory and computational cost of using multi-scale features in the original Transformer; 3) our $\rm O^2DETR$ can be another new benchmark in the field of oriented object detection, which achieves up to 3.85 mAP improvement over Faster R-CNN and RetinaNet. We simply fine-tune the head mounted on $\rm O^2DETR$ in a cascaded architecture and achieve a competitive performance over SOTA in the DOTA dataset.

cs.CV↗
arXiv2021

Value-distribution of quartic Hecke $L$-functions

Set $K=\mathbb{Q}(i)$ and suppose that $c\in \mathbb{Z}[i]$ is a square-free algebraic integer with $c\equiv 1 \imod{\langle16\rangle}$. Let $L(s,χ_{c})$ denote the Hecke $L$-function associated with the quartic residue character modulo $c$. For $σ>1/2$, we prove an asymptotic distribution function $F_σ$ for the values of the logarithm of \begin{equation*} L_c(s)= L(s,χ_c)L(s,\overlineχ_{c}), \end{equation*} as $c$ varies. Moreover, the characteristic function of $F_σ$ is expressed explicitly as a product over the prime ideals of $\mathbb{Z}[i]$.

math.NT↗
arXiv2021

PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table Image Recognition to Latex

This paper presents our solution for the ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX. This competition has two sub-tasks: Table Structure Reconstruction (TSR) and Table Content Reconstruction (TCR). We treat both sub-tasks as two individual image-to-sequence recognition problems. We leverage our previously proposed algorithm MASTER \cite{lu2019master}, which is originally proposed for scene text recognition. We optimize the MASTER model from several perspectives: network structure, optimizer, normalization method, pre-trained model, resolution of input image, data augmentation, and model ensemble. Our method achieves 0.7444 Exact Match and 0.8765 Exact Match @95\% on the TSR task, and obtains 0.5586 Exact Match and 0.7386 Exact Match 95\% on the TCR task.

cs.CV↗