arXiv subjects

Yuan Yao

Publications and source records attributed to Yuan Yao.

At least 217 records · Page 12Linked to original sources

arXiv2023

Thermal diffuse scattering: elastic and coherent

Thermal diffuse scattering (TDS) caused by the interaction of high-energy electrons with phonons has been investigated. An oscillating atom retains all of its elastic scattering capacity, although the vibration changes the spatial distribution of the scattering potential. Therefore, the scattering by the vibrating lattice or phonons is still elastic. Furthermore, the TDS electrons re-main coherent and form the diffuse interference patterns even under the time-averaging assump-tion. Elastic and coherent TDS electrons contribute most of the contrast of Kikuchi lines.

cond-mat.mtrl-sci↗

arXiv2023

Duality viewpoint of criticality

In this work, we study quantum many-body systems which are self-dual under duality transformation connecting different symmetry protected topological (SPT) phases. We provide a geometric explanation of the criticality of these self-dual models. More precisely, we show a ground state (quasi-)degeneracy under the periodic boundary conditions,i.e., the ingappability of the bulk spectrum. Equivalently, the symmetry group at criticality, including the duality symmetry, has a mixed 't Hooft anomaly. This approach can not only predict the spectrum of the self-dual model with ordinary 0-form symmetry, but also be applied to that with generalized symmetry, such as higher form and subsystem symmetry. As an application, we illustrate our results with several examples in one and two dimensions, which separate two different SPTs.

quant-ph↗

arXiv2022

Capacity Analysis of Holographic MIMO Channels with Practical Constraints

Holographic Multiple-Input and Multiple-Output (MIMO) is envisioned as a promising technology to realize unprecedented spectral efficiency by integrating a large number of antennas into a compact space. Most research on holographic MIMO is based on isotropic scattering environments, and the antenna gain is assumed to be unlimited by deployment space. However, the channel might not satisfy isotropic scattering because of generalized angle distributions, and the antenna gain is limited by the array aperture in reality. In this letter, we aim to analyze the holographic MIMO channel capacity under practical angle distribution and array aperture constraints. First, we calculate the spectral density for generalized angle distributions by introducing a wavenumber domain-based method. And then, the capacity under generalized angle distributions is analyzed and two different aperture schemes are considered. Finally, numerical results show that the capacity is obviously affected by angle distribution at high signal-to-noise ratio (SNR) but hardly affected at low SNR, and the capacity will not increase infinitely with antenna density due to the array aperture constraint.

eess.SP↗

arXiv2022

Exploring Vision Transformers as Diffusion Learners

Score-based diffusion models have captured widespread attention and funded fast progress of recent vision generative tasks. In this paper, we focus on diffusion model backbone which has been much neglected before. We systematically explore vision Transformers as diffusion learners for various generative tasks. With our improvements the performance of vanilla ViT-based backbone (IU-ViT) is boosted to be on par with traditional U-Net-based methods. We further provide a hypothesis on the implication of disentangling the generative backbone as an encoder-decoder structure and show proof-of-concept experiments verifying the effectiveness of a stronger encoder for generative tasks with ASymmetriC ENcoder Decoder (ASCEND). Our improvements achieve competitive results on CIFAR-10, CelebA, LSUN, CUB Bird and large-resolution text-to-image tasks. To the best of our knowledge, we are the first to successfully train a single diffusion model on text-to-image task beyond 64x64 resolution. We hope this will motivate people to rethink the modeling choices and the training pipelines for diffusion-based generative models.

cs.CV↗

arXiv2022

3D Point Cloud Pre-training with Knowledge Distillation from 2D Images

The recent success of pre-trained 2D vision models is mostly attributable to learning from large-scale datasets. However, compared with 2D image datasets, the current pre-training data of 3D point cloud is limited. To overcome this limitation, we propose a knowledge distillation method for 3D point cloud pre-trained models to acquire knowledge directly from the 2D representation learning model, particularly the image encoder of CLIP, through concept alignment. Specifically, we introduce a cross-attention mechanism to extract concept features from 3D point cloud and compare them with the semantic information from 2D images. In this scheme, the point cloud pre-trained models learn directly from rich information contained in 2D teacher models. Extensive experiments demonstrate that the proposed knowledge distillation scheme achieves higher accuracy than the state-of-the-art 3D pre-training methods for synthetic and real-world datasets on downstream tasks, including object classification, object detection, semantic segmentation, and part segmentation.

cs.CV↗

arXiv2022

Computing Embedded Contact Homology in Morse-Bott Settings

Given a contact three manifold $Y$ with a nondegenerate contact form $λ$, and an almost complex structure $J$ compatible with $λ$, its embedded contact homology $ECH(Y,λ)$ is defined (arXiv:1303.5789) and only depends on the contact structure. In this paper we explain how to compute ECH for Morse-Bott contact forms whose Reeb orbits appear in $S^1$ families, assuming the almost complex structure $J$ can be chosen to satisfy certain transversality conditions (this is the case for instance for boundaries of concave or convex toric domains, or if all the curves of ECH index one have genus zero). We define the ECH chain complex for a Morse-Bott contact form via an enumeration of ECH index one cascades. We prove using gluing results from arXiv:2206.04334 that this chain complex computes the ECH of the contact manifold. This paper and arXiv:2206.04334 fill in some technical foundations for previous calculations in the literature (arXiv:1608.07988, arXiv:math/0410061).

math.SG↗

arXiv2022

PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models

Vision-language pre-training (VLP) has shown impressive performance on a wide range of cross-modal tasks, where VLP models without reliance on object detectors are becoming the mainstream due to their superior computation efficiency and competitive performance. However, the removal of object detectors also deprives the capability of VLP models in explicit object modeling, which is essential to various position-sensitive vision-language (VL) tasks, such as referring expression comprehension and visual commonsense reasoning. To address the challenge, we introduce PEVL that enhances the pre-training and prompt tuning of VLP models with explicit object position modeling. Specifically, PEVL reformulates discretized object positions and language in a unified language modeling framework, which facilitates explicit VL alignment during pre-training, and also enables flexible prompt tuning for various downstream tasks. We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on position-insensitive tasks with grounded inputs. We make the data and code for this paper publicly available at https://github.com/thunlp/PEVL.

cs.CV↗

arXiv2022

StrokeGAN+: Few-Shot Semi-Supervised Chinese Font Generation with Stroke Encoding

The generation of Chinese fonts has a wide range of applications. The currently predominated methods are mainly based on deep generative models, especially the generative adversarial networks (GANs). However, existing GAN-based models usually suffer from the well-known mode collapse problem. When mode collapse happens, the kind of GAN-based models will be failure to yield the correct fonts. To address this issue, we introduce a one-bit stroke encoding and a few-shot semi-supervised scheme (i.e., using a few paired data as semi-supervised information) to explore the local and global structure information of Chinese characters respectively, motivated by the intuition that strokes and characters directly embody certain local and global modes of Chinese characters. Based on these ideas, this paper proposes an effective model called \textit{StrokeGAN+}, which incorporates the stroke encoding and the few-shot semi-supervised scheme into the CycleGAN model. The effectiveness of the proposed model is demonstrated by amounts of experiments. Experimental results show that the mode collapse issue can be effectively alleviated by the introduced one-bit stroke encoding and few-shot semi-supervised training scheme, and that the proposed model outperforms the state-of-the-art models in fourteen font generation tasks in terms of four important evaluation metrics and the quality of generated characters. Besides CycleGAN, we also show that the proposed idea can be adapted to other existing models to improve their performance. The effectiveness of the proposed model for the zero-shot traditional Chinese font generation is also evaluated in this paper.

cs.CV↗

arXiv2022

Scheduling to Optimize Sojourn Time of Successful Jobs

Deep neural networks training jobs and other iterative computations frequently include checkpoints where jobs can be canceled based on the current value of monitored metrics. While most of existing results focus on the performance of all jobs (both successfully completed and canceled), in this work we explore scheduling policies that improve the sojourn time of successful jobs, which are typically more valuable to the user. Our model assumes that each job has a known discrete size distribution (e.g., estimated from previous execution logs) where the largest size value indicates a successful completion, while other size values correspond to termination checkpoints. In the single-server case where all jobs are available for scheduling simultaneously, we prove that optimal schedules do not preempt jobs, even when preemption overhead is negligible. Based on this, we develop a scheduling policy that minimizes the sojourn time of successful jobs asymptotically, i.e., when the number of jobs grows to infinity. Through an extensive numerical study, we show that this policy performs better than existing alternatives even when the number of jobs is finite. For more realistic scenarios with multiple servers and dynamic jobs arrivals, we propose an online approach based on our single-server scheduling policy. Through an extensive simulation study, using real-world traces, we demonstrate that this online approach results in better average sojourn time for successful jobs as compared to existing techniques.

cs.PF↗

arXiv2022

Scaling of Berry-curvature monopole dominated large linear positive magnetoresistance

The linear positive magnetoresistance (LPMR) is a widely observed phenomenon in topological materials, which is promising for potential applications on topological spintronics. However, its mechanism remains ambiguous yet and the effect is thus uncontrollable. Here, we report a quantitative scaling model that correlates the LPMR with the Berry curvature, based on a ferromagnetic Weyl semimetal CoS2 that bears the largest LPMR of over 500% at 2 Kelvin and 9 Tesla, among known magnetic topological semimetals. In this system, masses of Weyl nodes existing near the Fermi level, revealed by theoretical calculations, serve as Berry-curvature monopoles and low-effective-mass carriers. Based on the Weyl picture, we propose a relation \[\text{MR}=\frac{e}{\hbar }B{{Ω}_{\text{F}}}\], with B being the applied magnetic field and \[{{Ω}_{\text{F}}}\] the average Berry curvature near the Fermi surface, and further introduce temperature factor to both MR/B slope (MR per unit field) and anomalous Hall conductivity, which establishes the connection between the model and experimental measurements. A clear picture of the linearly slowing down of carriers, i.e., the LPMR effect, is demonstrated under the cooperation of the k-space Berry curvature and real-space magnetic field. Our study not only provides an experimental evidence of Berry curvature induced LPMR for the first time, but also promotes the common understanding and functional designing of the large Berry-curvature MR in topological Dirac/Weyl systems for magnetic sensing or information storage.

cond-mat.mtrl-sci↗

arXiv2022

Confidence Matters: Inspecting Backdoors in Deep Neural Networks via Distribution Transfer

Backdoor attacks have been shown to be a serious security threat against deep learning models, and detecting whether a given model has been backdoored becomes a crucial task. Existing defenses are mainly built upon the observation that the backdoor trigger is usually of small size or affects the activation of only a few neurons. However, the above observations are violated in many cases especially for advanced backdoor attacks, hindering the performance and applicability of the existing defenses. In this paper, we propose a backdoor defense DTInspector built upon a new observation. That is, an effective backdoor attack usually requires high prediction confidence on the poisoned training samples, so as to ensure that the trained model exhibits the targeted behavior with a high probability. Based on this observation, DTInspector first learns a patch that could change the predictions of most high-confidence data, and then decides the existence of backdoor by checking the ratio of prediction changes after applying the learned patch on the low-confidence data. Extensive evaluations on five backdoor attacks, four datasets, and three advanced attacking types demonstrate the effectiveness of the proposed defense.

cs.CR↗

arXiv2022

Fine-Grained Scene Graph Generation with Data Transfer

Scene graph generation (SGG) is designed to extract (subject, predicate, object) triplets in images. Recent works have made a steady progress on SGG, and provide useful tools for high-level vision and language understanding. However, due to the data distribution problems including long-tail distribution and semantic ambiguity, the predictions of current SGG models tend to collapse to several frequent but uninformative predicates (e.g., on, at), which limits practical application of these models in downstream tasks. To deal with the problems above, we propose a novel Internal and External Data Transfer (IETrans) method, which can be applied in a plug-and-play fashion and expanded to large SGG with 1,807 predicate classes. Our IETrans tries to relieve the data distribution problem by automatically creating an enhanced dataset that provides more sufficient and coherent annotations for all predicates. By training on the enhanced dataset, a Neural Motif model doubles the macro performance while maintaining competitive micro performance. The code and data are publicly available at https://github.com/waxnkw/IETrans-SGG.pytorch.

cs.CV↗

arXiv2022

Geometric approach to Lieb-Schultz-Mattis theorem without translation symmetry under inversion or rotation symmetry

We propose a geometric {approach to Lieb-Schultz-Mattis theorem for} quantum many-body systems with discrete spin-rotation symmetries and lattice inversion or rotation symmetry, but without translation symmetry assumed. Under symmetry-twisting on a $(d-1)$-dimensional plane, we find that any $d$-dimensional inversion-symmetric spin system possesses a doubly degenerate spectrum when it hosts a half-integer spin at the inversion-symmetric point. We also show that any rotation-symmetric generalized spin model with a projective representation at the rotation center has a similar degeneracy under symmetry-twisting. We argue that these degeneracies imply that {a unique symmetric gapped ground state that is smoothly connected to product states} is forbidden in the original untwisted systems -- generalized inversional/rotational Lieb-Schultz-Mattis theorems without lattice translation symmetry imposed. The traditional Lieb-Schultz-Mattis theorems with translations also fit in the proposed framework.

cond-mat.str-el↗

arXiv2022

DCT-Net: Domain-Calibrated Translation for Portrait Stylization

This paper introduces DCT-Net, a novel image translation architecture for few-shot portrait stylization. Given limited style exemplars ($\sim$100), the new architecture can produce high-quality style transfer results with advanced ability to synthesize high-fidelity contents and strong generality to handle complicated scenes (e.g., occlusions and accessories). Moreover, it enables full-body image translation via one elegant evaluation network trained by partial observations (i.e., stylized heads). Few-shot learning based style transfer is challenging since the learned model can easily become overfitted in the target domain, due to the biased distribution formed by only a few training examples. This paper aims to handle the challenge by adopting the key idea of "calibration first, translation later" and exploring the augmented global structure with locally-focused translation. Specifically, the proposed DCT-Net consists of three modules: a content adapter borrowing the powerful prior from source photos to calibrate the content distribution of target samples; a geometry expansion module using affine transformations to release spatially semantic constraints; and a texture translation module leveraging samples produced by the calibrated distribution to learn a fine-grained conversion. Experimental results demonstrate the proposed method's superiority over the state of the art in head stylization and its effectiveness on full image translation with adaptive deformations.

cs.CV↗

arXiv2022

Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation

Shape information is a strong and valuable prior in segmenting organs in medical images. However, most current deep learning based segmentation algorithms have not taken shape information into consideration, which can lead to bias towards texture. We aim at modeling shape explicitly and using it to help medical image segmentation. Previous methods proposed Variational Autoencoder (VAE) based models to learn the distribution of shape for a particular organ and used it to automatically evaluate the quality of a segmentation prediction by fitting it into the learned shape distribution. Based on which we aim at incorporating VAE into current segmentation pipelines. Specifically, we propose a new unsupervised domain adaptation pipeline based on a pseudo loss and a VAE reconstruction loss under a teacher-student learning paradigm. Both losses are optimized simultaneously and, in return, boost the segmentation task performance. Extensive experiments on three public Pancreas segmentation datasets as well as two in-house Pancreas segmentation datasets show consistent improvements with at least 2.8 points gain in the Dice score, demonstrating the effectiveness of our method in challenging unsupervised domain adaptation scenarios for medical image segmentation. We hope this work will advance shape analysis and geometric learning in medical imaging.

cs.CV↗

arXiv2022

Tracking the nematicity in cuprate superconductors: a resistivity study under uniaxial pressure

Overshadowing the superconducting dome in hole-doped cuprates, the pseudogap state is still one of the mysteries that no consensus can be achieved. It has been suggested that the rotational symmetry is broken in this state and may result in a nematic phase transition, whose temperature seems to coincide with the onset temperature of the pseudogap state $T^*$ around optimal doping level, raising the question whether the pseudogap results from the establishment of the nematic order. Here we report results of resistivity measurements under uniaxial pressure on several hole-doped cuprates, where the normalized slope of the elastoresistivity $ζ$ can be obtained as illustrated in iron-based superconductors. The temperature dependence of $ζ$ along particular lattice axis exhibits kink feature at $T_{k}$ and shows Curie-Weiss-like behavior above it, which may suggest a spontaneous nematic transition. While $T_{k}$ seems to be the same as $T^*$ around the optimal doping and in the overdoped region, they become very different in underdoped La$_{2-x}$Sr$_{x}$CuO$_4$. Our results suggest that the nematic order, if indeed existing, is an electronic phase within the pseudogap state.

cond-mat.supr-con↗

arXiv2022

Gappability Index for Quantum Many-Body Systems

We propose an index $\mathcal{I}_G$ which characterizes the degree of ingappability, namely the difficulty to induce a unique ground state with a nonvanishing excitation gap, in the presence of a symmetry $G$. $\mathcal{I}_G$ represents the dimension of the subspace of ambient uniquely-gapped in the entire $G$-invariant "theory space". The celebrated Lieb-Schultz-Mattis theorem corresponds, in our formulation, to the case $\mathcal{I}_G=0$ (completely ingappable) for the symmetry $G$ including the lattice translation symmetry. We illustrate the usefulness of the index by discussing the phase diagram of spin-$1/2$ antiferromagnets in various dimensions, which do not necessarily have the translation symmetry.

cond-mat.str-el↗

arXiv2022

On Private Online Convex Optimization: Optimal Algorithms in $\ell_p$-Geometry and High Dimensional Contextual Bandits

Differentially private (DP) stochastic convex optimization (SCO) is ubiquitous in trustworthy machine learning algorithm design. This paper studies the DP-SCO problem with streaming data sampled from a distribution and arrives sequentially. We also consider the continual release model where parameters related to private information are updated and released upon each new data, often known as the online algorithms. Despite that numerous algorithms have been developed to achieve the optimal excess risks in different $\ell_p$ norm geometries, yet none of the existing ones can be adapted to the streaming and continual release setting. To address such a challenge as the online convex optimization with privacy protection, we propose a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction to update and reveal the parameters upon each data. Combined with the adaptive differential privacy analysis, our online algorithm achieves in linear time the optimal excess risk when $1<p\leq 2$ and the state-of-the-art excess risk meeting the non-private lower ones when $2<p\leq\infty$. Our algorithm can also be extended to the case $p=1$ to achieve nearly dimension-independent excess risk. While previous variance reduction results on recursive gradient have theoretical guarantee only in the independent and identically distributed sample setting, we establish such a guarantee in a non-stationary setting. To demonstrate the virtues of our method, we design the first DP algorithm for high-dimensional generalized linear bandits with logarithmic regret. Comparative experiments with a variety of DP-SCO and DP-Bandit algorithms exhibit the efficacy and utility of the proposed algorithms.

cs.LG↗