arXiv subjects

Yong Liu

Publications and source records attributed to Yong Liu.

At least 757 records · Page 42Linked to original sources

arXiv2022

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this paper, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, two observability-aware modules are leveraged: (i) The information-theoretic data selection policy selects only the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. (ii) The observability-aware state update mechanism in nonlinear least-squares optimization updates only the identifiable directions in the state space with truncated singular value decomposition (TSVD), which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments. We also open source our codebase to benefit the research community: {\url{https://github.com/APRIL-ZJU/OA-LICalib}}.

cs.RO↗

arXiv2022

Stable Ferromagnetism and High Curie Temperature in VGe$_2$N$_4$

The discovery of monolayer MA$_2$Z$_4$ (M = transition metals; A = IVA elements; Z = VA elements, Science 369, 2020, 670-674) family has led another advance for facilitating and harnessing magnetism in low-dimensional materials. However, only Cr and V based MA$_2$N$_4$ compounds exhibit intrinsic magnetism yet with unsatisfied magnetic ordering temperature. Herein, we identify a stable ferromagnetic number of this family, i.e., VGe$_2$Z$_4$ monolayer, by means of first-principles calculations. It is found that the magnetic configuration sustains under both compression and tensile uniaxial in-plane strain, and the former can act as a positive modulator to enhance magnetic ordering temperature (Tc). Electronic structure calculations reveal a large band gap in the spin down channel while band-gapless in the spin up channel, an impressive near-half-metallic character, which is a favorable candidate for spintronic device.

cond-mat.mtrl-sci↗

arXiv2022

Ridgeless Regression with Random Features

Recent theoretical studies illustrated that kernel ridgeless regression can guarantee good generalization ability without an explicit regularization. In this paper, we investigate the statistical properties of ridgeless regression with random features and stochastic gradient descent. We explore the effect of factors in the stochastic gradient and random features, respectively. Specifically, random features error exhibits the double-descent curve. Motivated by the theoretical findings, we propose a tunable kernel algorithm that optimizes the spectral density of kernel during training. Our work bridges the interpolation theory and practical algorithm.

cs.LG↗

arXiv2022

Stability and Generalization of Differentially Private Minimax Problems

In the field of machine learning, many problems can be formulated as the minimax problem, including reinforcement learning, generative adversarial networks, to just name a few. So the minimax problem has attracted a huge amount of attentions from researchers in recent decades. However, there is relatively little work on studying the privacy of the general minimax paradigm. In this paper, we focus on the privacy of the general minimax setting, combining differential privacy together with minimax optimization paradigm. Besides, via algorithmic stability theory, we theoretically analyze the high probability generalization performance of the differentially private minimax algorithm under the strongly-convex-strongly-concave condition. To the best of our knowledge, this is the first time to analyze the generalization performance of general minimax paradigm, taking differential privacy into account.

cs.LG↗

arXiv2022

Sharper Utility Bounds for Differentially Private Models

In this paper, by introducing Generalized Bernstein condition, we propose the first $\mathcal{O}\big(\frac{\sqrt{p}}{nε}\big)$ high probability excess population risk bound for differentially private algorithms under the assumptions $G$-Lipschitz, $L$-smooth, and Polyak-Łojasiewicz condition, based on gradient perturbation method. If we replace the properties $G$-Lipschitz and $L$-smooth by $α$-H{ö}lder smoothness (which can be used in non-smooth setting), the high probability bound comes to $\mathcal{O}\big(n^{-\fracα{1+2α}}\big)$ w.r.t $n$, which cannot achieve $\mathcal{O}\left(1/n\right)$ when $α\in(0,1]$. To solve this problem, we propose a variant of gradient perturbation method, \textbf{max$\{1,g\}$-Normalized Gradient Perturbation} (m-NGP). We further show that by normalization, the high probability excess population risk bound under assumptions $α$-H{ö}lder smooth and Polyak-Łojasiewicz condition can achieve $\mathcal{O}\big(\frac{\sqrt{p}}{nε}\big)$, which is the first $\mathcal{O}\left(1/n\right)$ high probability excess population risk bound w.r.t $n$ for differentially private algorithms under non-smooth conditions. Moreover, we evaluate the performance of the new proposed algorithm m-NGP, the experimental results show that m-NGP improves the performance of the differentially private model over real datasets. It demonstrates that m-NGP improves the utility bound and the accuracy of the DP model on real datasets simultaneously.

cs.LG↗

arXiv2022

Two-Dimensional Ferromagnetic Half-Metallic Janus V2AsP Monolayer

Two-dimensional (2D) ferromagnetic materials present promising candidates for spintronic devices, and the half-metallic materials with 100% spin polarization at Fermi energy level are highly desired for many spin-based devices. 2D Janus materials have attracted great attention in recent years due to their excellent properties induced by breaking the symmetry. Here, using the density functional theory, we report that the Janus V2AsP monolayer demonstrates a charming ferromagnetic half-metallic feature. It is dynamically stable in view of the absence of imaginary frequency phonon. The half-metallic gap is about 0.38 eV and the spin splitting of about 1.34eV for the V2AsP monolayer. Interestingly, a tensile strain of 4.9% can induce it to undergo a phase transition from ferromagnetic to anti-ferromagnetic state. Moreover, the Curie temperature (Tc) enhances with the increase of compressive strain. All there appealing properties make the half-metallic Janus V2AsP monolayer a promising material for 2D spintronic applications.

cond-mat.mtrl-sci↗

arXiv2022

CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow

Optical flow estimation aims to find the 2D motion field by identifying corresponding pixels between two images. Despite the tremendous progress of deep learning-based optical flow methods, it remains a challenge to accurately estimate large displacements with motion blur. This is mainly because the correlation volume, the basis of pixel matching, is computed as the dot product of the convolutional features of the two images. The locality of convolutional features makes the computed correlations susceptible to various noises. On large displacements with motion blur, noisy correlations could cause severe errors in the estimated flow. To overcome this challenge, we propose a new architecture "CRoss-Attentional Flow Transformer" (CRAFT), aiming to revitalize the correlation volume computation. In CRAFT, a Semantic Smoothing Transformer layer transforms the features of one frame, making them more global and semantically stable. In addition, the dot-product correlations are replaced with transformer Cross-Frame Attention. This layer filters out feature noises through the Query and Key projections, and computes more accurate correlations. On Sintel (Final) and KITTI (foreground) benchmarks, CRAFT has achieved new state-of-the-art performance. Moreover, to test the robustness of different models on large motions, we designed an image shifting attack that shifts input images to generate large artificial motions. Under this attack, CRAFT performs much more robustly than two representative methods, RAFT and GMA. The code of CRAFT is is available at https://github.com/askerlee/craft.

cs.CV↗

arXiv2022

Region-Aware Face Swapping

This paper presents a novel Region-Aware Face Swapping (RAFSwap) network to achieve identity-consistent harmonious high-resolution face generation in a local-global manner: \textbf{1)} Local Facial Region-Aware (FRA) branch augments local identity-relevant features by introducing the Transformer to effectively model misaligned cross-scale semantic interaction. \textbf{2)} Global Source Feature-Adaptive (SFA) branch further complements global identity-relevant cues for generating identity-consistent swapped faces. Besides, we propose a \textit{Face Mask Predictor} (FMP) module incorporated with StyleGAN2 to predict identity-relevant soft facial masks in an unsupervised manner that is more practical for generating harmonious high-resolution faces. Abundant experiments qualitatively and quantitatively demonstrate the superiority of our method for generating more identity-consistent high-resolution swapped faces over SOTA methods, \eg, obtaining 96.70 ID retrieval that outperforms SOTA MegaFS by 5.87$\uparrow$.

cs.CV↗

arXiv2022

Towards Efficient and Scalable Sharpness-Aware Minimization

Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training large-scale models such as vision transformers. However, the update rule of SAM requires two sequential (non-parallelizable) gradient computations at each step, which can double the computational overhead. In this paper, we propose a novel algorithm LookSAM - that only periodically calculates the inner gradient ascent, to significantly reduce the additional training cost of SAM. The empirical results illustrate that LookSAM achieves similar accuracy gains to SAM while being tremendously faster - it enjoys comparable computational complexity with first-order optimizers such as SGD or Adam. To further evaluate the performance and scalability of LookSAM, we incorporate a layer-wise modification and perform experiments in the large-batch training scenario, which is more prone to converge to sharp local minima. We are the first to successfully scale up the batch size when training Vision Transformers (ViTs). With a 64k batch size, we are able to train ViTs from scratch in minutes while maintaining competitive performance.

cs.LG↗

arXiv2022

Topological semimetal phases in a family of monolayer X3YZ6 (X=Nb,Ta, Y=Si,Ge,Sn, Z=S,Se,Te) with abundant nodal lines and nodes

The electronic and topological properties of single-layer X3YZ6 (X=Nb,Ta, Y=Si,Ge,Sn, Z=S,Se,Te) materials have been studied with the aid of first principles calculations. This kind of materials belong to topological semimetals (TMs) with abundant nodal lines and nodes. Considering their similar properties, we focus on the analysis of Ta3SnTe6 and Ta3SiSe6. The present of spin-orbit coupling (SOC) leads to the transition from type-I nodal lines to Dirac points as well as the disappear of type-II Dirac points. The three-dimensional (3D) band diagrams reproduce vividly the characteristics of nodes and nodal lines. The appearance of the flat bands in (110) edge states further confirm their nontrivial topological properties. We also explore the relationship among different nodal lines (nodes), crystal symmetry and SOC. The type-I nodal lines are protected by Mz and My symmetry in the absent of SOC. Symmetry breaking leads to band splitting even in the presence of SOC. The single-layer X3YZ6 can be used as candidates for two-dimensional (2D) TMs and provide a platform for further study of interesting physical phenomena.

cond-mat.mtrl-sci↗

arXiv2022

Guide Local Feature Matching by Overlap Estimation

Local image feature matching under large appearance, viewpoint, and distance changes is challenging yet important. Conventional methods detect and match tentative local features across the whole images, with heuristic consistency checks to guarantee reliable matches. In this paper, we introduce a novel Overlap Estimation method conditioned on image pairs with TRansformer, named OETR, to constrain local feature matching in the commonly visible region. OETR performs overlap estimation in a two-step process of feature correlation and then overlap regression. As a preprocessing module, OETR can be plugged into any existing local feature detection and matching pipeline, to mitigate potential view angle or scale variance. Intensive experiments show that OETR can boost state-of-the-art local feature matching performance substantially, especially for image pairs with small shared regions. The code will be publicly available at https://github.com/AbyssGaze/OETR.

cs.CV↗

arXiv2022

A Deep Reinforcement Learning based Approach for NOMA-based Random Access Network with Truncated Channel Inversion Power Control

As a main use case of 5G and Beyond wireless network, the ever-increasing machine type communications (MTC) devices pose critical challenges over MTC network in recent years. It is imperative to support massive MTC devices with limited resources. To this end, Non-orthogonal multiple access (NOMA) based random access network has been deemed as a prospective candidate for MTC network. In this paper, we propose a deep reinforcement learning (RL) based approach for NOMA-based random access network with truncated channel inversion power control. Specifically, each MTC device randomly selects a pre-defined power level with a certain probability for data transmission. Devices are using channel inversion power control yet subject to the upper bound of the transmission power. Due to the stochastic feature of the channel fading and the limited transmission power, devices with different achievable power levels have been categorized as different types of devices. In order to achieve high throughput with considering the fairness between all devices, two objective functions are formulated. One is to maximize the minimum long-term expected throughput of all MTC devices, the other is to maximize the geometric mean of the long-term expected throughput for all MTC devices. A Policy based deep reinforcement learning approach is further applied to tune the transmission probabilities of each device to solve the formulated optimization problems. Extensive simulations are conducted to show the merits of our proposed approach.

cs.NI↗

arXiv2022

Thoughts on the Consistency between Ricci Flow and Neural Network Behavior

The Ricci flow is a partial differential equation for evolving the metric in a Riemannian manifold to make it more regular. On the other hand, neural networks seem to have similar geometric behavior for specific tasks. In this paper, we construct the linearly nearly Euclidean manifold as a background to observe the evolution of Ricci flow and the training of neural networks. Under the Ricci-DeTurck flow, we prove the dynamical stability and convergence of the linearly nearly Euclidean metric for an $L^2$-Norm perturbation. In practice, from the information geometry and mirror descent points of view, we give the steepest descent gradient flow for neural networks on the linearly nearly Euclidean manifold. During the training process of the neural network, we observe that its metric will also regularly converge to the linearly nearly Euclidean metric, which is consistent with the convergent behavior of linearly nearly Euclidean metrics under the Ricci-DeTurck flow.

cs.LG↗

arXiv2022

Dynamically Stable Poincaré Embeddings for Neural Manifolds

In a Riemannian manifold, the Ricci flow is a partial differential equation for evolving the metric to become more regular. We hope that topological structures from such metrics may be used to assist in the tasks of machine learning. However, this part of the work is still missing. In this paper, we propose Ricci flow assisted Eucl2Hyp2Eucl neural networks that bridge this gap between the Ricci flow and deep neural networks by mapping neural manifolds from the Euclidean space to the dynamically stable Poincaré ball and then back to the Euclidean space. As a result, we prove that, if initial metrics have an $L^2$-norm perturbation which deviates from the Hyperbolic metric on the Poincaré ball, the scaled Ricci-DeTurck flow of such metrics smoothly and exponentially converges to the Hyperbolic metric. Specifically, the role of the Ricci flow is to serve as naturally evolving to the stable Poincaré ball. For such dynamically stable neural manifolds under the Ricci flow, the convergence of neural networks embedded with such manifolds is not susceptible to perturbations. And we show that Ricci flow assisted Eucl2Hyp2Eucl neural networks outperform with their all Euclidean counterparts on image classification tasks.

cs.LG↗

arXiv2022

A Survey of Visual Sensory Anomaly Detection

Visual sensory anomaly detection (AD) is an essential problem in computer vision, which is gaining momentum recently thanks to the development of AI for good. Compared with semantic anomaly detection which detects anomaly at the label level (semantic shift), visual sensory AD detects the abnormal part of the sample (covariate shift). However, no thorough review has been provided to summarize this area for the computer vision community. In this survey, we are the first one to provide a comprehensive review of visual sensory AD and category into three levels according to the form of anomalies. Furthermore, we classify each kind of anomaly according to the level of supervision. Finally, we summarize the challenges and provide open directions for this community. All resources are available at https://github.com/M-3LAB/awesome-visual-sensory-anomaly-detection.

cs.CV↗

arXiv2022

First-principles study the structural, magnetic, optical properties and doping effect in chromium arsenide

We systematically study the pristine and doped chromium arsenide (CrAs) in six different crystal structures to investigate the structural, magnetic, and optical properties for real applications by first-principles calculations. First, we found that the ground-state structure is an orthorhombic MnP-type structure with antiferromagnetic spin order. The rocksalt structure is an low-energy metastable phase and a ferromagnetic metal with high spin polarization at the Fermi level. Secondly, the NiAs structure and MnP structure have a higher absorption coefficient than other structures in the infrared region and ultraviolet region, respectively. In the visible light region, the wurtzite and zincblende structures are more transparent than other structures. At last, we found that Ti substitution of Cr and Te substitution of As can lead to a phase transition in ground-state structure and ground-state magnetic order, respectively. These results can promote the application of the CrAs system into spintronics.

cond-mat.mtrl-sci↗

arXiv2022

Concurrent Adversarial Learning for Large-Batch Training

Large-batch training has become a commonly used technique when training neural networks with a large number of GPU/TPU processors. As batch size increases, stochastic optimizers tend to converge to sharp local minima, leading to degraded test performance. Current methods usually use extensive data augmentation to increase the batch size, but we found the performance gain with data augmentation decreases as batch size increases, and data augmentation will become insufficient after certain point. In this paper, we propose to use adversarial learning to increase the batch size in large-batch training. Despite being a natural choice for smoothing the decision surface and biasing towards a flat region, adversarial learning has not been successfully applied in large-batch training since it requires at least two sequential gradient computations at each step, which will at least double the running time compared with vanilla training even with a large number of processors. To overcome this issue, we propose a novel Concurrent Adversarial Learning (ConAdv) method that decouple the sequential gradient computations in adversarial learning by utilizing staled parameters. Experimental results demonstrate that ConAdv can successfully increase the batch size on ResNet-50 training on ImageNet while maintaining high accuracy. In particular, we show ConAdv along can achieve 75.3\% top-1 accuracy on ImageNet ResNet-50 training with 96K batch size, and the accuracy can be further improved to 76.2\% when combining ConAdv with data augmentation. This is the first work successfully scales ResNet-50 training batch size to 96K.

cs.LG↗

arXiv2022

Lightweight Object-level Topological Semantic Mapping and Long-term Global Localization based on Graph Matching

Mapping and localization are two essential tasks for mobile robots in real-world applications. However, largescale and dynamic scenes challenge the accuracy and robustness of most current mature solutions. This situation becomes even worse when computational resources are limited. In this paper, we present a novel lightweight object-level mapping and localization method with high accuracy and robustness. Different from previous methods, our method does not need a prior constructed precise geometric map, which greatly releases the storage burden, especially for large-scale navigation. We use object-level features with both semantic and geometric information to model landmarks in the environment. Particularly, a learning topological primitive is first proposed to efficiently obtain and organize the object-level landmarks. On the basis of this, we use a robot-centric mapping framework to represent the environment as a semantic topology graph and relax the burden of maintaining global consistency at the same time. Besides, a hierarchical memory management mechanism is introduced to improve the efficiency of online mapping with limited computational resources. Based on the proposed map, the robust localization is achieved by constructing a novel local semantic scene graph descriptor, and performing multi-constraint graph matching to compare scene similarity. Finally, we test our method on a low-cost embedded platform to demonstrate its advantages. Experimental results on a large scale and multi-session real-world environment show that the proposed method outperforms the state of arts in terms of lightweight and robustness.

cs.RO↗