arXiv subjects

Jian Zhang

Publications and source records attributed to Jian Zhang.

At least 469 records · Page 26Linked to original sources

arXiv2023

$^{75}$As NMR study of the antiferromagnetic Kondo lattice compound CeNiAsO

We revisit the magnetic properties of the antiferromagnetic Kondo lattice CeNiAsO by $^{75}$As nuclear magnetic resonance measurements. Our results confirm two successive antiferromagnetic transitions of Ce moments at $T_{N1}=9.0(3)$ K and $T_{N2}=7.0(3)$ K. Incommensurate and commensurate antiferromagnetic orders are suggested for $T_{N2}<T<T_{N1}$ and $T<T_{N2}$ respectively, consistent with previous neutron and muon experiments. A Knight shift anomaly, characterized by the failure of $K(T)-χ(T)$ scaling, is observed below $T^*\sim15$ K, which gives a measure of the onset of coherent $c-f$ correlations. This energy scale is further confirmed by the spin-lattice relaxation rate ($1/T_1$). The analysis of spin dynamics also reveals a quasi-two-dimensional character of spin fluctuations in CeNiAsO. This work paves the way for further $^{75}$As nuclear magnetic resonance studies under pressure.

cond-mat.str-el↗

arXiv2022

Position Embedding Needs an Independent Layer Normalization

The Position Embedding (PE) is critical for Vision Transformers (VTs) due to the permutation-invariance of self-attention operation. By analyzing the input and output of each encoder layer in VTs using reparameterization and visualization, we find that the default PE joining method (simply adding the PE and patch embedding together) operates the same affine transformation to token embedding and PE, which limits the expressiveness of PE and hence constrains the performance of VTs. To overcome this limitation, we propose a simple, effective, and robust method. Specifically, we provide two independent layer normalizations for token embeddings and PE for each layer, and add them together as the input of each layer's Muti-Head Self-Attention module. Since the method allows the model to adaptively adjust the information of PE for different layers, we name it as Layer-adaptive Position Embedding, abbreviated as LaPE. Extensive experiments demonstrate that LaPE can improve various VTs with different types of PE and make VTs robust to PE types. For example, LaPE improves 0.94% accuracy for ViT-Lite on Cifar10, 0.98% for CCT on Cifar100, and 1.72% for DeiT on ImageNet-1K, which is remarkable considering the negligible extra parameters, memory and computational cost brought by LaPE. The code is publicly available at https://github.com/Ingrid725/LaPE.

cs.CV↗

arXiv2022

Latent Evolution Model for Change Point Detection in Time-varying Networks

Graph-based change point detection (CPD) play an irreplaceable role in discovering anomalous graphs in the time-varying network. While several techniques have been proposed to detect change points by identifying whether there is a significant difference between the target network and successive previous ones, they neglect the natural evolution of the network. In practice, real-world graphs such as social networks, traffic networks, and rating networks are constantly evolving over time. Considering this problem, we treat the problem as a prediction task and propose a novel CPD method for dynamic graphs via a latent evolution model. Our method focuses on learning the low-dimensional representations of networks and capturing the evolving patterns of these learned latent representations simultaneously. After having the evolving patterns, a prediction of the target network can be achieved. Then, we can detect the change points by comparing the prediction and the actual network by leveraging a trade-off strategy, which balances the importance between the prediction network and the normal graph pattern extracted from previous networks. Intensive experiments conducted on both synthetic and real-world datasets show the effectiveness and superiority of our model.

cs.SI↗

arXiv2022

Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model

Most existing Image Restoration (IR) models are task-specific, which can not be generalized to different degradation operators. In this work, we propose the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring. DDNM only needs a pre-trained off-the-shelf diffusion model as the generative prior, without any extra training or network modifications. By refining only the null-space contents during the reverse diffusion process, we can yield diverse results satisfying both data consistency and realness. We further propose an enhanced and robust version, dubbed DDNM+, to support noisy restoration and improve restoration quality for hard tasks. Our experiments on several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot IR methods. We also demonstrate that DDNM+ can solve complex real-world applications, e.g., old photo restoration.

cs.CV↗

arXiv2022

GAN Prior based Null-Space Learning for Consistent Super-Resolution

Consistency and realness have always been the two critical issues of image super-resolution. While the realness has been dramatically improved with the use of GAN prior, the state-of-the-art methods still suffer inconsistencies in local structures and colors (e.g., tooth and eyes). In this paper, we show that these inconsistencies can be analytically eliminated by learning only the null-space component while fixing the range-space part. Further, we design a pooling-based decomposition (PD), a universal range-null space decomposition for super-resolution tasks, which is concise, fast, and parameter-free. PD can be easily applied to state-of-the-art GAN Prior based SR methods to eliminate their inconsistencies, neither compromising the realness nor bringing extra parameters or computational costs. Besides, our ablation studies reveal that PD can replace pixel-wise losses for training and achieve better generalization performance when facing unseen downsamplings or even real-world degradation. Experiments show that the use of PD refreshes state-of-the-art SR performance and speeds up the convergence of training up to 2~10 times.

cs.CV↗

arXiv2022

Complementary Labels Learning with Augmented Classes

Complementary Labels Learning (CLL) arises in many real-world tasks such as private questions classification and online learning, which aims to alleviate the annotation cost compared with standard supervised learning. Unfortunately, most previous CLL algorithms were in a stable environment rather than an open and dynamic scenarios, where data collected from unseen augmented classes in the training process might emerge in the testing phase. In this paper, we propose a novel problem setting called Complementary Labels Learning with Augmented Classes (CLLAC), which brings the challenge that classifiers trained by complementary labels should not only be able to classify the instances from observed classes accurately, but also recognize the instance from the Augmented Classes in the testing phase. Specifically, by using unlabeled data, we propose an unbiased estimator of classification risk for CLLAC, which is guaranteed to be provably consistent. Moreover, we provide generalization error bound for proposed method which shows that the optimal parametric convergence rate is achieved for estimation error. Finally, the experimental results on several benchmark datasets verify the effectiveness of the proposed method.

cs.LG↗

arXiv2022

Masked Vision-Language Transformers for Scene Text Recognition

Scene text recognition (STR) enables computers to recognize and read the text in various real-world scenes. Recent STR models benefit from taking linguistic information in addition to visual cues into consideration. We propose a novel Masked Vision-Language Transformers (MVLT) to capture both the explicit and the implicit linguistic information. Our encoder is a Vision Transformer, and our decoder is a multi-modal Transformer. MVLT is trained in two stages: in the first stage, we design a STR-tailored pretraining method based on a masking strategy; in the second stage, we fine-tune our model and adopt an iterative correction method to improve the performance. MVLT attains superior results compared to state-of-the-art STR models on several benchmarks. Our code and model are available at https://github.com/onealwj/MVLT.

cs.CV↗

arXiv2022

Tunable quantum dots from atomically precise graphene nanoribbons using a multi-gate architecture

Atomically precise graphene nanoribbons (GNRs) are increasingly attracting interest due to their largely modifiable electronic properties, which can be tailored by controlling their width and edge structure during chemical synthesis. In recent years, the exploitation of GNR properties for electronic devices has focused on GNR integration into field-effect-transistor (FET) geometries. However, such FET devices have limited electrostatic tunability due to the presence of a single gate. Here, we report on the device integration of 9-atom wide armchair graphene nanoribbons (9-AGNRs) into a multi-gate FET geometry, consisting of an ultra-narrow finger gate and two side gates. We use high-resolution electron-beam lithography (EBL) for defining finger gates as narrow as 12 nm and combine them with graphene electrodes for contacting the GNRs. Low-temperature transport spectroscopy measurements reveal quantum dot (QD) behavior with rich Coulomb diamond patterns, suggesting that the GNRs form QDs that are connected both in series and in parallel. Moreover, we show that the additional gates enable differential tuning of the QDs in the nanojunction, providing the first step towards multi-gate control of GNR-based multi-dot systems.

cond-mat.mes-hall↗

arXiv2022

The observation of quantum fluctuations in a kagome Heisenberg antiferromagnet

The search for the experimental evidence of quantum spin liquid (QSL) states is critical but extremely challenging, as the quenched interaction randomness introduced by structural imperfection is usually inevitable in real materials. YCu$_3$(OH)$_{6.5}$Br$_{2.5}$ (YCOB) is a spin-1/2 kagome Heisenberg antiferromagnet (KHA) with strong coupling of $\langle J_1\rangle\sim$ 51 K but without conventional magnetic freezing down to 50 mK $\sim$ 0.001$\langle J_1\rangle$. Here, we report a Br nuclear magnetic resonance (NMR) study of the local spin susceptibility and dynamics on the single crystal of YCOB. The temperature dependence of NMR main-line shifts and broadening can be well understood within the frame of the KHA model with randomly distributed hexagons of alternate exchanges, compatible with the formation of a randomness-induced QSL state at low temperatures. The in-plane spin fluctuations as measured by the spin-lattice relaxation rates ($1/T_1$) exhibit a weak temperature dependence down to $T$ $\sim$ 0.03$\langle J_1\rangle$. Our results demonstrate that the majority of spins remain highly fluctuating at low temperatures despite the quenched disorder in YCOB.

cond-mat.str-el↗

arXiv2022

Multi-Agent Automated Machine Learning

In this paper, we propose multi-agent automated machine learning (MA2ML) with the aim to effectively handle joint optimization of modules in automated machine learning (AutoML). MA2ML takes each machine learning module, such as data augmentation (AUG), neural architecture search (NAS), or hyper-parameters (HPO), as an agent and the final performance as the reward, to formulate a multi-agent reinforcement learning problem. MA2ML explicitly assigns credit to each agent according to its marginal contribution to enhance cooperation among modules, and incorporates off-policy learning to improve search efficiency. Theoretically, MA2ML guarantees monotonic improvement of joint optimization. Extensive experiments show that MA2ML yields the state-of-the-art top-1 accuracy on ImageNet under constraints of computational cost, e.g., $79.7\%/80.5\%$ with FLOPs fewer than 600M/800M. Extensive ablation studies verify the benefits of credit assignment and off-policy learning of MA2ML.

cs.LG↗

arXiv2022

Overlap-guided Gaussian Mixture Models for Point Cloud Registration

Probabilistic 3D point cloud registration methods have shown competitive performance in overcoming noise, outliers, and density variations. However, registering point cloud pairs in the case of partial overlap is still a challenge. This paper proposes a novel overlap-guided probabilistic registration approach that computes the optimal transformation from matched Gaussian Mixture Model (GMM) parameters. We reformulate the registration problem as the problem of aligning two Gaussian mixtures such that a statistical discrepancy measure between the two corresponding mixtures is minimized. We introduce a Transformer-based detection module to detect overlapping regions, and represent the input point clouds using GMMs by guiding their alignment through overlap scores computed by this detection module. Experiments show that our method achieves superior registration accuracy and efficiency than state-of-the-art methods when handling point clouds with partial overlap and different densities on synthetic and real-world datasets. https://github.com/gfmei/ogmm

cs.CV↗

arXiv2022

COTReg:Coupled Optimal Transport based Point Cloud Registration

Generating a set of high-quality correspondences or matches is one of the most critical steps in point cloud registration. This paper proposes a learning framework COTReg by jointly considering the pointwise and structural matchings to predict correspondences of 3D point cloud registration. Specifically, we transform the two matchings into a Wasserstein distance-based and a Gromov-Wasserstein distance-based optimizations, respectively. Thus the task of establishing the correspondences can be naturally reshaped to a coupled optimal transport problem. Furthermore, we design a network to predict the confidence score of being an inlier for each point of the point clouds, which provides the overlap region information to generate correspondences. Our correspondence prediction pipeline can be easily integrated into either learning-based features like FCGF or traditional descriptors like FPFH. We conducted comprehensive experiments on 3DMatch, KITTI, 3DCSR, and ModelNet40 benchmarks, showing the state-of-art performance of the proposed method.

cs.CV↗

arXiv2022

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach

This paper proposes an eco-driving framework for electric connected vehicles (CVs) based on reinforcement learning (RL) to improve vehicle energy efficiency at signalized intersections. The vehicle agent is specified by integrating the model-based car-following policy, lane-changing policy, and the RL policy, to ensure safe operation of a CV. Subsequently, a Markov Decision Process (MDP) is formulated, which enables the vehicle to perform longitudinal control and lateral decisions, jointly optimizing the car-following and lane-changing behaviors of the CVs in the vicinity of intersections. Then, the hybrid action space is parameterized as a hierarchical structure and thereby trains the agents with two-dimensional motion patterns in a dynamic traffic environment. Finally, our proposed methods are evaluated in SUMO software from both a single-vehicle-based perspective and a flow-based perspective. The results show that our strategy can significantly reduce energy consumption by learning proper action schemes without any interruption of other human-driven vehicles (HDVs).

cs.RO↗

arXiv2022

Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understanding

Unsupervised learning on 3D point clouds has undergone a rapid evolution, especially thanks to data augmentation-based contrastive methods. However, data augmentation is not ideal as it requires a careful selection of the type of augmentations to perform, which in turn can affect the geometric and semantic information learned by the network during self-training. To overcome this issue, we propose an augmentation-free unsupervised approach for point clouds to learn transferable point-level features via soft clustering, named SoftClu. SoftClu assumes that the points belonging to a cluster should be close to each other in both geometric and feature spaces. This differs from typical contrastive learning, which builds similar representations for a whole point cloud and its augmented versions. We exploit the affiliation of points to their clusters as a proxy to enable self-training through a pseudo-label prediction task. Under the constraint that these pseudo-labels induce the equipartition of the point cloud, we cast SoftClu as an optimal transport problem. We formulate an unsupervised loss to minimize the standard cross-entropy between pseudo-labels and predicted labels. Experiments on downstream applications, such as 3D object classification, part segmentation, and semantic segmentation, show the effectiveness of our framework in outperforming state-of-the-art techniques.

cs.CV↗

arXiv2022

Towards Multimodal Multitask Scene Understanding Models for Indoor Mobile Agents

The perception system in personalized mobile agents requires developing indoor scene understanding models, which can understand 3D geometries, capture objectiveness, analyze human behaviors, etc. Nonetheless, this direction has not been well-explored in comparison with models for outdoor environments (e.g., the autonomous driving system that includes pedestrian prediction, car detection, traffic sign recognition, etc.). In this paper, we first discuss the main challenge: insufficient, or even no, labeled data for real-world indoor environments, and other challenges such as fusion between heterogeneous sources of information (e.g., RGB images and Lidar point clouds), modeling relationships between a diverse set of outputs (e.g., 3D object locations, depth estimation, and human poses), and computational efficiency. Then, we describe MMISM (Multi-modality input Multi-task output Indoor Scene understanding Model) to tackle the above challenges. MMISM considers RGB images as well as sparse Lidar points as inputs and 3D object detection, depth completion, human pose estimation, and semantic segmentation as output tasks. We show that MMISM performs on par or even better than single-task models; e.g., we improve the baseline 3D object detection results by 11.7% on the benchmark ARKitScenes dataset.

cs.CV↗

arXiv2022

Spatial-temporal Analysis for Automated Concrete Workability Estimation

Concrete workability measure is mostly determined based on subjective assessment of a certified assessor with visual inspections. The potential human error in measuring the workability and the resulting unnecessary adjustments for the workability is a major challenge faced by the construction industry, leading to significant costs, material waste and delay. In this paper, we try to apply computer vision techniques to observe the concrete mixing process and estimate the workability. Specifically, we collected the video data and then built three different deep neural networks for spatial-temporal regression. The pilot study demonstrates a practical application with computer vision techniques to estimate the concrete workability during the mixing process.

cs.CV↗

arXiv2022

Ultimately-scaled electrodes for contacting individual atomically-precise graphene nanoribbons

Bottom-up synthesized graphene nanoribbons (GNRs) are quantum materials that can be structured with atomic precision, providing unprecedented control over their physical properties. Accessing the intrinsic functionality of GNRs for quantum technology applications requires the manipulation of single charges, spins, or photons at the level of an individual GNR. However, experimentally, contacting individual GNRs remains challenging due to their nanometer-sized width and length as well as their high density on the metallic growth substrate. Here, we demonstrate the contacting and electrical characterization of individual GNRs in a multi-gate device architecture using single-walled carbon nanotubes (SWNTs) as ultimately-scaled electrodes. The GNR-SWNT devices exhibit well-defined quantum transport phenomena, including Coulomb blockade, excited states, and Franck-Condon blockade, all characteristics pointing towards the contacting of an individual GNR. Combined with the multi-gate architecture, this contacting method opens a road for the integration of GNRs in quantum devices to exploit their topologically trivial and non-trivial nature.

cond-mat.mes-hall↗

arXiv2022

Exploring fundamental laws of classical mechanics via predicting the orbits of planets based on neural networks

Neural networks have provided powerful approaches to solve various scientific problems. Many of them are even difficult for human experts who are good at accessing the physical laws from experimental data. We investigate whether neural networks can assist us in exploring the fundamental laws of classical mechanics from data of planetary motion. Firstly, we predict the orbits of planets in the geocentric system using the gate recurrent unit, one of the common neural networks. We find that the precision of the prediction is obviously improved when the information of the Sun is included in the training set. This result implies that the Sun is particularly important in the geocentric system without any prior knowledge, which inspires us to gain Copernicus' heliocentric theory. Secondly, we turn to the heliocentric system and make successfully mutual predictions between the position and velocity of planets. We hold that the successful prediction is due to the existence of enough conserved quantities (such as conservations of mechanical energy and angular momentum) in the system. Our research provides a new way to explore the existence of conserved quantities in mechanics system based on neural networks.

astro-ph.EP↗