arXiv subjects

Xiang Zhang

Publications and source records attributed to Xiang Zhang.

At least 379 records · Page 21Linked to original sources

arXiv2022

Time-varying Graph Learning Under Structured Temporal Priors

This paper endeavors to learn time-varying graphs by using structured temporal priors that assume underlying relations between arbitrary two graphs in the graph sequence. Different from many existing chain structure based methods in which the priors like temporal homogeneity can only describe the variations of two consecutive graphs, we propose a structure named \emph{temporal graph} to characterize the underlying real temporal relations. Under this framework, the chain structure is actually a special case of our temporal graph. We further proposed Alternating Direction Method of Multipliers (ADMM), a distributed algorithm, to solve the induced optimization problem. Numerical experiments demonstrate the superiorities of our method.

cs.LG↗

arXiv2022

Exploring Edge Disentanglement for Node Classification

Edges in real-world graphs are typically formed by a variety of factors and carry diverse relation semantics. For example, connections in a social network could indicate friendship, being colleagues, or living in the same neighborhood. However, these latent factors are usually concealed behind mere edge existence due to the data collection and graph formation processes. Despite rapid developments in graph learning over these years, most models take a holistic approach and treat all edges as equal. One major difficulty in disentangling edges is the lack of explicit supervisions. In this work, with close examination of edge patterns, we propose three heuristics and design three corresponding pretext tasks to guide the automatic edge disentanglement. Concretely, these self-supervision tasks are enforced on a designed edge disentanglement module to be trained jointly with the downstream node classification task to encourage automatic edge disentanglement. Channels of the disentanglement module are expected to capture distinguishable relations and neighborhood interactions, and outputs from them are aggregated as node representations. The proposed DisGNN is easy to be incorporated with various neural architectures, and we conduct experiments on $6$ real-world datasets. Empirical results show that it can achieve significant performance gains.

cs.SI↗

arXiv2022

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Knowledge distillation (KD) methods compress large models into smaller students with manually-designed student architectures given pre-specified computational cost. This requires several trials to find a viable student, and further repeating the process for each student or computational budget change. We use Neural Architecture Search (NAS) to automatically distill several compressed students with variable cost from a large model. Current works train a single SuperLM consisting of millions of subnetworks with weight-sharing, resulting in interference between subnetworks of different sizes. Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e.g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training. Fully task-agnostic training and search allow students to be reused for fine-tuning on any downstream task. Experiments on GLUE benchmark against state-of-the-art KD and NAS methods demonstrate AutoDistil to outperform leading compression techniques with upto 2.7x reduction in computational cost and negligible loss in task performance.

cs.CL↗

arXiv2022

High throughput data-driven design of laser crystallized 2D MoS2 chemical sensors

High throughput characterization and processing techniques are becoming increasingly necessary to navigate multivariable, data-driven design challenges for sensors and electronic devices. For two-dimensional materials, device performance is highly dependent upon a vast array of material properties including number of layers, lattice strain, carrier concentration, defect density, and grain structure. In this work, laser-crystallization was used to locally pattern and transform hundreds of regions of amorphous MoS2 thin films into 2D 2H-MoS2. A high throughput Raman spectroscopy approach was subsequently used to assess the process-dependent structural and compositional variations for each illuminated region, yielding over 5500 distinct non-resonant, resonant, and polarized Raman spectra. The rapid generation of a comprehensive library of structural and compositional data elucidated important trends between structure-property-processing relationships involving laser-crystallized MoS2, including the relationships between grain size, grain orientation, and intrinsic strain. Moreover, extensive analysis of structure/property relationships allowed for intelligent design, and evaluation of major contributions to, device performance in MoS2 chemical sensors. In particular, it is found that sensor performance is strongly dependent on the orientation of the MoS2 grains relative to the crystal plane.

cond-mat.mtrl-sci↗

arXiv2022

Stability of Oxygenated Groups on Pristine and Defective Diamond Surfaces

The surface functionalization of diamond has been extensively studied through a variety of techniques, such as oxidation. Several oxygen groups have been correspondingly detected on the oxidized diamond, such as COC (ester), CO (ketonic), and COH (hydroxyl). However, the composition and relative concentration of these groups on diamond surfaces can be affected by the type of oxygenation treatment and the diamond surface quality. To investigate the stability of the oxygenated groups at specific diamond surfaces, we evaluated through fully atomistic reactive molecular mechanics (FARMM) simulations, using the ReaxFF force field, the formation energies of CO, COC, and COH groups on pristine and defective diamond surfaces (110), (111), and (311). According to our findings, the COH group has the lowest formation energy on a perfect (110) surface, while the COC is favored on a defective surface. As for the (111) surface, the COC group is the most stable for both pristine and defective surfaces. Similarly, COC group is also the most stable one on the defective/perfect (311) surface. In this way, our results suggest that if in a diamond film the (110) surface is the major exposed facet, the most adsorbed oxygen group could be either COH or COC, in which the COC would depend on the level of surface defects.

cond-mat.mtrl-sci↗

arXiv2022

BM-NAS: Bilevel Multimodal Neural Architecture Search

Deep neural networks (DNNs) have shown superior performances on various multimodal learning problems. However, it often requires huge efforts to adapt DNNs to individual multimodal tasks by manually engineering unimodal features and designing multimodal feature fusion strategies. This paper proposes Bilevel Multimodal Neural Architecture Search (BM-NAS) framework, which makes the architecture of multimodal fusion models fully searchable via a bilevel searching scheme. At the upper level, BM-NAS selects the inter/intra-modal feature pairs from the pretrained unimodal backbones. At the lower level, BM-NAS learns the fusion strategy for each feature pair, which is a combination of predefined primitive operations. The primitive operations are elaborately designed and they can be flexibly combined to accommodate various effective feature fusion modules such as multi-head attention (Transformer) and Attention on Attention (AoA). Experimental results on three multimodal tasks demonstrate the effectiveness and efficiency of the proposed BM-NAS framework. BM-NAS achieves competitive performances with much less search time and fewer model parameters in comparison with the existing generalized multimodal NAS methods.

cs.CV↗

arXiv2022

AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators

Contrastive learning has been widely applied to graph representation learning, where the view generators play a vital role in generating effective contrastive samples. Most of the existing contrastive learning methods employ pre-defined view generation methods, e.g., node drop or edge perturbation, which usually cannot adapt to input data or preserve the original semantic structures well. To address this issue, we propose a novel framework named Automated Graph Contrastive Learning (AutoGCL) in this paper. Specifically, AutoGCL employs a set of learnable graph view generators orchestrated by an auto augmentation strategy, where every graph view generator learns a probability distribution of graphs conditioned by the input. While the graph view generators in AutoGCL preserve the most representative structures of the original graph in generation of every contrastive sample, the auto augmentation learns policies to introduce adequate augmentation variances in the whole contrastive learning procedure. Furthermore, AutoGCL adopts a joint training strategy to train the learnable view generators, the graph encoder, and the classifier in an end-to-end manner, resulting in topological heterogeneity yet semantic similarity in the generation of contrastive samples. Extensive experiments on semi-supervised learning, unsupervised learning, and transfer learning demonstrate the superiority of our AutoGCL framework over the state-of-the-arts in graph contrastive learning. In addition, the visualization results further confirm that the learnable view generators can deliver more compact and semantically meaningful contrastive samples compared against the existing view generation methods.

cs.LG↗

arXiv2021

Anomalous Josephson Effect in Topological Insulator-Based Josephson Trijunction

We studied anomalous Josephson effect (AJE) in Josephson trijunctions fabricated on Bi$_2$Se$_3$, and found that the AJE in T-shaped trijunctions significantly alters the Majorana phase diagram of the trijunctions, when an in-plane magnetic field is applied parallel to two of the three single junctions. Such a phenomenon in topological insulator-based Josephson trijunction provides unambiguous evidence for the existence of AJE in the system, and might provide an additional knob for controlling the Majorana bound states in the Fu-Kane scheme of topological quantum computation.

cond-mat.supr-con↗

arXiv2021

Controlling Neural Networks with Rule Representations

We propose a novel training method that integrates rules into deep learning, in a way the strengths of the rules are controllable at inference. Deep Neural Networks with Controllable Rule Representations (DeepCTRL) incorporates a rule encoder into the model coupled with a rule-based objective, enabling a shared representation for decision making. DeepCTRL is agnostic to data type and model architecture. It can be applied to any kind of rule defined for inputs and outputs. The key aspect of DeepCTRL is that it does not require retraining to adapt the rule strength -- at inference, the user can adjust it based on the desired operation point on accuracy vs. rule verification ratio. In real-world domains where incorporating rules is critical -- such as Physics, Retail and Healthcare -- we show the effectiveness of DeepCTRL in teaching rules for deep learning. DeepCTRL improves the trust and reliability of the trained models by significantly increasing their rule verification ratio, while also providing accuracy gains at downstream tasks. Additionally, DeepCTRL enables novel use cases such as hypothesis testing of the rules on data samples, and unsupervised adaptation based on shared rules between datasets.

cs.LG↗

arXiv2021

Scalable synthesis of 2D van der Waals superlattices

Heterostructure materials form the basis of much of modern electronics, from transistors to lasers and light-emitting diodes. Recent years have seen a renewed focus on creating heterostructures through the vertical integration of two-dimensional materials, including graphene, hexagonal boron nitride, and transition metal dichalcogenides (TMDCs). However, fundamental challenges associated with materials processing have limited material quality and impeded scalability. We demonstrate a method to convert sub-nanometer metal films deposited on silicon and sapphire into TMDC heterostructures through vapor-phase processing. The resulting heterostructures and superlattices exhibit novel properties compared with stand-alone TMDCs, including reduced bandgap, enhanced light-matter coupling, and improved catalytic performance. This robust and scalable synthetic method provides new opportunities to generate a wide range of artificially stacked 2D superlattices with controlled morphology and composition.

cond-mat.mtrl-sci↗

arXiv2021

Safe Online Gain Optimization for Variable Impedance Control

Smooth behaviors are preferable for many contact-rich manipulation tasks. Impedance control arises as an effective way to regulate robot movements by mimicking a mass-spring-damping system. Consequently, the robot behavior can be determined by the impedance gains. However, tuning the impedance gains for different tasks is tricky, especially for unstructured environments. Moreover, online adapting the optimal gains to meet the time-varying performance index is even more challenging. In this paper, we present Safe Online Gain Optimization for Variable Impedance Control (Safe OnGO-VIC). By reformulating the dynamics of impedance control as a control-affine system, in which the impedance gains are the inputs, we provide a novel perspective to understand variable impedance control. Additionally, we innovatively formulate an optimization problem with online collected force information to obtain the optimal impedance gains in real-time. Safety constraints are also embedded in the proposed framework to avoid unwanted collisions. We experimentally validated the proposed algorithm on three manipulation tasks. Comparison results with a constant gain baseline and an adaptive control method prove that the proposed algorithm is effective and generalizable to different scenarios.

cs.RO↗

arXiv2021

ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks

Electrocardiography (ECG) signal is a highly applied measurement for individual heart condition, and much effort have been endeavored towards automatic heart arrhythmia diagnosis based on machine learning. However, traditional machine learning models require large investment of time and effort for raw data preprocessing and feature extraction, as well as challenged by poor classification performance. Here, we propose a novel deep learning model, named Attention-Based Convolutional Neural Networks (ABCNN) that taking advantage of CNN and multi-head attention, to directly work on the raw ECG signals and automatically extract the informative dependencies for accurate arrhythmia detection. To evaluate the proposed approach, we conduct extensive experiments over a benchmark ECG dataset. Our main task is to find the arrhythmia from normal heartbeats and, at the meantime, accurately recognize the heart diseases from five arrhythmia types. We also provide convergence analysis of ABCNN and intuitively show the meaningfulness of extracted representation through visualization. The experimental results show that the proposed ABCNN outperforms the widely used baselines, which puts one step closer to intelligent heart disease diagnosis system.

eess.SP↗

arXiv2021

InfoGCL: Information-Aware Graph Contrastive Learning

Various graph contrastive learning models have been proposed to improve the performance of learning tasks on graph datasets in recent years. While effective and prevalent, these models are usually carefully customized. In particular, although all recent researches create two contrastive views, they differ greatly in view augmentations, architectures, and objectives. It remains an open question how to build your graph contrastive learning model from scratch for particular graph learning tasks and datasets. In this work, we aim to fill this gap by studying how graph information is transformed and transferred during the contrastive learning process and proposing an information-aware graph contrastive learning framework called InfoGCL. The key point of this framework is to follow the Information Bottleneck principle to reduce the mutual information between contrastive parts while keeping task-relevant information intact at both the levels of the individual module and the entire framework so that the information loss during graph representation learning can be minimized. We show for the first time that all recent graph contrastive learning methods can be unified by our framework. We empirically validate our theoretical analysis on both node and graph classification benchmark datasets, and demonstrate that our algorithm significantly outperforms the state-of-the-arts.

cs.LG↗

arXiv2021

Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks

This paper introduces a discrete-continuous action space to learn insertion primitives for robotic assembly tasks. Primitive is a sequence of elementary actions with certain exit conditions, such as "pushing down the peg until contact". Since the primitive is an abstraction of robot control commands and encodes human prior knowledge, it reduces the exploration difficulty and yields better learning efficiency. In this paper, we learn robot assembly skills via primitives. Specifically, we formulate insertion primitives as parameterized actions: hybrid actions consisting of discrete primitive types and continuous primitive parameters. Compared with the previous work using a set of discretized parameters for each primitive, the agent in our method can freely choose primitive parameters from a continuous space, which is more flexible and efficient. To learn these insertion primitives, we propose Twin-Smoothed Multi-pass Deep Q-Network (TS-MP-DQN), an advanced version of MP-DQN with twin Q-network to reduce the Q-value over-estimation. Extensive experiments are conducted in the simulation and real world for validation. From experiment results, our approach achieves higher success rates than three baselines: MP-DQN with parameterized actions, primitives with discrete parameters, and continuous velocity control. Furthermore, learned primitives are robust to sim-to-real transfer and can generalize to challenging assembly tasks such as tight round peg-hole and complex shaped electric connectors with promising success rates. Experiment videos are available at https://msc.berkeley.edu/research/insertion-primitives.html.

cs.RO↗

arXiv2021

A Q-Learning-based Approach for Distributed Beam Scheduling in mmWave Networks

We consider the problem of distributed downlink beam scheduling and power allocation for millimeter-Wave (mmWave) cellular networks where multiple base stations (BSs) belonging to different service operators share the same unlicensed spectrum with no central coordination or cooperation among them. Our goal is to design efficient distributed beam scheduling and power allocation algorithms such that the network-level payoff, defined as the weighted sum of the total throughput and a power penalization term, can be maximized. To this end, we propose a distributed scheduling approach to power allocation and adaptation for efficient interference management over the shared spectrum by modeling each BS as an independent Q-learning agent. As a baseline, we compare the proposed approach to the state-of-the-art non-cooperative game-based approach which was previously developed for the same problem. We conduct extensive experiments under various scenarios to verify the effect of multiple factors on the performance of both approaches. Experiment results show that the proposed approach adapts well to different interference situations by learning from experience and can achieve higher payoff than the game-based approach. The proposed approach can also be integrated into our previously developed Lyapunov stochastic optimization framework for the purpose of network utility maximization with optimality guarantee. As a result, the weights in the payoff function can be automatically and optimally determined by the virtual queue values from the sub-problems derived from the Lyapunov optimization framework.

cs.IT↗

arXiv2021

MindID: Person Identification from Brain Waves through Attention-based Recurrent Neural Network

Person identification technology recognizes individuals by exploiting their unique, measurable physiological and behavioral characteristics. However, the state-of-the-art person identification systems have been shown to be vulnerable, e.g., contact lenses can trick iris recognition and fingerprint films can deceive fingerprint sensors. EEG (Electroencephalography)-based identification, which utilizes the users' brainwave signals for identification and offers a more resilient solution, draw a lot of attention recently. However, the accuracy still requires improvement and very little work is focusing on the robustness and adaptability of the identification system. We propose MindID, an EEG-based biometric identification approach, achieves higher accuracy and better characteristics. At first, the EEG data patterns are analyzed and the results show that the Delta pattern contains the most distinctive information for user identification. Then the decomposed Delta pattern is fed into an attention-based Encoder-Decoder RNNs (Recurrent Neural Networks) structure which assigns various attention weights to different EEG channels based on the importance of channels. The discriminative representations learned from the attention-based RNN are used to recognize the user identification through a boosting classifier. The proposed approach is evaluated over 3 datasets (two local and one public). One local dataset (EID-M) is used for performance assessment and the result illustrates that our model achieves an accuracy of 0.982 which outperforms the baselines and the state-of-the-art. Another local dataset (EID-S) and a public dataset (EEG-S) are utilized to demonstrate the robustness and adaptability, respectively. The results indicate that the proposed approach has the potential to be largely deployed in the practice environment.

cs.HC↗

arXiv2021

Multiscale Reduced-Order Modeling of a Titanium Skin Panel Subjected to Thermo-Mechanical Loading

This manuscript presents the formulation, implementation, calibration and application of a multiscale reduced-order model to simulate a titanium panel structure subjected to thermo-mechanical loading associated with high-speed flight. The formulation is based on the eigenstrain-based reduced-order homogenization model (EHM) and further considers thermal strain as well as temperature dependent material properties and evolution laws. The material microstructure (i.e., at the scale of a polycrystalline representative volume element (RVE)) and underlying microstructural mechanisms are directly incorporated and fully coupled with a structural analysis, allowing concurrently probing the response at the structural scale and the material microscale. The proposed approach was fully calibrated using a series of uniaxial tension tests of Ti-6242S at a wide range of temperatures and two different strain rates. The calibrated model is then adopted to study the response of a generic aircraft skin panel subjected to thermo-mechanical loading associated with high-speed flight. The analysis focuses on demonstrating the capability of the model to predict not only the structural scale response, but simultaneously the microscale response, and further studies the effects of temperature and texture on the response.

cs.CE↗

arXiv2021

Tracking Fast Neural Adaptation by Globally Adaptive Point Process Estimation for Brain-Machine Interface

Brain-machine interfaces (BMIs) help the disabled restore body functions by translating neural activity into digital commands to control external devices. Neural adaptation, where the brain signals change in response to external stimuli or movements, plays an important role in BMIs. When subjects purely use neural activity to brain-control a prosthesis, some neurons will actively explore a new tuning property to accomplish the movement task. The prediction of this neural tuning property can help subjects adapt more efficiently to brain control and maintain good decoding performance. Existing prediction methods track the slow change of the tuning property in the manual control, which is not suitable for the fast neural adaptation in brain control. In order to identify the active neurons in brain control and track their tuning property changes, we propose a globally adaptive point process method (GaPP) to estimate the neural modulation state from spike trains, decompose the states into the hyper preferred direction and reconstruct the kinematics in a dual-model framework. We implement the method on real data from rats performing a two-lever discrimination task under manual control and brain control. The results show our method successfully predicts the neural modulation state and identifies the neurons that become active in brain control. Compared to existing methods, ours tracks the fast changes of the hyper preferred direction from manual control to brain control more accurately and efficiently and reconstructs the kinematics better and faster.

eess.SP↗