arXiv subjects

Jianyu Zhang

Publications and source records attributed to Jianyu Zhang.

45 records · Page 3Linked to original sources

arXiv2023

Learning useful representations for shifting tasks and distributions

Does the dominant approach to learn representations (as a side effect of optimizing an expected cost for a single training distribution) remain a good approach when we are dealing with multiple distributions? Our thesis is that such scenarios are better served by representations that are richer than those obtained with a single optimization episode. We support this thesis with simple theoretical arguments and with experiments utilizing an apparently na\"ıve ensembling technique: concatenating the representations obtained from multiple training episodes using the same data, model, algorithm, and hyper-parameters, but different random seeds. These independently trained networks perform similarly. Yet, in a number of scenarios involving new distributions, the concatenated representation performs substantially better than an equivalently sized network trained with a single training run. This proves that the representations constructed by multiple training episodes are in fact different. Although their concatenation carries little additional information about the training task under the training distribution, it becomes substantially more informative when tasks or distributions change. Meanwhile, a single training episode is unlikely to yield such a redundant representation because the optimization process has no reason to accumulate features that do not incrementally improve the training performance.

cs.LG↗

arXiv2022

Rich Feature Construction for the Optimization-Generalization Dilemma

There often is a dilemma between ease of optimization and robust out-of-distribution (OoD) generalization. For instance, many OoD methods rely on penalty terms whose optimization is challenging. They are either too strong to optimize reliably or too weak to achieve their goals. We propose to initialize the networks with a rich representation containing a palette of potentially useful features, ready to be used by even simple models. On the one hand, a rich representation provides a good initialization for the optimizer. On the other hand, it also provides an inductive bias that helps OoD generalization. Such a representation is constructed with the Rich Feature Construction (RFC) algorithm, also called the Bonsai algorithm, which consists of a succession of training episodes. During discovery episodes, we craft a multi-objective optimization criterion and its associated datasets in a manner that prevents the network from using the features constructed in the previous iterations. During synthesis episodes, we use knowledge distillation to force the network to simultaneously represent all the previously discovered features. Initializing the networks with Bonsai representations consistently helps six OoD methods achieve top performance on ColoredMNIST benchmark. The same technique substantially outperforms comparable results on the Wilds Camelyon17 task, eliminates the high result variance that plagues other methods, and makes hyperparameter tuning and model selection more reliable.

cs.LG↗

arXiv2020

Non-Hermitian coherent coupling of nanomagnets by exchange spin waves

Non-Hermitian physics has recently attracted much attention in optics and photonics. Less explored is non-Hermitian magnonics that provides opportunities to take advantage of the inevitable dissipation of magnons or spin waves in magnetic systems. Here we demonstrate non-Hermitian coherent coupling of two distant nanomagnets by fast spin waves with sub-50 nm wavelengths. Magnons in two nanomagnets are unidirectionally phase-locked with phase shifts controlled by magnon spin torque and spin-wave propagation. Our results are attractive for analog neuromorphic computing that requires unidirectional information transmission.

cond-mat.mes-hall↗

arXiv2020

Symplectic Recurrent Neural Networks

We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories. An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiple-step training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems. We show that SRNNs succeed reliably on complex and noisy Hamiltonian systems. We also show how to augment the SRNN integration scheme in order to handle stiff dynamical systems such as bouncing billiards.

cs.LG↗

arXiv2019

Chiral spin-wave velocities induced by all-garnet interfacial Dzyaloshinskii-Moriya interaction in ultrathin yttrium iron garnet films

Spin waves can probe the Dzyaloshinskii-Moriya interaction (DMI) which gives rise to topological spin textures, such as skyrmions. However, the DMI has not yet been reported in yttrium iron garnet (YIG) with arguably the lowest damping for spin waves. In this work, we experimentally evidence the interfacial DMI in a 7~nm-thick YIG film by measuring the nonreciprocal spin wave propagation in terms of frequency, amplitude and most importantly group velocities using all electrical spin-wave spectroscopy. The velocities of propagating spin waves show chirality among three vectors, i.e. the film normal direction, applied field and spin-wave wavevector. By measuring the asymmetric group velocities, we extract a DMI constant of 16~$μ$J/m$^{2}$ which we independently confirm by Brillouin light scattering. Thickness-dependent measurements reveal that the DMI originates from the oxide interface between the YIG and garnet substrate. The interfacial DMI discovered in the ultrathin YIG films is of key importance for functional chiral magnonics as ultra-low spin-wave damping can be achieved.

cond-mat.mtrl-sci↗

arXiv2019

Excitation of unidirectional exchange spin waves by a nanoscale magnetic grating

Magnon spintronics is a prosperous field that promises beyond-CMOS technology based on elementary excitations of the magnetic order that act as information carriers for future computational architectures. Unidirectional propagation of spin waves is key to the realization of magnonic logic devices. However, previous efforts to enhance the Damon-Eshbach-type nonreciprocity did not realize (let alone control) purely unidirectional propagation. Here we experimentally demonstrate excitations of unidirectional exchange spin waves by a nanoscale magnetic grating consisting of Co nanowires fabricated on an ultrathin yttrium iron garnet film. We explain and model the nearly perfect unidirectional excitation by the chirality of the magneto-dipolar interactions between the Kittel mode of the nanowires and the exchange spin waves of the film. Reversal of the magnetic configurations of film and nanowire array from parallel to antiparallel changes the direction of the excited spin waves. Our results raise the prospect of a chiral magnonic logic without the need for fragile surface states.

physics.app-ph↗

arXiv2017

Progress Estimation and Phase Detection for Sequential Processes

Process modeling and understanding are fundamental for advanced human-computer interfaces and automation systems. Most recent research has focused on activity recognition, but little has been done on sensor-based detection of process progress. We introduce a real-time, sensor-based system for modeling, recognizing and estimating the progress of a work process. We implemented a multimodal deep learning structure to extract the relevant spatio-temporal features from multiple sensory inputs and used a novel deep regression structure for overall completeness estimation. Using process completeness estimation with a Gaussian mixture model, our system can predict the phase for sequential processes. The performance speed, calculated using completeness estimation, allows online estimation of the remaining time. To train our system, we introduced a novel rectified hyperbolic tangent (rtanh) activation function and conditional loss. Our system was tested on data obtained from the medical process (trauma resuscitation) and sports events (Olympic swimming competition). Our system outperformed the existing trauma-resuscitation phase detectors with a phase detection accuracy of over 86%, an F1-score of 0.67, a completeness estimation error of under 12.6%, and a remaining-time estimation error of less than 7.5 minutes. For the Olympic swimming dataset, our system achieved an accuracy of 88%, an F1-score of 0.58, a completeness estimation error of 6.3% and a remaining-time estimation error of 2.9 minutes.

cs.LG↗

arXiv2017

Concurrent Activity Recognition with Multimodal CNN-LSTM Structure

We introduce a system that recognizes concurrent activities from real-world data captured by multiple sensors of different types. The recognition is achieved in two steps. First, we extract spatial and temporal features from the multimodal data. We feed each datatype into a convolutional neural network that extracts spatial features, followed by a long-short term memory network that extracts temporal information in the sensory data. The extracted features are then fused for decision making in the second step. Second, we achieve concurrent activity recognition with a single classifier that encodes a binary output vector in which elements indicate whether the corresponding activity types are currently in progress. We tested our system with three datasets from different domains recorded using different sensors and achieved performance comparable to existing systems designed specifically for those domains. Our system is the first to address the concurrent activity recognition with multisensory data using a single model, which is scalable, simple to train and easy to deploy.

cs.CV↗

arXiv2016

How large should the QM region be in QM/MM calculations? The case of catechol O-methyltransferase

Hybrid quantum mechanical-molecular mechanical (QM/MM) simulations are widely used in studies of enzymatic catalysis. Until recently, it has been cost prohibitive to determine the asymptotic limit of key energetic and structural properties with respect to increasingly large QM regions. Leveraging recent advances in electronic structure efficiency and accuracy, we investigate catalytic properties in catechol O-methyltransferase, a representative example of a methyltransferase critical to human health. Using QM regions ranging in size from reactants-only (64 atoms) to nearly one-third of the entire protein (940 atoms), we show that properties such as the activation energy approach within chemical accuracy of the large-QM asymptotic limits rather slowly, requiring approximately 500-600 atoms if the QM residues are chosen simply by distance from the substrate. This slow approach to asymptotic limit is due to charge transfer from protein residues to the reacting substrates. Our large QM/MM calculations enable identification of charge separation for fragments in the transition state as a key component of enzymatic methyl transfer rate enhancement. We introduce charge shift analysis that reveals the minimum number of protein residues (ca. 11-16 residues or 200-300 atoms for COMT) needed for quantitative agreement with large-QM simulations. The identified residues are not those that would be typically selected using criteria such as chemical intuition or proximity. These results provide a recipe for a more careful determination of QM region sizes in future QM/MM studies of enzymes.

q-bio.BM↗