arXiv subjects

Peng Li

Publications and source records attributed to Peng Li.

At least 361 records · Page 20Linked to original sources

arXiv2022

Interferometry based on quantum Kibble-Zurek mechanism

We propose an interferometry within the framework of quantum Kibble-Zurek mechanism by exemplifying two prototypical quench protocols, namely the round-trip and quarter-turn ones, on the transverse Ising and quantum $XY$ chains. Each protocol contains two linear ramps that drive the system across quantum critical point twice. The two linear ramps arouse two respective nonadiabatic critical dynamics that are well described by the quantum Kibble-Zurek mechanism. However, in combination, the two critical dynamics can interfere with each other deeply. As an effect of the interference, the dynamical phase is exposed in the final excitation probability, which leads to a quantum coherent many-body oscillation in the density of defects with predictable characteristic period. Thus such an interference is available for direct experimental observations. In the quantum $XY$ model, we show that an interference can also arise from the interplay between two different critical dynamics derived from a critical point and a tricritical point. Furthermore, we demonstrate that the interference influences the dephasing of the excited quasiparticle modes intricately by disclosing a phenomenon of multiple length scales, diagonal and off-diagonal ones, in the defect-defect correlators. It turns out that the dephased result relies on how the diagonal and off-diagonal lengths are modulated by the controllable parameter in a quench protocol.

quant-ph↗

arXiv2022

Packed Levitated Marker for Entity and Relation Extraction

Recent entity and relation extraction works focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the interrelation between spans (pairs). In this work, we propose a novel span representation approach, named Packed Levitated Markers (PL-Marker), to consider the interrelation between the spans (pairs) by strategically packing the markers in the encoder. In particular, we propose a neighborhood-oriented packing strategy, which considers the neighbor spans integrally to better model the entity boundary information. Furthermore, for those more complicated span pair classification tasks, we design a subject-oriented packing strategy, which packs each subject and all its objects to model the interrelation between the same-subject span pairs. The experimental results show that, with the enhanced marker feature, our model advances baselines on six NER benchmarks, and obtains a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models on ACE04 and ACE05.

cs.CL↗

arXiv2022

A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

cs.LG↗

arXiv2022

MoEfication: Transformer Feed-forward Layers are Mixtures of Experts

Recent work has shown that feed-forward networks (FFNs) in pre-trained Transformers are a key component, storing various linguistic and factual knowledge. However, the computational patterns of FFNs are still unclear. In this work, we study the computational patterns of FFNs and observe that most inputs only activate a tiny ratio of neurons of FFNs. This phenomenon is similar to the sparsity of the human brain, which drives research on functional partitions of the human brain. To verify whether functional partitions also emerge in FFNs, we propose to convert a model into its MoE version with the same parameters, namely MoEfication. Specifically, MoEfication consists of two phases: (1) splitting the parameters of FFNs into multiple functional partitions as experts, and (2) building expert routers to decide which experts will be used for each input. Experimental results show that MoEfication can conditionally use 10% to 30% of FFN parameters while maintaining over 95% original performance for different models on various downstream tasks. Besides, MoEfication brings two advantages: (1) it significantly reduces the FLOPS of inference, i.e., 2x speedup with 25% of FFN parameters, and (2) it provides a fine-grained perspective to study the inner mechanism of FFNs. The source code of this paper can be obtained from https://github.com/thunlp/MoEfication.

cs.CL↗

arXiv2022

OPAL: Occlusion Pattern Aware Loss for Unsupervised Light Field Disparity Estimation

Light field disparity estimation is an essential task in computer vision with various applications. Although supervised learning-based methods have achieved both higher accuracy and efficiency than traditional optimization-based methods, the dependency on ground-truth disparity for training limits the overall generalization performance not to say for real-world scenarios where the ground-truth disparity is hard to capture. In this paper, we argue that unsupervised methods can achieve comparable accuracy, but, more importantly, much higher generalization capacity and efficiency than supervised methods. Specifically, we present the Occlusion Pattern Aware Loss, named OPAL, which successfully extracts and encodes the general occlusion patterns inherent in the light field for loss calculation. OPAL enables: i) accurate and robust estimation by effectively handling occlusions without using any ground-truth information for training and ii) much efficient performance by significantly reducing the network parameters required for accurate inference. Besides, a transformer-based network and a refinement module are proposed for achieving even more accurate results. Extensive experiments demonstrate our method not only significantly improves the accuracy compared with the SOTA unsupervised methods, but also possesses strong generalization capacity, even for real-world data, compared with supervised methods. Our code will be made publicly available.

cs.CV↗

arXiv2022

Tunable magnetically induced transparency spectra in magnon-magnon coupled Y3Fe5O12/permalloy bilayers

Hybrid magnonic systems host a variety of characteristic quantum phenomena such as the magnetically-induced transparency (MIT) and Purcell effect, which are considered useful for future coherent quantum information processing. In this work, we experimentally demonstrate a tunable MIT effect in the Y3Fe5O12(YIG)/Permalloy(Py) magnon-magnon coupled system via changing the magnetic field orientations. By probing the magneto-optic effects of Py and YIG, we identify clear features of MIT spectra induced by the mode hybridization between the uniform mode of Py and the perpendicular standing spin-wave modes of YIG. By changing the external magnetic field orientations, we observe a tunable coupling strength between the YIG's spin-wave modes and the Py's uniform mode, upon the application of an out-of-plane magnetic field. This observation is theoretically interpreted by a geometrical consideration of the Py and YIG magnetization under the oblique magnetic field even at a constant interfacial exchange coupling. Our findings show high promise for investigating tunable coherent phenomena with hybrid magnonic platforms.

cond-mat.mtrl-sci↗

arXiv2022

Deterministic relation between optical polarization and lattice symmetry revealed in ion-doped single microcrystals

Rare-earth ions doped crystals are of great significance for micro-sensing and quantum information, whilst the ions in the crystals emit light with spontaneous partial polarization, which is, though believed to be originated from the crystal lattice structure, still lacking a deterministic explanation that can be tested with quantitative accuracy. We report the experimental evidence showing the profound physical relation between the polarization degree of light emitted by the doped ion and the lattice symmetry, by demonstrating, with unprecedented precision, that the lattice constant ratio c/a directly quantifies the macroscopic effective polar angle of the electric and magnetic dipoles, which essentially determines the linear polarization degree of the emission. Based on this discovery, we further propose a pure optical technology to identify the three-dimensional orientation of a rod-shaped single microcrystal using the polarization-resolved micro-spectroscopy. Our results, revealing the physical origin of light polarization in ion-doped crystals, open the way towards on-demand polarization control with crystallography, and provide a versatile platform for polarization-based microscale sensing in dynamical systems.

physics.optics↗

arXiv2022

Fully Hyperbolic Neural Networks

Hyperbolic neural networks have shown great potential for modeling complex data. However, existing hyperbolic networks are not completely hyperbolic, as they encode features in a hyperbolic space yet formalize most of their operations in the tangent space (a Euclidean subspace) at the origin of the hyperbolic space. This hybrid method greatly limits the modeling ability of networks. In this paper, we propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model by adapting the Lorentz transformations (including boost and rotation) to formalize essential operations of neural networks. Moreover, we also prove that linear transformation in tangent spaces used by existing hyperbolic networks is a relaxation of the Lorentz rotation and does not include the boost, implicitly limiting the capabilities of existing hyperbolic networks. The experimental results on four NLP tasks show that our method has better performance for building both shallow and deep networks. Our code will be released to facilitate follow-up research.

cs.CL↗

arXiv2022

Quantum Interference Transport in two-dimensional Semi-Dirac Semimetals

Semi-Dirac semimetals have received enthusiastic research both theoretically and experimentally in the recent years. Due to the anisotropic dispersion, its physical properties are highly direction-dependent. In this work we employ the Feynman diagrammatic perturbation theory to study the transport properties in quantum diffusive regime. The magneto-conductivity with quantum interference corrections is derived, which demonstrate the weak localization effect in the semi-Dirac semimetal. Furthermore, the origin of anomalous Hall conductivity is also clarified, where both the intrinsic and side-jump contributions vanish and only the skew-scattering gives rise to non-zero transverse conductivity. The conductance fluctuations in both mesoscopic and quantum diffusive regimes are investigated in detail. Our work provides theoretical predictions for transport experiments, which can be examined by conductivity measurements at sufficiently low temperature.

cond-mat.mes-hall↗

arXiv2022

Binary Neural Networks as a general-propose compute paradigm for on-device computer vision

For binary neural networks (BNNs) to become the mainstream on-device computer vision algorithm, they must achieve a superior speed-vs-accuracy tradeoff than 8-bit quantization and establish a similar degree of general applicability in vision tasks. To this end, we propose a BNN framework comprising 1) a minimalistic inference scheme for hardware-friendliness, 2) an over-parameterized training scheme for high accuracy, and 3) a simple procedure to adapt to different vision tasks. The resultant framework overtakes 8-bit quantization in the speed-vs-accuracy tradeoff for classification, detection, segmentation, super-resolution and matching: our BNNs not only retain the accuracy levels of their 8-bit baselines but also showcase 1.3-2.4$\times$ faster FPS on mobile CPUs. Similar conclusions can be drawn for prototypical systolic-array-based AI accelerators, where our BNNs promise 2.8-7$\times$ fewer execution cycles than 8-bit and 2.1-2.7$\times$ fewer cycles than alternative BNN designs. These results suggest that the time for large-scale BNN adoption could be upon us.

cs.CV↗

arXiv2022

Growth, Electronic Structure and Superconductivity of Ultrathin Epitaxial CoSi2 Films

We report growth, electronic structure and superconductivity of ultrathin epitaxial CoSi2 films on Si(111). At low coverages, preferred islands with 2, 5 and 6 monolayers height develop, which agrees well with the surface energy calculation. We observe clear quantum well states as a result of electronic confinement and their dispersion agrees well with density functional theory calculations, indicating weak correlation effect despite strong contributions from Co 3d electrons. Ex-situ transport measurements show that superconductivity persists down to at least 10 monolayers, with reduced Tc but largely enhanced upper critical field. Our study opens up the opportunity to study the interplay between quantum confinement, interfacial symmetry breaking and superconductivity in an epitaxial silicide film, which is technologically relevant in microelectronics.

cond-mat.mes-hall↗

arXiv2021

Signal and Image Reconstruction with Tight Frames via Unconstrained $\ell_1-α\ell_2$-Analysis Minimizations

In the paper, we introduce an unconstrained analysis model based on the $\ell_{1}-α\ell_{2}$ $(0< α\leq1)$ minimization for the signal and image reconstruction. We develop some new technology lemmas for tight frame, and the recovery guarantees based on the restricted isometry property adapted to frames. The effective algorithm is established for the proposed nonconvex analysis model. We illustrate the performance of the proposed model and algorithm for the signal and compressed sensing MRI reconstruction via extensive numerical experiments. And their performance is better than that of the existing methods.

cs.IT↗

arXiv2021

Commensurate stacking phase transitions in an intercalated transition metal dichalcogenide

Intercalation and stacking order modulation are two active ways in manipulating the interlayer interaction of transition metal dichalcogenides (TMDCs), which lead to a variety of emergent phases and allow for engineering material properties. Herein, the growth of Pb intercalated TMDCs--Pb(Ta$_{1+x}$Se$_2$)$_2$, the first 124-phase, is reported. Pb(Ta$_{1+x}$Se$_2$)$_2$ exhibits a unique two-step first-order structural phase transition at around 230 K. The transitions are solely associated with the stacking degree of freedom, evolving from a high temperature phase with ABC stacking and symmetry $R3m$ to an intermediate phase with AB stacking and $P3m1$, and finally to a low temperature phase with again symmetry $R3m$, but with ACB stacking. Each step involves a rigid slide of building blocks by a vector [1/3, 2/3, 0]. Intriguingly, gigantic lattice contractions occur at the transitions on warming. At low temperature, bulk superconductivity with $T_\textrm{c}\approx$ 1.8 K is observed. The underlying physics of the structural phase transitions are discussed from first-principle calculations. The symmetry analysis reveals topological nodal lines in the band structure. Our results demonstrate the possibility to realize higher order metal intercalated phases of TMDCs, advance our knowledge of polymorphic transitions and may inspire stacking order engineering in TMDCs and beyond.

cond-mat.mtrl-sci↗

arXiv2021

Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models

As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly reduce the retraining computational cost and the potential environmental side-effects. In this paper, we explore a novel model reuse paradigm, Knowledge Amalgamation~(KA) for PLMs. Without human annotations available, KA aims to merge the knowledge from different teacher-PLMs, each of which specializes in a different classification problem, into a versatile student model. The achieve this, we design a Model Uncertainty--aware Knowledge Amalgamation~(MUKA) framework, which identifies the potential adequate teacher using Monte-Carlo Dropout for approximating the golden supervision to guide the student. Experimental results demonstrate that MUKA achieves substantial improvements over baselines on benchmark datasets. Further analysis shows that MUKA can generalize well under several complicate settings with multiple teacher models, heterogeneous teachers, and even cross-dataset teachers.

cs.CL↗

arXiv2021

Room temperature energy-efficient spin-orbit torque switching in wafer-scale all-vdW heterostructure

The emergent two-dimensional (2D) ferromagnetic materials with unique magnetic properties have endowed great potential for next-generation spintronic devices with extraordinary merits of high flexibility, easy controllability, and high heretointegrability, which is expected to promote the development of Moore's Law continuously. However, it is extremely challenging to realize magnetic switching with ultra-low power consumption at room temperature. Here, we demonstrate the room-temperature spin-orbit torque (SOT) driven magnetization switching in a well-epitaxial all-van der Waals (vdW) heterostructure. The topological insulator Bi2Te3 not only helps to elevate the Curie temperature of Fe3GeTe2 (FGT) through interfacial exchange coupling but also works as a spin current source allowing to switch FGT at a low current density of 2.2 * 106 A cm2. A large SOT efficiency of 0.7 is measured at room temperature, and the thickness of FGT is further adjusted to reduce the influence of the thermal contribution on the second-harmonic signal. Furthermore, the temperature and thickness-dependent SOT efficiency prove that the large SOT in our system mainly originates from the nontrivial origin of topological materials. Our experiment has enabled an all-vdW SOT structure and lays a solid foundation for the implementation of room-temperature all-vdW spintronic devices in the future.

physics.app-ph↗

arXiv2021

Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach

Despite decades of efforts, robot navigation in a real scenario with volatility, uncertainty, complexity, and ambiguity (VUCA for short), remains a challenging topic. Inspired by the central nervous system (CNS), we propose a hierarchical multi-expert learning framework for autonomous navigation in a VUCA environment. With a heuristic exploration mechanism considering target location, path cost, and safety level, the upper layer performs simultaneous map exploration and route-planning to avoid trapping in a blind alley, similar to the cerebrum in the CNS. Using a local adaptive model fusing multiple discrepant strategies, the lower layer pursuits a balance between collision-avoidance and go-straight strategies, acting as the cerebellum in the CNS. We conduct simulation and real-world experiments on multiple platforms, including legged and wheeled robots. Experimental results demonstrate our algorithm outperforms the existing methods in terms of task achievement, time efficiency, and security.

cs.RO↗

arXiv2021

BioLeaF: A Bio-plausible Learning Framework for Training of Spiking Neural Networks

Our brain consists of biological neurons encoding information through accurate spike timing, yet both the architecture and learning rules of our brain remain largely unknown. Comparing to the recent development of backpropagation-based (BP-based) methods that are able to train spiking neural networks (SNNs) with high accuracy, biologically plausible methods are still in their infancy. In this work, we wish to answer the question of whether it is possible to attain comparable accuracy of SNNs trained by BP-based rules with bio-plausible mechanisms. We propose a new bio-plausible learning framework, consisting of two components: a new architecture, and its supporting learning rules. With two types of cells and four types of synaptic connections, the proposed local microcircuit architecture can compute and propagate error signals through local feedback connections and support training of multi-layers SNNs with a globally defined spiking error function. Under our microcircuit architecture, we employ the Spike-Timing-Dependent-Plasticity (STDP) rule operating in local compartments to update synaptic weights and achieve supervised learning in a biologically plausible manner. Finally, We interpret the proposed framework from an optimization point of view and show the equivalence between it and the BP-based rules under a special circumstance. Our experiments show that the proposed framework demonstrates learning accuracy comparable to BP-based rules and may provide new insights on how learning is orchestrated in biological systems.

cs.NE↗

arXiv2021

FedGraph: Federated Graph Learning with Intelligent Sampling

Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. However, existing work of federated learning mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular in many applications. Graph Convolutional Network (GCN) has been proposed as one of the most promising techniques for graph learning, but its federated setting has been seldom explored. In this paper, we propose FedGraph for federated graph learning among multiple computing clients, each of which holds a subgraph. FedGraph provides strong graph learning capability across clients by addressing two unique challenges. First, traditional GCN training needs feature data sharing among clients, leading to risk of privacy leakage. FedGraph solves this issue using a novel cross-client convolution operation. The second challenge is high GCN training overhead incurred by large graph size. We propose an intelligent graph sampling algorithm based on deep reinforcement learning, which can automatically converge to the optimal sampling policies that balance training speed and accuracy. We implement FedGraph based on PyTorch and deploy it on a testbed for performance evaluation. The experimental results of four popular datasets demonstrate that FedGraph significantly outperforms existing work by enabling faster convergence to higher accuracy.

cs.LG↗