arXiv subjects

Bo Yuan

Publications and source records attributed to Bo Yuan.

At least 199 records · Page 11Linked to original sources

arXiv2018

Neutron Scattering Investigation of Rhenium Orbital Ordering in $3d-5d$ Double Perovskite Ca$_2$FeReO$_6$

We have carried out inelastic neutron scattering experiments to study magnetic excitations in ordered double perovskite Ca$_2$FeReO$_6$. We found a well-defined magnon mode with a bandwidth of $\sim$50meV below the ferri-magnetic ordering temperature ($T_c\sim$520K), similar to previously studied Ba$_2$FeReO$_6$. The spin excitation is gapless for most temperatures within the magnetically ordered phase. However, a spin gap of $\sim$10meV opens up below $\sim$150K, which is well below the magnetic ordering temperature but coincides with a previously reported metal-insulator transition and onset of structural distortion. The observed temperature dependence of spin gap provides strong evidence for ordering of Re orbitals at $\sim$150~K, in accordance with earlier proposal put forward by Oikawa $\it{et.\,al}$ based on neutron diffraction [J. Phys. Soc. Jpn., $\bf{72}$, 1411 (2003)] as well as recent theoretical work by Lee and Marianetti [Phys. Rev. B, $\bf{97}$, 045102 (2018)]. The presence of separate orbital and magnetic ordering in Ca$_2$FeReO$_6$ suggests weak coupling between spin and orbital degrees of freedom and hints towards a sub-dominant role played by spin orbit coupling in describing its magnetism. In addition, we observed only one well-defined magnon band near magnetic zone boundary, which is incompatible with simple ferrimagnetic spin waves arising from Fe and Re local moments, but suggests a strong damping of Re magnon mode.

cond-mat.str-el↗

arXiv2018

SGAD: Soft-Guided Adaptively-Dropped Neural Network

Deep neural networks (DNNs) have been proven to have many redundancies. Hence, many efforts have been made to compress DNNs. However, the existing model compression methods treat all the input samples equally while ignoring the fact that the difficulties of various input samples being correctly classified are different. To address this problem, DNNs with adaptive dropping mechanism are well explored in this work. To inform the DNNs how difficult the input samples can be classified, a guideline that contains the information of input samples is introduced to improve the performance. Based on the developed guideline and adaptive dropping mechanism, an innovative soft-guided adaptively-dropped (SGAD) neural network is proposed in this paper. Compared with the 32 layers residual neural networks, the presented SGAD can reduce the FLOPs by 77% with less than 1% drop in accuracy on CIFAR-10.

cs.LG↗

arXiv2018

Spin-orbit coupled systems in the "atomic" limit: rhenates, osmates, iridates

Motivated by RIXS experiments on a wide range of complex heavy oxides, including rhenates, osmates, and iridates, we discuss the theory of RIXS for site-localized $t_{2g}$ orbital systems with strong spin-orbit coupling. For such systems, we present exact diagonalization results for the spectrum at different electron fillings, showing that it accesses "single-particle" and "multi-particle" excitations. This leads to a simple picture for the energies and intensities of the RIXS spectra in Mott insulators such as double perovskites which feature highly localized electrons, and yields estimates of the spin-orbit coupling and Hund's coupling in correlated $5d$ oxides. We present new higher resolution RIXS data at the Re-L$_3$ edge in Ba$_2$YReO$_6$ which finds a previously unresolved peak splitting, providing further confirmation of our theoretical predictions. Using ab initio electronic structure calculations on Ba$_2$${\cal M}$ReO$_6$ (with ${\cal M}$=Re, Os, Ir) we show that while the atomic limit yields a reasonable effective Hamiltonian description of the experimental observations, effects such as $t_{2g}$-$e_g$ interactions and hybridization with oxygen are important. Our ab initio estimate for the strength of the intersite exchange coupling shows that, compared to the osmates, the exchange is one or two orders of magnitude weaker in the rhenates and iridates, which may partly explain the suppression of long-range magnetic order in the latter compounds. As a way to interpolate between the site-localized picture and our electronic structure band calculations, we discuss the spin-orbital levels of the ${\cal M}$O$_6$ cluster. This suggests a possible role for non-dispersive intra-cluster excitons in Ba$_2$YIrO$_6$ which may lead to a weak breakdown of the atomic $J_{\rm eff}=0$ picture and to small magnetic moments.

cond-mat.str-el↗

arXiv2018

On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networks

Large-scale deep neural networks are both memory intensive and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated in both industry and academia. Specific forms of binary neural networks (BNNs) and stochastic computing based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper we prove that the "ideal" SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior). The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a "bridge" to prove for BNNs. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growing of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable for hardware.

cs.LG↗

arXiv2018

Towards Budget-Driven Hardware Optimization for Deep Convolutional Neural Networks using Stochastic Computing

Recently, Deep Convolutional Neural Network (DCNN) has achieved tremendous success in many machine learning applications. Nevertheless, the deep structure has brought significant increases in computation complexity. Largescale deep learning systems mainly operate in high-performance server clusters, thus restricting the application extensions to personal or mobile devices. Previous works on GPU and/or FPGA acceleration for DCNNs show increasing speedup, but ignore other constraints, such as area, power, and energy. Stochastic Computing (SC), as a unique data representation and processing technique, has the potential to enable the design of fully parallel and scalable hardware implementations of large-scale deep learning systems. This paper proposed an automatic design allocation algorithm driven by budget requirement considering overall accuracy performance. This systematic method enables the automatic design of a DCNN where all design parameters are jointly optimized. Experimental results demonstrate that proposed algorithm can achieve a joint optimization of all design parameters given the comprehensive budget of a DCNN.

cs.NE↗

arXiv2018

Asymmetric viscothermal acoustic propagation and implication on flow measurement

In the application of high frequency acoustic flow measurement, viscothermal dissipation and asymmetric acoustic modes cannot be overlooked. Present paper mathematically formulates asymmetric linear disturbance dynamics in terms of velocity and temperature disturbances based on the conservations of mass, momentum and energy. An iterative calculation procedure, which is similar to Galerkin method, is presented. Numerical analysis of asymmetric acoustic features (phase velocity and attenuation coefficient) are comprehensively given under the effects of viscothermal dissipation and shear flow convection. In the end, flow measurement performance of asymmetric acoustic modes is literally discussed. Numerical study shows that viscothermal dissipation affects the cut-on frequency of acoustic modes and couples nonlinearly with shear convection when the flow Mach number is large. These parameters impose significant influences on measurement performance. Each acoustic mode has inherent measurement derivation which can be theoretically used to compensate the acoustic flow measurement error. Apparent prediction error may occur if the viscothermal dissipation is taken out of consideration

physics.flu-dyn↗

arXiv2018

Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs

Both industry and academia have extensively investigated hardware accelerations. In this work, to address the increasing demands in computational capability and memory requirement, we propose structured weight matrices (SWM)-based compression techniques for both \emph{field programmable gate array} (FPGA) and \emph{application-specific integrated circuit} (ASIC) implementations. In algorithm part, SWM-based framework adopts block-circulant matrices to achieve a fine-grained tradeoff between accuracy and compression ratio. The SWM-based technique can reduce computational complexity from O($n^2$) to O($n\log n$) and storage complexity from O($n^2$) to O($n$) for each layer and both training and inference phases. For FPGA implementations on deep convolutional neural networks (DCNNs), we achieve at least 152X and 72X improvement in performance and energy efficiency, respectively using the SWM-based framework, compared with the baseline of IBM TrueNorth processor under same accuracy constraints using the data set of MNIST, SVHN, and CIFAR-10. For FPGA implementations on long short term memory (LSTM) networks, the proposed SWM-based LSTM can achieve up to 21X enhancement in performance and 33.5X gains in energy efficiency compared with the baseline accelerator. For ASIC implementations, the SWM-based ASIC design exhibits impressive advantages in terms of power, throughput, and energy efficiency. Experimental results indicate that this method is greatly suitable for applying DNNs onto both FPGAs and mobile/IoT devices.

cs.DC↗

arXiv2018

C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs

Recently, significant accuracy improvement has been achieved for acoustic recognition systems by increasing the model size of Long Short-Term Memory (LSTM) networks. Unfortunately, the ever-increasing size of LSTM model leads to inefficient designs on FPGAs due to the limited on-chip resources. The previous work proposes to use a pruning based compression technique to reduce the model size and thus speedups the inference on FPGAs. However, the random nature of the pruning technique transforms the dense matrices of the model to highly unstructured sparse ones, which leads to unbalanced computation and irregular memory accesses and thus hurts the overall performance and energy efficiency. In contrast, we propose to use a structured compression technique which could not only reduce the LSTM model size but also eliminate the irregularities of computation and memory accesses. This approach employs block-circulant instead of sparse matrices to compress weight matrices and reduces the storage requirement from $\mathcal{O}(k^2)$ to $\mathcal{O}(k)$. Fast Fourier Transform algorithm is utilized to further accelerate the inference by reducing the computational complexity from $\mathcal{O}(k^2)$ to $\mathcal{O}(k\text{log}k)$. The datapath and activation functions are quantized as 16-bit to improve the resource utilization. More importantly, we propose a comprehensive framework called C-LSTM to automatically optimize and implement a wide range of LSTM variants on FPGAs. According to the experimental results, C-LSTM achieves up to 18.8X and 33.5X gains for performance and energy efficiency compared with the state-of-the-art LSTM implementation under the same experimental setup, and the accuracy degradation is very small.

cs.LG↗

arXiv2018

Evolution of Magnetic Excitations Across the Metal-Insulator Transition in a Pyrochlore Iridate Eu$_{2}$Ir$_{2}$O$_{7}$

We report Resonant Inelastic X-ray Scattering (RIXS) study of the magnetic excitation spectrum in a highly insulating Eu$_{2}$Ir$_{2}$O$_{7}$ single crystal that exhibits a metal-insulator transition at $T_{MI}$ = 111(7) K. A propagating magnon mode with 20 meV bandwidth and 28 meV magnon gap is found in the excitation spectrum at 7 K, which is expected in the all-in-all-out (AIAO) magnetically ordered state. This magnetic excitation exhibits substantial softening as temperature is raised towards $T_{MI}$, and turns into highly damped excitation in the paramagnetic phase. Remarkably, the softening occurs throughout the whole Brillouin zone including the zone boundary. This observation is inconsistent with magnon renormalization expected in a local moment system, and indicates that the strength of electron correlation in Eu$_{2}$Ir$_{2}$O$_{7}$ is only moderate, so that electron itinerancy should be taken into account in describing its magnetism.

cond-mat.str-el↗

arXiv2018

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which is applicable to different DNN types, sizes, and application scenarios. The algorithm part adopts the general block-circulant matrices to achieve a fine-grained tradeoff between accuracy and compression ratio. It applies to both fully-connected and convolutional layers and contains a mathematically rigorous proof of the effectiveness of the method. The proposed algorithm reduces computational complexity per layer from O($n^2$) to O($n\log n$) and storage complexity from O($n^2$) to O($n$), both for training and inference. The hardware part consists of highly efficient Field Programmable Gate Array (FPGA)-based implementations using effective reconfiguration, batch processing, deep pipelining, resource re-using, and hierarchical control. Experimental results demonstrate that the proposed framework achieves at least 152X speedup and 71X energy efficiency gain compared with IBM TrueNorth processor under the same test accuracy. It achieves at least 31X energy efficiency gain compared with the reference FPGA-based work.

cs.LG↗

arXiv2017

Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank

Recently low displacement rank (LDR) matrices, or so-called structured matrices, have been proposed to compress large-scale neural networks. Empirical results have shown that neural networks with weight matrices of LDR matrices, referred as LDR neural networks, can achieve significant reduction in space and computational complexity while retaining high accuracy. We formally study LDR matrices in deep learning. First, we prove the universal approximation property of LDR neural networks with a mild condition on the displacement operators. We then show that the error bounds of LDR neural networks are as efficient as general neural networks with both single-layer and multiple-layer structure. Finally, we propose back-propagation based training algorithm for general LDR neural networks.

cs.LG↗

arXiv2017

CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices

Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning; 2) the increased training complexity; and 3) the lack of rigorous guarantee of compression ratio and inference accuracy. To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from O(n2) to O(nlogn) and the storage complexity from O(n2) to O(n), with negligible accuracy loss. Compared to other approaches, CirCNN is distinct due to its mathematical rigor: it can converge to the same effectiveness as DNNs without compression. The CirCNN architecture, a universal DNN inference engine that can be implemented on various hardware/software platforms with configurable network architecture. To demonstrate the performance and energy efficiency, we test CirCNN in FPGA, ASIC and embedded processors. Our results show that CirCNN architecture achieves very high energy efficiency and performance with a small hardware footprint. Based on the FPGA implementation and ASIC synthesis results, CirCNN achieves 6-102X energy efficiency improvements compared with the best state-of-the-art results.

cs.CV↗

arXiv2017

Determination of Hund's coupling in 5d Oxides using Resonant Inelastic X-ray Scattering

We report resonant inelastic X-ray scattering (RIXS) measurements on ordered double perovskite samples containing Re$^{5+}$ and Ir$^{5+}$ with $5d^2$ and $5d^4$ electronic configurations respectively. In particular, the observed RIXS spectra of Ba$_2$YReO$_6$ and Sr$_2$MIrO$_6$ (M=Y, Gd) show sharp intra-t$_{2g}$ transitions, which can be quantitatively understood using a minimal `atomic' Hamiltonian incorporating spin-orbit coupling ($λ$) and Hund's coupling ($J_H$). Our analysis yields $λ=0.38(2)$eV with $J_H=0.26(2)$eV for Re$^{5+}$, and $λ=0.42(2)$eV with $J_H=0.25(4)$eV for Ir$^{5+}$. Our results provide the first sharp estimates for the Hund's coupling in $5d$ oxides, and suggest that it should be treated on equal footing with spin-orbit interaction in multi-orbital $5d$ transition metal compounds.

cond-mat.str-el↗

arXiv2017

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.

cs.CV↗

arXiv2016

LLR-based Successive-Cancellation List Decoder for Polar Codes with Multi-bit Decision

Due to their capacity-achieving property, polar codes have become one of the most attractive channel codes. To date, the successive cancellation list (SCL) decoding algorithm is the primary approach that can guarantee outstanding error-correcting performance of polar codes. However, the hardware designs of the original SCL decoder have large silicon area and long decoding latency. Although some recent efforts can reduce either the area or latency of SCL decoders, these two metrics still cannot be optimized at the same time. This paper, for the first time, proposes a general log-likelihood-ratio (LLR)-based SCL decoding algorithm with multi-bit decision. This new algorithm, referred as LLR-2Kb-SCL, can determine 2K bits simultaneously for arbitrary K with the use of LLR messages. In addition, a reduced-data-width scheme is presented to reduce the critical path of the sorting block. Then, based on the proposed algorithm, a VLSI architecture of the new SCL decoder is developed. Synthesis results show that for an example (1024, 512) polar code with list size 4, the proposed LLR-2Kb-SCL decoders achieve significant reduction in both area and latency as compared to prior works. As a result, the hardware efficiency of the proposed designs with K=2 and 3 are 2.33 times and 3.32 times of that of the state-of-the-art works, respectively.

cs.IT↗

arXiv2015

Automatic exploration of structural regularities in networks

Complex networks provide a powerful mathematical representation of complex systems in nature and society. To understand complex networks, it is crucial to explore their internal structures, also called structural regularities. The task of network structure exploration is to determine how many groups in a complex network and how to group the nodes of the network. Most existing structure exploration methods need to specify either a group number or a certain type of structure when they are applied to a network. In the real world, however, not only the group number but also the certain type of structure that a network has are usually unknown in advance. To automatically explore structural regularities in complex networks, without any prior knowledge about the group number or the certain type of structure, we extend a probabilistic mixture model that can handle networks with any type of structure but needs to specify a group number using Bayesian nonparametric theory and propose a novel Bayesian nonparametric model, called the Bayesian nonparametric mixture (BNPM) model. Experiments conducted on a large number of networks with different structures show that the BNPM model is able to automatically explore structural regularities in networks with a stable and state-of-the-art performance.

cs.SI↗

arXiv2015

Successive Cancellation Decoding of Polar Codes using Stochastic Computing

Polar codes have emerged as the most favorable channel codes for their unique capacity-achieving property. To date, numerous works have been reported for efficient design of polar codes decoder. However, these prior efforts focused on design of polar decoders via deterministic computation, while the behavior of stochastic polar decoder, which can have potential advantages such as low complexity and strong error-resilience, has not been studied in existing literatures. This paper, for the first time, investigates polar decoding using stochastic logic. Specifically, the commonly-used successive cancellation (SC) algorithm is reformulated into the stochastic form. Several methods that can potentially improve decoding performance are discussed and analyzed. Simulation results show that a stochastic SC decoder can achieve similar error-correcting performance as its deterministic counterpart. This work can pave the way for future hardware design of stochastic polar codes decoders.

cs.IT↗

arXiv2014

Successive Cancellation List Polar Decoder using Log-likelihood Ratios

Successive cancellation list (SCL) decoding algorithm is a powerful method that can help polar codes achieve excellent error-correcting performance. However, the current SCL algorithm and decoders are based on likelihood or log-likelihood forms, which render high hardware complexity. In this paper, we propose a log-likelihood-ratio (LLR)-based SCL (LLR-SCL) decoding algorithm, which only needs half the computation and storage complexity than the conventional one. Then, based on the proposed algorithm, we develop low-complexity VLSI architectures for LLR-SCL decoders. Analysis results show that the proposed LLR-SCL decoder achieves 50% reduction in hardware and 98% improvement in hardware efficiency.

cs.IT↗