arXiv subjects

Lei Wu

Publications and source records attributed to Lei Wu.

At least 235 records · Page 13Linked to original sources

arXiv2022

Learning a Single Neuron for Non-monotonic Activation Functions

We study the problem of learning a single neuron $\mathbf{x}\mapsto σ(\mathbf{w}^T\mathbf{x})$ with gradient descent (GD). All the existing positive results are limited to the case where $σ$ is monotonic. However, it is recently observed that non-monotonic activation functions outperform the traditional monotonic ones in many applications. To fill this gap, we establish learnability without assuming monotonicity. Specifically, when the input distribution is the standard Gaussian, we show that mild conditions on $σ$ (e.g., $σ$ has a dominating linear part) are sufficient to guarantee the learnability in polynomial time and polynomial samples. Moreover, with a stronger assumption on the activation function, the condition of input distribution can be relaxed to a non-degeneracy of the marginal distribution. We remark that our conditions on $σ$ are satisfied by practical non-monotonic activation functions, such as SiLU/Swish and GELU. We also discuss how our positive results are related to existing negative results on training two-layer neural networks.

stat.ML↗

arXiv2022

Logarithmic cotangent bundles, Chern-Mather classes, and the Huh-Sturmfels Involution conjecture

Using compactifications in the logarithmic cotangent bundle, we obtain a formula for the Chern classes of the pushforward of Lagrangian cycles under an open embedding with normal crossing complement. This generalizes earlier results of Aluffi and Wu-Zhou. The first application of our formula is a geometric description of Chern-Mather classes of an arbitrary very affine variety, generalizing earlier results of Huh which held under the smooth and schon assumptions. As the second application, we confirm an involution formula relating sectional maximum likelihood (ML) degrees and ML bidegrees, which was conjectured by Huh and Sturmfels in 2013.

math.AG↗

arXiv2022

On the emergence of heat waves in the transient thermal grating geometry

The propagation of heat in the transient thermal grating geometry is studied based on phonon Boltzmann transport equation (BTE) in different phonon transport regimes. Our analytical and numerical results show that the phonon dispersion relation and temperature play a significant role in the emergence of heat wave. For the frequency-independent BTE, the heat wave appears as long as the phonon resistive scattering is not sufficient, while for the frequency-dependent BTE, the heat wave could disappear in the ballistic regime, depending on the grating period and temperature. We predict that the heat wave could appear in the suspended graphene and silicon in extremely low temperature but disappear at room temperature.

cond-mat.mes-hall↗

arXiv2022

A kinetic model for rarefied flows of molecular gas with vibrational modes

A kinetic model is proposed for rarefied flows of molecular gas with rotational and temperature-dependent vibrational degrees of freedom. The model reduces to the Boltzmann equation for monatomic gas when the energy exchange between the translational and internal modes is absent, thus the influence of intermolecular potential can be captured. Moreover, not only the transport coefficients but also their fundamental relaxation processes are recovered. The accuracy of our kinetic model is validated by the direct simulation Monte Carlo method in several rarefied gas flows, including the shock wave, Fourier flow, Couette flow, and the creep flow driven by Maxwell's demon. Then the kinetic model is adopted to investigate thermally-induced flows. By adjusting the viscosity index in the Boltzmann collision operator, we find that the intermolecular potential significantly influences the velocity and Knudsen force. Interestingly, in the transition flow regime, the Knudsen force exerting on a heated beam could reverse the direction when the viscosity index changes from 0.5 (hard-sphere gas) to 1 (Maxwell gas). This discovery is useful in the design of micro-electromechanical systems for microstructure actuation and gas sensing.

physics.flu-dyn↗

arXiv2021

Direct Detection of Spin-Dependent Sub-GeV Dark Matter via Migdal Effect

Motivated by the current strong constraints on the spin-independent dark matter (DM)-nucleus scattering, we investigate the spin-dependent (SD) interactions of the light Majorana DM with the nucleus mediated by an axial-vector boson. Due to the small nucleus recoil energy, the ionization signals have now been used to probe the light dark matter particles in direct detection experiments. With the existing ionization data, we derive the exclusion limits on the SD DM-nucleus scattering through Migdal effect in the MeV-GeV DM mass range. It is found that the lower limit of the DM mass can reach about several MeVs. Due to the momentum transfer correction induced by the light mediator, the bounds on the SD DM-nucleus scattering cross sections can be weakened in comparison with the heavy mediator.

hep-ph↗

arXiv2021

Detecting an axion-like particle with machine learning at the LHC

Axion-like particles (ALPs) appear in various new physics models with spontaneous global symmetry breaking. When the ALP mass is in the range of MeV to GeV, the cosmology and astrophysics bounds are so far quite weak. In this work, we investigate such light ALPs through the ALP-strahlung production processes $pp \to W^\pm a, Z a$ with the sequential decay $a \to γγ$ at the 14 TeV LHC with an integrated luminosity of 3000 fb$^{-1}$ (HL-LHC). Building on the concept of jet image which uses calorimeter towers as the pixels of the image and measures a jet as an image, we investigate the potential of machine learning techniques based on convolutional neural network (CNN) to identify the highly boosted ALPs which decay to a pair of highly collimated photons. With the CNN tagging algorithm, we demonstrate that our approach can extend current LHC sensitivity and probe the ALP mass range from 0.3~GeV to 5~GeV. The obtained bounds are stronger than the existing limits on the ALP-photon coupling.

hep-ph↗

arXiv2021

Photon-jet events as a probe of axion-like particles at the LHC

Axion-like particles (ALPs) are predicted by many extensions of the Standard Model (SM). When ALP mass lies in the range of MeV to GeV, the cosmology and astrophysics will be largely irrelevant. In this work, we investigate such light ALPs through the ALP-strahlung process $pp \to V a (\to γγ)$ at the 14 TeV LHC with an integrated luminosity of 3000 fb$^{-1}$ (HL-LHC). With the photon-jet algorithm, we demonstrate that our approach can probe the mass range of ALPs, which is inaccessible to previous LHC experiments. The obtained result can surpass the existing limits on ALP-photon coupling in the ALP mass range from 0.3 GeV to 10 GeV.

hep-ph↗

arXiv2021

Blockchain Enabled Secure Authentication for Unmanned Aircraft Systems

The integration of air and ground smart vehicles is becoming a new paradigm of future transportation. A decent number of smart unmanned vehicles or UAS will be sharing the national airspace for various purposes, such as express delivery, surveillance, etc. However, the proliferation of UAS also brings challenges considering the safe integration of them into the current Air Traffic Management (ATM) systems. Especially when the current Automatic Dependent Surveillance Broadcasting (ADS-B) systems do not have message authentication mechanisms, it can not distinguish whether an authorized UAS is using the corresponding airspace. In this paper, we aim to address these practical challenges in two folds. We first use blockchain to provide a secure authentication platform for flight plan approval and sharing between the existing ATM facilities. We then use the fountain code to encode the authentication payloads and adapt them into the de facto communication protocol of ATM. This maintains backward compatibility and ensures the verification success rate under the noisy broadcasting channel. We simulate the realistic wireless communication scenarios and theoretically prove that our proposed authentication framework is with low latency and highly compatible with existing ATM communication protocols.

cs.CR↗

arXiv2021

Understanding the Evolution of Blockchain Ecosystems: A Longitudinal Measurement Study of Bitcoin, Ethereum, and EOSIO

The continuing expansion of the blockchain ecosystems has attracted much attention from the research community. However, although a large number of research studies have been proposed to understand the diverse characteristics of individual blockchain systems (e.g., Bitcoin or Ethereum), little is known at a comprehensive level on the evolution of blockchain ecosystems at scale, longitudinally, and across multiple blockchains. We argue that understanding the dynamics of blockchain ecosystems could provide unique insights that cannot be achieved through studying a single static snapshot or a single blockchain network alone. Based on billions of transaction records collected from three representative and popular blockchain systems (Bitcoin, Ethereum and EOSIO) over 10 years, we conduct the first study on the evolution of multiple blockchain ecosystems from different perspectives. Our exploration suggests that, although the overall blockchain ecosystem shows promising growth over the last decade, a number of worrying outliers exist that have disrupted its evolution.

cs.CR↗

arXiv2021

ECMO: Peripheral Transplantation to Rehost Embedded Linux Kernels

Dynamic analysis based on the full-system emulator QEMU is widely used for various purposes. However, it is challenging to run firmware images of embedded devices in QEMU, especially the process to boot the Linux kernel (we call this process rehosting the Linux kernel.) That's because embedded devices usually use different system-on-chips (SoCs) from multiple vendors and only a limited number of SoCs are currently supported in QEMU. In this work, we propose a technique called peripheral transplantation. The main idea is to transplant the device drivers of designated peripherals into the Linux kernel. By doing so, it can replace the peripherals in the kernel that are currently unsupported in QEMU with supported ones, thus making the Linux kernel rehostable. After that, various applications can be built upon. We implemented this technique inside a prototype system called ECMO and applied it to 815 firmware images, which consist of 20 kernel versions, 37 device models, and 24 vendors. The result shows that ECMO can successfully transplant peripherals for all the 815 Linux kernels. Among them,710 kernels can be successfully rehosted, i.e., launching a user-space shell (87.1% success rate). The failed cases are mainly because the root file system format (ramfs) is not supported by the kernel. We further build three applications, i.e., kernel crash analysis, rootkit forensic analysis, and kernel fuzzing, based on the rehosted kernels to demonstrate the usage scenarios of ECMO

cs.AR↗

arXiv2021

A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms

The dynamic behavior of RMSprop and Adam algorithms is studied through a combination of careful numerical experiments and theoretical explanations. Three types of qualitative features are observed in the training loss curve: fast initial convergence, oscillations, and large spikes in the late phase. The sign gradient descent (signGD) flow, which is the limit of Adam when taking the learning rate to 0 while keeping the momentum parameters fixed, is used to explain the fast initial convergence. For the late phase of Adam, three different types of qualitative patterns are observed depending on the choice of the hyper-parameters: oscillations, spikes, and divergence. In particular, Adam converges much smoother and even faster when the values of the two momentum factors are close to each other. This observation is particularly important for scientific computing tasks, for which the training process usually proceeds into the high precision regime.

cs.LG↗

arXiv2021

Efficient Medical Image Segmentation Based on Knowledge Distillation

Recent advances have been made in applying convolutional neural networks to achieve more precise prediction results for medical image segmentation problems. However, the success of existing methods has highly relied on huge computational complexity and massive storage, which is impractical in the real-world scenario. To deal with this problem, we propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network. This architecture empowers the lightweight network to get a significant improvement on segmentation capability while retaining its runtime efficiency. We further devise a novel distillation module tailored for medical image segmentation to transfer semantic region information from teacher to student network. It forces the student network to mimic the extent of difference of representations calculated from different tissue regions. This module avoids the ambiguous boundary problem encountered when dealing with medical imaging but instead encodes the internal information of each semantic region for transferring. Benefited from our module, the lightweight network could receive an improvement of up to 32.6% in our experiment while maintaining its portability in the inference phase. The entire structure has been verified on two widely accepted public CT datasets LiTS17 and KiTS19. We demonstrate that a lightweight network distilled by our method has non-negligible value in the scenario which requires relatively high operating speed and low storage usage.

eess.IV↗

arXiv2021

Automatically Locating ARM Instructions Deviation between Real Devices and CPU Emulators

Emulator is widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systemsand architectures. However, whether the emulator is consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target ARM architecture, which provides machine readable specification. Based on the specification, we propose a test case generator by designing and implementing the first symbolic execution engine for ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing with these instruction streams between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7-a, and ARMv8-a) and the state-of-the-art emulators (i.e., QEMU). We locate 155,642 inconsistent instruction streams, which cover 30% of all instruction encodings and 47.8% of the instructions. We find undefined implementation in ARM manual and implementation bugs of QEMU are the major causes of inconsistencies. Furthermore, we discover four QEMU bugs, which are confirmed and patched by thedevelopers, covering 13 instruction encodings including the most commonly used ones (e.g.,STR,BLX). With the inconsistent instructions, we build three security applications and demonstrate thecapability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing.

cs.CR↗

arXiv2021

Probing triple Higgs coupling with machine learning at the LHC

Measuring the triple Higgs coupling is a crucial task in the LHC and future collider experiments. We apply the Message Passing Neural Network (MPNN) to the study of the non-resonant Higgs pair production process $pp \to hh$ in the final state with $2b + 2\ell + E_{\rm T}^{\rm miss}$ at the LHC. Although the MPNN can improve the signal significance, it is still challenging to observe such a process at the LHC. We find that a $2σ$ upper bound (including a 10\% systematic uncertainty) on the production cross section of the Higgs pair is 3.7 times the predicted SM cross section at the LHC with the luminosity of 3000 fb$^{-1}$, which will limit the triple Higgs coupling to the range of $[-3,11.5]$.

hep-ph↗

arXiv2021

GUT-scale constrained SUSY in light of E989 muon g-2 measurement

The new FNAL result of the muon $g-2$, combined with the BNL result, shows a 4.2$σ$ deviation from the SM. We use the new data of the muon $g-2$ to revisit several GUT-scale constrained SUSY models with the constraints from the LHC searches, the dark matter detection, the flavor data and the electroweak vacuum stability. We first demonstrate the tension between the muon $g-2$ and other experimental measurements in the CMSSM/mSUGRA. Then after discussing the possible ways to alleviate such a tension and showing the muon $g-2$ in pMSSM under relevant experimental constraints, we survey several extensions of the CMSSM/mSUGRA with different types of universal boundary conditions at the GUT scale. Finally, we briefly discuss the muon $g-2$ in other popular SUSY breaking mechanisms, namely the GMSB and AMSB mechanisms and their extensions.

hep-ph↗

arXiv2021

A common origin of muon g-2 anomaly, Galaxy Center GeV excess and AMS-02 anti-proton excess in the NMSSM

The supersymmetric model is one of the most attractive extensions of the Standard Model of particle physics. In light of the most recently reported anomaly of the muon g-2 measurement by the FermiLab E989 experiment, and the excesses of gamma rays at the Galactic center observed by Fermi-LAT space telescope, as well as the antiproton excess observed by the Alpha Magnetic Spectrometer, we propose to account for all these anomalies or excesses in the Next-to-Minimal Supersymmetric Standard Model. Considering various experimental constraints including the Higgs mass, B-physics, collider data, dark matter relic density and direct detections, we find that a $\sim 60$ GeV bino-like neutralino is able to successfully explain all these observations. Our scenario can be sensitively probed by future direct detection experiments.

hep-ph↗

arXiv2021

A fast-converging scheme for the Phonon Boltzmann equation with dual relaxation times

Callaway's dual relaxation times model, which takes into account the normal and resistive scatterings of phonon, is used to describe the heat conduction in materials like graphene. For steady-state problems, the Callaway model is usually solved by the conventional iterative scheme (CIS), which is efficient in the ballistic regime, but inefficient in the diffusive/hydrodynamic regime. In this paper, a general synthetic iterative scheme (GSIS) is proposed to expedite the convergence to steady-state solutions. First, macroscopic synthetic equations are designed to guide the evolution of equilibrium distribution functions for normal and resistive scatterings, so that fast convergence can be achieved even in the diffusive/hydrodynamic regime. Second, the Fourier stability analysis is conducted to find the convergence rate for both CIS and GSIS, which rigorously proves the efficiency of GSIS over CIS. Finally, several numerical simulations are carried out to demonstrate the accuracy and efficiency of GSIS, where up to three orders of magnitude of convergence acceleration is achieved.

physics.comp-ph↗

arXiv2021

Self-Supervised Multimodal Domino: in Search of Biomarkers for Alzheimer's Disease

Sensory input from multiple sources is crucial for robust and coherent human perception. Different sources contribute complementary explanatory factors. Similarly, research studies often collect multimodal imaging data, each of which can provide shared and unique information. This observation motivated the design of powerful multimodal self-supervised representation-learning algorithms. In this paper, we unify recent work on multimodal self-supervised learning under a single framework. Observing that most self-supervised methods optimize similarity metrics between a set of model components, we propose a taxonomy of all reasonable ways to organize this process. We first evaluate models on toy multimodal MNIST datasets and then apply them to a multimodal neuroimaging dataset with Alzheimer's disease patients. We find that (1) multimodal contrastive learning has significant benefits over its unimodal counterpart, (2) the specific composition of multiple contrastive objectives is critical to performance on a downstream task, (3) maximization of the similarity between representations has a regularizing effect on a neural network, which can sometimes lead to reduced downstream performance but still reveal multimodal relations. Results show that the proposed approach outperforms previous self-supervised encoder-decoder methods based on canonical correlation analysis (CCA) or the mixture-of-experts multimodal variational autoEncoder (MMVAE) on various datasets with a linear evaluation protocol. Importantly, we find a promising solution to uncover connections between modalities through a jointly shared subspace that can help advance work in our search for neuroimaging biomarkers.

cs.LG↗