arXiv subjects

Xiao Li

Publications and source records attributed to Xiao Li.

At least 379 records · Page 21Linked to original sources

arXiv2022

AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension

Recent machine reading comprehension datasets such as ReClor and LogiQA require performing logical reasoning over text. Conventional neural models are insufficient for logical reasoning, while symbolic reasoners cannot directly apply to text. To meet the challenge, we present a neural-symbolic approach which, to predict an answer, passes messages over a graph representing logical relations between text units. It incorporates an adaptive logic graph network (AdaLoGN) which adaptively infers logical relations to extend the graph and, essentially, realizes mutual and iterative reinforcement between neural and symbolic reasoning. We also implement a novel subgraph-to-node message passing mechanism to enhance context-option interaction for answering multiple-choice questions. Our approach shows promising results on ReClor and LogiQA.

cs.CL↗

arXiv2022

Stabilized exponential-SAV schemes preserving energy dissipation law and maximum bound principle for the Allen-Cahn type equations

It is well-known that the Allen-Cahn equation not only satisfies the energy dissipation law but also possesses the maximum bound principle (MBP) in the sense that the absolute value of its solution is pointwise bounded for all time by some specific constant under appropriate initial/boundary conditions. In recent years, the scalar auxiliary variable (SAV) method and many of its variants have attracted much attention in numerical solution for gradient flow problems due to their inherent advantage of preserving certain discrete analogues of the energy dissipation law. However, existing SAV schemes usually fail to preserve the MBP when applied to the Allen-Cahn equation. In this paper, we develop and analyze new first- and second-order stabilized exponential-SAV schemes for a class of Allen-Cahn type equations, which are shown to simultaneously preserve the energy dissipation law and MBP in discrete settings. In addition, optimal error estimates for the numerical solutions are rigorously obtained for both schemes. Extensive numerical tests and comparisons are also conducted to demonstrate the performance of the proposed schemes.

math.NA↗

arXiv2022

On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features

When training deep neural networks for classification tasks, an intriguing empirical phenomenon has been widely observed in the last-layer classifiers and features, where (i) the class means and the last-layer classifiers all collapse to the vertices of a Simplex Equiangular Tight Frame (ETF) up to scaling, and (ii) cross-example within-class variability of last-layer activations collapses to zero. This phenomenon is called Neural Collapse (NC), which seems to take place regardless of the choice of loss functions. In this work, we justify NC under the mean squared error (MSE) loss, where recent empirical evidence shows that it performs comparably or even better than the de-facto cross-entropy loss. Under a simplified unconstrained feature model, we provide the first global landscape analysis for vanilla nonconvex MSE loss and show that the (only!) global minimizers are neural collapse solutions, while all other critical points are strict saddles whose Hessian exhibit negative curvature directions. Furthermore, we justify the usage of rescaled MSE loss by probing the optimization landscape around the NC solutions, showing that the landscape can be improved by tuning the rescaling hyperparameters. Finally, our theoretical findings are experimentally verified on practical network architectures.

cs.LG↗

arXiv2022

Generalized SAV-exponential integrator schemes for Allen-Cahn type gradient flows

The energy dissipation law and the maximum bound principle (MBP) are two important physical features of the well-known Allen-Cahn equation. While some commonly-used first-order time stepping schemes have turned out to preserve unconditionally both energy dissipation law and MBP for the equation, restrictions on the time step size are still needed for existing second-order or even higher-order schemes in order to have such simultaneous preservation. In this paper, we develop and analyze novel first- and second-order linear numerical schemes for a class of Allen-Cahn type gradient flows. Our schemes combine the generalized scalar auxiliary variable (SAV) approach and the exponential time integrator with a stabilization term, while the standard central difference stencil is used for discretization of the spatial differential operator. We not only prove their unconditional preservation of the energy dissipation law and the MBP in the discrete setting, but also derive their optimal temporal error estimates under fixed spatial mesh. Numerical experiments are also carried out to demonstrate the properties and performance of the proposed schemes.

math.NA↗

arXiv2022

Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training

Normalization techniques have become a basic component in modern convolutional neural networks (ConvNets). In particular, many recent works demonstrate that promoting the orthogonality of the weights helps train deep models and improve robustness. For ConvNets, most existing methods are based on penalizing or normalizing weight matrices derived from concatenating or flattening the convolutional kernels. These methods often destroy or ignore the benign convolutional structure of the kernels; therefore, they are often expensive or impractical for deep ConvNets. In contrast, we introduce a simple and efficient "Convolutional Normalization" (ConvNorm) method that can fully exploit the convolutional structure in the Fourier domain and serve as a simple plug-and-play module to be conveniently incorporated into any ConvNets. Our method is inspired by recent work on preconditioning methods for convolutional sparse coding and can effectively promote each layer's channel-wise isometry. Furthermore, we show that our ConvNorm can reduce the layerwise spectral norm of the weight matrices and hence improve the Lipschitzness of the network, leading to easier training and improved robustness for deep ConvNets. Applied to classification under noise corruptions and generative adversarial network (GAN), we show that the ConvNorm improves the robustness of common ConvNets such as ResNet and the performance of GAN. We verify our findings via numerical experiments on CIFAR and ImageNet.

cs.CV↗

arXiv2021

Stability of scar states in 2D PXP model against random disorders

Recently a class of quantum systems exhibiting weak ergodicity breaking has attracted much attention. These systems feature a special band of eigenstates called quantum many-body scar states in the energy spectrum. In this work we study the fate of quantum many-body scar states in a two-dimensional lattice against random disorders. We show that in both the square lattice and the honeycomb lattice the scar states can persist up to a finite disorder strength, before eventually being erased by the disorder. We further study the localization properties of the system in the presence of even stronger disorders and show that whether a full localization transition occurs depends on the type of disorder we introduce. Our study thus reveals the fascinating interplay between disorder and quantum many-body scarring in a two-dimensional system.

cond-mat.dis-nn↗

arXiv2021

Reliable Propagation-Correction Modulation for Video Object Segmentation

Error propagation is a general but crucial problem in online semi-supervised video object segmentation. We aim to suppress error propagation through a correction mechanism with high reliability. The key insight is to disentangle the correction from the conventional mask propagation process with reliable cues. We introduce two modulators, propagation and correction modulators, to separately perform channel-wise re-calibration on the target frame embeddings according to local temporal correlations and reliable references respectively. Specifically, we assemble the modulators with a cascaded propagation-correction scheme. This avoids overriding the effects of the reliable correction modulator by the propagation modulator. Although the reference frame with the ground truth label provides reliable cues, it could be very different from the target frame and introduce uncertain or incomplete correlations. We augment the reference cues by supplementing reliable feature patches to a maintained pool, thus offering more comprehensive and expressive object representations to the modulators. In addition, a reliability filter is designed to retrieve reliable patches and pass them in subsequent frames. Our model achieves state-of-the-art performance on YouTube-VOS18/19 and DAVIS17-Val/Test benchmarks. Extensive experiments demonstrate that the correction mechanism provides considerable performance gain by fully utilizing reliable guidance. Code is available at: https://github.com/JerryX1110/RPCMVOS.

cs.CV↗

arXiv2021

BarrierNet: A Safety-Guaranteed Layer for Neural Networks

This paper introduces differentiable higher-order control barrier functions (CBF) that are end-to-end trainable together with learning systems. CBFs are usually overly conservative, while guaranteeing safety. Here, we address their conservativeness by softening their definitions using environmental dependencies without loosing safety guarantees, and embed them into differentiable quadratic programs. These novel safety layers, termed a BarrierNet, can be used in conjunction with any neural network-based controller, and can be trained by gradient descent. BarrierNet allows the safety constraints of a neural controller be adaptable to changing environments. We evaluate them on a series of control problems such as traffic merging and robot navigations in 2D and 3D space, and demonstrate their effectiveness compared to state-of-the-art approaches.

cs.LG↗

arXiv2021

Video Instance Segmentation by Instance Flow Assembly

Instance segmentation is a challenging task aiming at classifying and segmenting all object instances of specific classes. While two-stage box-based methods achieve top performances in the image domain, they cannot easily extend their superiority into the video domain. This is because they usually deal with features or images cropped from the detected bounding boxes without alignment, failing to capture pixel-level temporal consistency. We embrace the observation that bottom-up methods dealing with box-free features could offer accurate spacial correlations across frames, which can be fully utilized for object and pixel level tracking. We first propose our bottom-up framework equipped with a temporal context fusion module to better encode inter-frame correlations. Intra-frame cues for semantic segmentation and object localization are simultaneously extracted and reconstructed by corresponding decoders after a shared backbone. For efficient and robust tracking among instances, we introduce an instance-level correspondence across adjacent frames, which is represented by a center-to-center flow, termed as instance flow, to assemble messy dense temporal correspondences. Experiments demonstrate that the proposed method outperforms the state-of-the-art online methods (taking image-level input) on the challenging Youtube-VIS dataset.

cs.CV↗

arXiv2021

Modeling and Measurements for Multi-path Mitigation with Reconfigurable Intelligent Surfaces

A reconfigurable intelligent surface (RIS) is capable of manipulating electromagnetic waves with its flexibly configurable unit cells, thus is an appealing technology to resist fast fading caused by multi-path in wireless communications. In this paper, a two-path propagation model for RIS-assisted wireless communications is proposed by considering both the direct path from the transmitter to the receiver and the assisted path provided by the RIS. The proposed propagation model unveils that the phase shifts of RISs can be optimized by appropriate configuration for multi-path fading mitigation. In particular, four types of RISs with different configuration capabilities are introduced and their performances on improving received signal power in virtue of the assisted path to resist fast fading are compared through extensive simulation results. In addition, an RIS operating at 35 GHz is used for experimental measurement. The experimental results verify that an RIS has the ability to combat fast fading and thus improves the receiving performance, which may lay a foundation for further researches.

cs.IT↗

arXiv2021

Asymmetrical Uplink and Downlink Transceivers in Massive MIMO Systems

Even if massive multiple-input multiple-output (MIMO) can theoretically bring huge benefits, it incurs substantial hardware complexity and expensive hardware costs. To address these issues while maintaining the system performance simultaneously, we develop an asymmetrical transceiver architecture for massive MIMO systems in this paper by releasing the shackles on radio frequency (RF) chains. Specifically, we first develop the architecture for the asymmetrical transceiver where the number of receive RF chains is different from that of the transmit RF chains. Due to this unequal number of RF chains, channel inconsistency appears. To overcome the channel inconsistency and thus fully harness the system's downlink data transmission capability, we propose two uplink-to-downlink channel transfer algorithms. The cost and power consumption models for the asymmetrical transceiver are also developed and the uplink signal-to-noise loss due to the channel inconsistency is investigated. Through analyzing system spectral, cost, and energy efficiency, we demonstrate that the proposed asymmetrical transceiver-based massive MIMO system can achieve excellent downlink spectral efficiency while maintaining a reasonable energy efficiency.

cs.IT↗

arXiv2021

Reversible modulation of metal-insulator transition in VO2 via chemically-induced oxygen migration

Metal-insulator transitions (MIT),an intriguing correlated phenomenon induced by the subtle competition of the electrons' repulsive Coulomb interaction and kinetic energy, is of great potential use for electronic applications due to the dramatic change in resistivity. Here, we demonstrate a reversible control of MIT in VO2 films via oxygen stoichiometry engineering. By facilely depositing and dissolving a water-soluble yet oxygen-active Sr3Al2O6 capping layer atop the VO2 at room temperature, oxygen ions can reversibly migrate between VO2 and Sr3Al2O6, resulting in a gradual suppression and a complete recovery of MIT in VO2. The migration of the oxygen ions is evidenced in a combination of transport measurement, structural characterization and first-principles calculations. This approach of chemically-induced oxygen migration using a water-dissolvable adjacent layer could be useful for advanced electronic and iontronic devices and studying oxygen stoichiometry effects on the MIT.

cond-mat.mtrl-sci↗

arXiv2021

Attention-Based 3D Seismic Fault Segmentation Training by a Few 2D Slice Labels

Detection faults in seismic data is a crucial step for seismic structural interpretation, reservoir characterization and well placement. Some recent works regard it as an image segmentation task. The task of image segmentation requires huge labels, especially 3D seismic data, which has a complex structure and lots of noise. Therefore, its annotation requires expert experience and a huge workload. In this study, we present lambda-BCE and lambda-smooth L1loss to effectively train 3D-CNN by some slices from 3D seismic data, so that the model can learn the segmentation of 3D seismic data from a few 2D slices. In order to fully extract information from limited data and suppress seismic noise, we propose an attention module that can be used for active supervision training and embedded in the network. The attention heatmap label is generated by the original label, and letting it supervise the attention module using the lambda-smooth L1loss. The experiment demonstrates the effectiveness of our loss function, the method can extract 3D seismic features from a few 2D slice labels. And it also shows the advanced performance of the attention module, which can significantly suppress the noise in the seismic data while increasing the model's sensitivity to the foreground. Finally, on the public test set, we only use the 2D slice labels training that accounts for 3.3% of the 3D volume label, and achieve similar performance to the 3D volume label training.

cs.CV↗

arXiv2021

High fidelity entanglement of neutral atoms via a Rydberg-mediated single-modulated-pulse controlled-PHASE gate

Neutral atom platform has become an attractive choice to study the science of quantum information and quantum simulation, where intense efforts have been devoted to the entangling processes between individual atoms. For the development of this area, two-qubit controlled-PHASE gate via Rydberg blockade is one of the most essential elements. Recent theoretical studies have suggested the advantages of introducing non-trivial waveform modulation into the gate protocol, which is anticipated to improve its performance towards the next stage. We report our recent experimental results in realizing a two-qubit controlled-PHASE($C_Z$) gate via off-resonant modulated driving(ORMD) embedded in two-photon transition for Rb atoms. It relies upon a single modulated driving pulse with a carefully calculated smooth waveform to gain the appropriate phase accumulations required by the two-qubit gate. Combining this $C_Z$ gate with global microwave pulses, two-atom entanglement is generated with the raw fidelity of 0.945(6). Accounting for state preparation and measurement (SPAM) errors, we extract the entanglement operation fidelity to be 0.980(7). Our work features completing the $C_Z$ gate operation within a single pulse to avoid shelved Rydberg population, thus demonstrate another promising route for realizing high-fidelity two-qubit gate for neutral atom platform.

quant-ph↗

arXiv2021

Experimental demonstration of the short bunch extraction by bunch rotation in a high-intensity rapid cycling proton synchrotron

Short bunch proton beams are of great significance for the applications of white neutron beams and muon beams. The accelerator complex of China Spallation Neutron Source (CSNS) was designed to support the applications mainly based on neutron scattering techniques where the proton pulse length is not very sensitive. Some theoretical and experimental studies have been performed to see if one can extract a short-bunch proton beam by bunch rotation from the rapid cycling synchrotron (RCS) at CSNS. The experimental results at RCS have evidently displayed the bunch lengthening and rotation process, which demonstrates the effectiveness of this method even with a very short available time for the RF gymnastic processes and a high-intensity beam. With a beam power of 50 kW and normal longitudinal emittance at the injection, the proton beam with a bunch length of about 53% with respect to the one in the normal operation mode was obtained and transported to the spallation target. With a reduced longitudinal emittance at injection and the beam power of 30 kW, the shortest extraction bunch length obtained is about 26% of the one in the normal operation mode. Different machine settings have also been tested to show the impact of the desynchronization between the RF and magnetic fields, the influence of the non-adiabatic risetime and the adiabatic decay time of the RF voltage on the extraction bunch length. The experimental results are well consistent with the theoretical and simulated ones. It is interesting to observe that space charge has a beneficial effect on the bunch lengthening which will result in a shorter bunch at the extraction with the later bunch rotation. The controlled desynchronization method between the RF and magnetic fields in an RCS was also proven successful.

physics.acc-ph↗

arXiv2021

Enabling Plug-and-Play and Crowdsourcing SLAM in Wireless Communication Systems

Simultaneous localization and mapping (SLAM) during communication is emerging. This technology promises to provide information on propagation environments and transceivers' location, thus creating several new services and applications for the Internet of Things and environment-aware communication. Using crowdsourcing data collected by multiple agents appears to be much potential for enhancing SLAM performance. However, the measurement uncertainties in practice and biased estimations from multiple agents may result in serious errors. This study develops a robust SLAM method with measurement plug-and-play and crowdsourcing mechanisms to address the above problems. First, we divide measurements into different categories according to their unknown biases and realize a measurement plug-and-play mechanism by extending the classic belief propagation (BP)-based SLAM method. The proposed mechanism can obtain the time-varying agent location, radio features, and corresponding measurement biases (such as clock bias, orientation bias, and received signal strength model parameters), with high accuracy and robustness in challenging scenarios without any prior information on anchors and agents. Next, we establish a probabilistic crowdsourcing-based SLAM mechanism, in which multiple agents cooperate to construct and refine the radio map in a decentralized manner. Our study presents the first BP-based crowdsourcing that resolves the "double count" and "data reliability" problems through the flexible application of probabilistic data association methods. Numerical results reveal that the crowdsourcing mechanism can further improve the accuracy of the mapping result, which, in turn, ensures the decimeter-level localization accuracy of each agent in a challenging propagation environment.

cs.IT↗

arXiv2021

Unconditionally stable exponential time differencing schemes for the mass-conserving Allen-Cahn equation with nonlocal and local effects

It is well known that the classic Allen-Cahn equation satisfies the maximum bound principle (MBP), that is, the absolute value of its solution is uniformly bounded for all time by certain constant under suitable initial and boundary conditions. In this paper, we consider numerical solutions of the modified Allen-Cahn equation with a Lagrange multiplier of nonlocal and local effects, which not only shares the same MBP as the original Allen-Cahn equation but also conserves the mass exactly. We reformulate the model equation with a linear stabilizing technique, then construct first- and second-order exponential time differencing schemes for its time integration. We prove the unconditional MBP preservation and mass conservation of the proposed schemes in the time discrete sense and derive their error estimates under some regularity assumptions. Various numerical experiments in two and three dimensions are also conducted to verify the theoretical results.

math.NA↗

arXiv2021

Whiteout: when do fixed-X knockoffs fail?

A core strength of knockoff methods is their virtually limitless customizability, allowing an analyst to exploit machine learning algorithms and domain knowledge without threatening the method's robust finite-sample false discovery rate control guarantee. While several previous works have investigated regimes where specific implementations of knockoffs are provably powerful, general negative results are more difficult to obtain for such a flexible method. In this work we recast the fixed-$X$ knockoff filter for the Gaussian linear model as a conditional post-selection inference method. It adds user-generated Gaussian noise to the ordinary least squares estimator $\hatβ$ to obtain a "whitened" estimator $\widetildeβ$ with uncorrelated entries, and performs inference using $\text{sgn}(\widetildeβ_j)$ as the test statistic for $H_j:\; β_j = 0$. We prove equivalence between our whitening formulation and the more standard formulation involving negative control predictor variables, showing how the fixed-$X$ knockoffs framework can be used for multiple testing on any problem with (asymptotically) multivariate Gaussian parameter estimates. Relying on this perspective, we obtain the first negative results that universally upper-bound the power of all fixed-$X$ knockoff methods, without regard to choices made by the analyst. Our results show roughly that, if the leading eigenvalues of $\text{Var}(\hatβ)$ are large with dense leading eigenvectors, then there is no way to whiten $\hatβ$ without irreparably erasing nearly all of the signal, rendering $\text{sgn}(\widetildeβ_j)$ too uninformative for accurate inference. We give conditions under which the true positive rate (TPR) for any fixed-$X$ knockoff method must converge to zero even while the TPR of Bonferroni-corrected multiple testing tends to one, and we explore several examples illustrating this phenomenon.

math.ST↗