arXiv Science⌕ Search

arXiv subjects

Chen Liu

Publications and source records attributed to Chen Liu.

At least 217 records · Page 12Linked to original sources
arXiv2022

Fast Adversarial Training with Adaptive Step Size

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to accelerate adversarial training is to substitute multi-step attacks (e.g., PGD) with single-step attacks (e.g., FGSM). However, these single-step methods suffer from catastrophic overfitting, where the accuracy against PGD attack suddenly drops to nearly 0% during training, destroying the robustness of the networks. In this work, we study the phenomenon from the perspective of training instances. We show that catastrophic overfitting is instance-dependent and fitting instances with larger gradient norm is more likely to cause catastrophic overfitting. Based on our findings, we propose a simple but effective method, Adversarial Training with Adaptive Step size (ATAS). ATAS learns an instancewise adaptive step size that is inversely proportional to its gradient norm. The theoretical analysis shows that ATAS converges faster than the commonly adopted non-adaptive counterparts. Empirically, ATAS consistently mitigates catastrophic overfitting and achieves higher robust accuracy on CIFAR10, CIFAR100 and ImageNet when evaluated on various adversarial budgets.

cs.LG↗
arXiv2022

Exploring Structural Sparsity of Deep Networks via Inverse Scale Spaces

The great success of deep neural networks is built upon their over-parameterization, which smooths the optimization landscape without degrading the generalization ability. Despite the benefits of over-parameterization, a huge amount of parameters makes deep networks cumbersome in daily life applications. Though techniques such as pruning and distillation are developed, they are expensive in fully training a dense network as backward selection methods, and there is still a void on systematically exploring forward selection methods for learning structural sparsity in deep networks. To fill in this gap, this paper proposes a new approach based on differential inclusions of inverse scale spaces, which generate a family of models from simple to complex ones along the dynamics via coupling a pair of parameters, such that over-parameterized deep models and their structural sparsity can be explored simultaneously. This kind of differential inclusion scheme has a simple discretization, dubbed Deep structure splitting Linearized Bregman Iteration (DessiLBI), whose global convergence in learning deep networks could be established under the Kurdyka-Lojasiewicz framework. Experimental evidence shows that our method achieves comparable and even better performance than the competitive optimizers in exploring the sparse structure of several widely used backbones on the benchmark datasets. Remarkably, with early stopping, our method unveils `winning tickets' in early epochs: the effective sparse network structures with comparable test accuracy to fully trained over-parameterized models, that are further transferable to similar alternative tasks. Furthermore, our method is able to grow networks efficiently with adaptive filter configurations, demonstrating a good performance with much less computational cost. Codes and models can be downloaded at {https://github.com/DessiLBI2020/DessiLBI}.

cs.LG↗
arXiv2022

Time-coded Spiking Fourier Transform in Neuromorphic Hardware

After several decades of continuously optimizing computing systems, the Moore's law is reaching itsend. However, there is an increasing demand for fast and efficient processing systems that can handlelarge streams of data while decreasing system footprints. Neuromorphic computing answers thisneed by creating decentralized architectures that communicate with binary events over time. Despiteits rapid growth in the last few years, novel algorithms are needed that can leverage the potential ofthis emerging computing paradigm and can stimulate the design of advanced neuromorphic chips.In this work, we propose a time-based spiking neural network that is mathematically equivalent tothe Fourier transform. We implemented the network in the neuromorphic chip Loihi and conductedexperiments on five different real scenarios with an automotive frequency modulated continuouswave radar. Experimental results validate the algorithm, and we hope they prompt the design of adhoc neuromorphic chips that can improve the efficiency of state-of-the-art digital signal processorsand encourage research on neuromorphic computing for signal processing.

cs.NE↗
arXiv2022

Neural Inertial Localization

This paper proposes the inertial localization problem, the task of estimating the absolute location from a sequence of inertial sensor measurements. This is an exciting and unexplored area of indoor localization research, where we present a rich dataset with 53 hours of inertial sensor data and the associated ground truth locations. We developed a solution, dubbed neural inertial localization (NILoc) which 1) uses a neural inertial navigation technique to turn inertial sensor history to a sequence of velocity vectors; then 2) employs a transformer-based neural architecture to find the device location from the sequence of velocities. We only use an IMU sensor, which is energy efficient and privacy preserving compared to WiFi, cameras, and other data sources. Our approach is significantly faster and achieves competitive results even compared with state-of-the-art methods that require a floorplan and run 20 to 30 times slower. We share our code, model and data at https://sachini.github.io/niloc.

cs.RO↗
arXiv2022

The Fate of Shear-Oscillated Amorphous Solids

The behavior of shear-oscillated amorphous materials is studied using a coarse-grained model. Samples are prepared at different degrees of annealing and then subject to athermal and quasistatic oscillatory deformations at various fixed amplitudes. The steady-state reached after several oscillations is fully determined by the initial preparation and the oscillation amplitude, as seen from stroboscopic stress and energy measurements. Under small oscillations, poorly annealed materials display shear-annealing, while ultra-stabilized materials are insensitive to them. Yet, beyond a critical oscillation amplitude, both kind of materials display a discontinuous transition to the same mixed state composed by a fluid shear-band embedded in a marginal solid. Quantitative relations between uniform shear and the steady-state reached with this protocol are established. The transient regime characterizing the growth and the motion of the shear band is also studied.

cond-mat.soft↗
arXiv2021

Facet Dependent Topological Phase Transition in Bi4Br4

The realization of the coexistence of various topologically nontrivial surface states in one material is expected to lay a foundation for new electric applications with selective robust spin current. Here we apply the magnetoconductivity characteristic and angle-resolved photoemission spectroscopy (ARPES) to visualize the surface-selected electronic features evolution of quasi-one-dimensional material Bi4Br4. The transport measurements indicate the quantum interference correction to conductivity possesses symbolic spin rotational characteristic correlated to the value of Berry phase with the effects of weak localization and weak antilocalization for (001) and (100) surfaces, respectively. The ARPES spectra provide the experimental evidence for quasi-one-dimensional massless Dirac surface state at the side (100) surface and anisotropic massive Dirac surface state at the top (001) surface, respectively, which is highly coincide with the angle-dependent scaling behavior of magnetoconductivity. Our results reveal the facet dependent topological phases in quasi-one-dimensional Bi4Br4, stimulating the further investigations of this dual topology classes and the applications of the feasible technologies of topological spintronics.

cond-mat.mtrl-sci↗
arXiv2021

Multi-objective Scheduling of Electric Vehicle Charging/Discharging with Time of Use Tariff

The increased uptake of electric vehicles (EVs) leads to increased demand for electricity, and sometimes pressure on power grids. Uncoordinated charging of EVs may result in stress on distribution networks, and often some form of optimization is required in the charging process. Optimal coordinated charging is a multi-objective optimization problem (MOOP) in nature, with objective functions such as minimum price charging and minimum disruptions to the grid. In this manuscript, we propose a general multi-objective EV charging/discharging schedule (MOEVCS) framework, where the time of use (TOU) tariff is designed according to the load request at each time stamp. To obtain the optimal scheduling scheme and balance the competing benefits from different stakeholders, such as EV owners, EV charging stations (EVCS), and the grid operator, we design three competing objective functions including EV owner cost, EVCS profit, and the network impact. Moreover, we create four application scenarios with different charging request distributions over the investigated periods. Due to different types of decision variables in this MOOP, we develop a constraint mixed-variable multi-objective evolutionary algorithm (MVMOEA) to implement the proposed MOEVCS framework. Our results demonstrate the effectiveness of MOEVCS in making a balance between three competing objectives.

eess.SY↗
arXiv2021

Large-Gap Quantum Spin Hall State and Temperature-Induced Lifshitz Transition in Bi4Br4

Searching for new quantum spin Hall insulators with large fully opened energy gap to overcome the thermal disturbance at room temperature has attracted tremendous attention due to the one-dimensional (1D) spin-momentum locked topological edge states serving as dissipationless channels for the practical applications in low consumption electronics and high performance spintronics. Here, we report the investigation of topological nature of monolayer Bi4Br4 by the techniques of scanning tunneling microscopy and angle-resolved photoemission spectroscopy (ARPES). The topological non-triviality of 1D edge state integrals within the large bulk energy gap (~ 0.2 eV) is revealed by the first-principle calculations. The ARPES measurements at different temperature show a temperature-induced Lifshitz transition, corresponding to the resistivity anomaly caused by the shift of chemical potential. The connection between the emergency of superconductivity and the Lifshitz transition is discussed.

cond-mat.mtrl-sci↗
arXiv2021

The sub-millimetre non-uniformity measurement of residual and coil-generated field in the magnetic shield using atomic vapor cell

Magnetic field source localization and imaging happen at different scales. The sensing baseline ranges from meter scale such as magnetic anomaly detection, centimeter scale such as brain field imaging to nanometer scale such as the imaging of magnetic skyrmion and single cell. Here we show how atomic vapor cell can be used to realize a baseline of 109.6 μm with a magnetic sensitivity of 10pT/sqrt(Hz)@0.6-100Hz and a dynamic range of 2062-4124nT.We use free induction decay (FID) scheme to suppress low-frequency noise and avoid scale factor variation for different domains due to light non-uniformity. The measurement domains are scanned by digital micro-mirror device (DMD). The currents of 22mA, 30mA, 38mA and 44mA are applied in the coils to generate different fields along the pumping axis which are measured respectively by fitting the FID signals of the probe light. The residual fields of every domain are obtained from the intercept of linearly-fitting of the measurement data corresponding to these four currents. The coil-generated fields are calculated by deducting the residual fields from the total fields. The results demonstrate that the hole of shield affects both the residual and the coil-generated field distribution. The potential impact of field distribution measurement with an outstanding comprehensive properties of spatial resolution, sensitivity and dynamic range is far-reaching. It could lead to capability of 3D magnetography for small stuffs and/or organs in millimeter or even smaller scale.

physics.atom-ph↗
arXiv2021

Training Provably Robust Models by Polyhedral Envelope Regularization

Training certifiable neural networks enables one to obtain models with robustness guarantees against adversarial attacks. In this work, we introduce a framework to bound the adversary-free region in the neighborhood of the input data by a polyhedral envelope, which yields finer-grained certified robustness. We further introduce polyhedral envelope regularization (PER) to encourage larger polyhedral envelopes and thus improve the provable robustness of the models. We demonstrate the flexibility and effectiveness of our framework on standard benchmarks; it applies to networks of different architectures and general activation functions. Compared with the state-of-the-art methods, PER has very little computational overhead and better robustness guarantees without over-regularizing the model.

cs.LG↗
arXiv2021

Hierarchy-Aware T5 with Path-Adaptive Mask Mechanism for Hierarchical Text Classification

Hierarchical Text Classification (HTC), which aims to predict text labels organized in hierarchical space, is a significant task lacking in investigation in natural language processing. Existing methods usually encode the entire hierarchical structure and fail to construct a robust label-dependent model, making it hard to make accurate predictions on sparse lower-level labels and achieving low Macro-F1. In this paper, we propose a novel PAMM-HiA-T5 model for HTC: a hierarchy-aware T5 model with path-adaptive mask mechanism that not only builds the knowledge of upper-level labels into low-level ones but also introduces path dependency information in label prediction. Specifically, we generate a multi-level sequential label structure to exploit hierarchical dependency across different levels with Breadth-First Search (BFS) and T5 model. To further improve label dependency prediction within each path, we then propose an original path-adaptive mask mechanism (PAMM) to identify the label's path information, eliminating sources of noises from other paths. Comprehensive experiments on three benchmark datasets show that our novel PAMM-HiA-T5 model greatly outperforms all state-of-the-art HTC approaches especially in Macro-F1. The ablation studies show that the improvements mainly come from our innovative approach instead of T5.

cs.CL↗
arXiv2021

FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks

In natural language processing (NLP), state-of-the-art (SOTA) semi-supervised learning (SSL) frameworks have shown great performance on deep pre-trained language models such as BERT, and are expected to significantly reduce the demand for manual labeling. However, our empirical studies indicate that these frameworks are not suitable for lightweight models such as TextCNN, LSTM and etc. In this work, we develop a new SSL framework called FLiText, which stands for Faster and Lighter semi-supervised Text classification. FLiText introduces an inspirer network together with the consistency regularization framework, which leverages a generalized regular constraint on the lightweight models for efficient SSL. As a result, FLiText obtains new SOTA performance for lightweight models across multiple SSL benchmarks on text classification. Compared with existing SOTA SSL methods on TextCNN, FLiText improves the accuracy of lightweight model TextCNN from 51.00% to 90.49% on IMDb, 39.8% to 58.06% on Yelp-5, and from 55.3% to 65.08% on Yahoo. In addition, compared with the fully supervised method on the full dataset, FLiText just uses less than 1% of labeled data to improve the accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDb, Yelp-5, and Yahoo respectively.

cs.CL↗
arXiv2021

Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation

Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event and the clinical decision of treatment planning. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features. In this paper, we propose a novel domain adaptation (DA) method with two components to address these problems. The first component is a stochastic class-balanced boosting sampling strategy that overcomes the imbalanced learning problem and improves the classification performance on poorly-predicted classes. The second component is a representation learning that guarantees three properties: 1) domain-transferability by prototype triplet loss, 2) discriminant by conditional maximum mean discrepancy loss, and 3) completeness by multi-view reconstruction loss. Particularly, we propose a domain translator and align the heterogeneous data to the estimated class prototypes (i.e., class centers) in a hyper-sphere manifold. Experiments on cross-site severity assessment of COVID-19 from CT images show that the proposed method can effectively tackle the imbalanced learning problem and outperform recent DA approaches.

eess.IV↗
arXiv2021

On the Dynamics of Liquids in the Large-Dimensional Limit

In this work, we analytically derive the exact closed dynamical equations for a liquid with short-ranged interactions in large spatial dimensions using the same statistical mechanics tools employed to analyze Brownian motion. Our derivation greatly simplifies the original path-integral-based route to these equations and provides new insight into the physical features associated with high-dimensional liquids and glass formation. Most importantly, our construction provides a facile route to the exact dynamical analysis of important related dynamical problems, as well as a means to devise cluster generalizations of the exact solution in infinite dimensions. This latter fact opens the door to the construction of increasingly accurate theories of vitrification in three-dimensional liquids.

cond-mat.stat-mech↗
arXiv2021

Fourier Transform Approximation as an Auxiliary Task for Image Classification

Image reconstruction is likely the most predominant auxiliary task for image classification, but we would like to think twice about this convention. In this paper, we investigated "approximating the Fourier Transform of the input image" as a potential alternative, in the hope that it may further boost the performances on the primary task or introduce novel constraints not well covered by image reconstruction. We experimented with five popular classification architectures on the CIFAR-10 dataset, and the empirical results indicated that our proposed auxiliary task generally improves the classification accuracy. More notably, the results showed that in certain cases our proposed auxiliary task may enhance the classifiers' resistance to adversarial attacks generated using the fast gradient sign method.

cs.CV↗
arXiv2021

MG-DVD: A Real-time Framework for Malware Variant Detection Based on Dynamic Heterogeneous Graph Learning

Detecting the newly emerging malware variants in real time is crucial for mitigating cyber risks and proactively blocking intrusions. In this paper, we propose MG-DVD, a novel detection framework based on dynamic heterogeneous graph learning, to detect malware variants in real time. Particularly, MG-DVD first models the fine-grained execution event streams of malware variants into dynamic heterogeneous graphs and investigates real-world meta-graphs between malware objects, which can effectively characterize more discriminative malicious evolutionary patterns between malware and their variants. Then, MG-DVD presents two dynamic walk-based heterogeneous graph learning methods to learn more comprehensive representations of malware variants, which significantly reduces the cost of the entire graph retraining. As a result, MG-DVD is equipped with the ability to detect malware variants in real time, and it presents better interpretability by introducing meaningful meta-graphs. Comprehensive experiments on large-scale samples prove that our proposed MG-DVD outperforms state-of-the-art methods in detecting malware variants in terms of effectiveness and efficiency.

cs.CR↗