arXiv subjects

Han Xiao

Publications and source records attributed to Han Xiao.

At least 127 records · Page 7Linked to original sources

arXiv2022

Token-Label Alignment for Vision Transformers

Data mixing strategies (e.g., CutMix) have shown the ability to greatly improve the performance of convolutional neural networks (CNNs). They mix two images as inputs for training and assign them with a mixed label with the same ratio. While they are shown effective for vision transformers (ViTs), we identify a token fluctuation phenomenon that has suppressed the potential of data mixing strategies. We empirically observe that the contributions of input tokens fluctuate as forward propagating, which might induce a different mixing ratio in the output tokens. The training target computed by the original data mixing strategy can thus be inaccurate, resulting in less effective training. To address this, we propose a token-label alignment (TL-Align) method to trace the correspondence between transformed tokens and the original tokens to maintain a label for each token. We reuse the computed attention at each layer for efficient token-label alignment, introducing only negligible additional training costs. Extensive experiments demonstrate that our method improves the performance of ViTs on image classification, semantic segmentation, objective detection, and transfer learning tasks. Code is available at: https://github.com/Euphoria16/TL-Align.

cs.CV↗

arXiv2022

Concise and interpretable multi-label rule sets

Multi-label classification is becoming increasingly ubiquitous, but not much attention has been paid to interpretability. In this paper, we develop a multi-label classifier that can be represented as a concise set of simple "if-then" rules, and thus, it offers better interpretability compared to black-box models. Notably, our method is able to find a small set of relevant patterns that lead to accurate multi-label classification, while existing rule-based classifiers are myopic and wasteful in searching rules,requiring a large number of rules to achieve high accuracy. In particular, we formulate the problem of choosing multi-label rules to maximize a target function, which considers not only discrimination ability with respect to labels, but also diversity. Accounting for diversity helps to avoid redundancy, and thus, to control the number of rules in the solution set. To tackle the said maximization problem we propose a 2-approximation algorithm, which relies on a novel technique to sample high-quality rules. In addition to our theoretical analysis, we provide a thorough experimental evaluation, which indicates that our approach offers a trade-off between predictive performance and interpretability that is unmatched in previous work.

cs.LG↗

arXiv2022

Concentrated subradiant modes in one-dimensional atomic array coupled with chiral waveguides

Non-Hermitian systems have recently attracted broad interest and exhibited intriguing physical phenomena, in which the non-Hermitian skin effect is one of the most remarkable quantum phenomena desiring detailed investigations and has been widely studied in various fermionic and bosonic systems. Here we propose a non-Hermitian atom-waveguide system composed of a tilted one-dimensional atomic array coupled with two identical waveguides with opposite chiralities. Such system creates an effective lattice model including nonreciprocal long-range hoppings through the chiral-waveguide photon-mediated interactions. We find the excitation of the collective atomic states concentrates in the middle interface, pointing towards the non-Hermitian skin effect associated with subradiant modes, while, on the contrary, superradiant modes exhibit extended features. Simulation results present subradiant funneling effect, with robustness against small atomic position disorders. Our work underpins the fundamental comprehension towards the non-Hermitian skin effect in open quantum systems and also provide prospective paths to study non-Hermitian systems in the area of quantum optics.

quant-ph↗

arXiv2022

On the Population Monotonicity of Independent Set Games

An independent set game is a cooperative game defined on graphs and dealing with profit-sharing in maximum independent set problems. A population monotonic allocation scheme is a rule specifying how to share the profit of each coalition among its participants such that every participant is better off when the coalition expands. In this paper, we provide a necessary and sufficient characterization for population monotonic allocation schemes in independent set games. Moreover, our characterization can be verified efficiently.

cs.GT↗

arXiv2022

Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search

In this paper, we propose a Shapley value based method to evaluate operation contribution (Shapley-NAS) for neural architecture search. Differentiable architecture search (DARTS) acquires the optimal architectures by optimizing the architecture parameters with gradient descent, which significantly reduces the search cost. However, the magnitude of architecture parameters updated by gradient descent fails to reveal the actual operation importance to the task performance and therefore harms the effectiveness of obtained architectures. By contrast, we propose to evaluate the direct influence of operations on validation accuracy. To deal with the complex relationships between supernet components, we leverage Shapley value to quantify their marginal contributions by considering all possible combinations. Specifically, we iteratively optimize the supernet weights and update the architecture parameters by evaluating operation contributions via Shapley value, so that the optimal architectures are derived by selecting the operations that contribute significantly to the tasks. Since the exact computation of Shapley value is NP-hard, the Monte-Carlo sampling based algorithm with early truncation is employed for efficient approximation, and the momentum update mechanism is adopted to alleviate fluctuation of the sampling process. Extensive experiments on various datasets and various search spaces show that our Shapley-NAS outperforms the state-of-the-art methods by a considerable margin with light search cost. The code is available at https://github.com/Euphoria16/Shapley-NAS.git

cs.LG↗

arXiv2022

AI Enlightens Wireless Communication: A Transformer Backbone for CSI Feedback

This paper is based on the background of the 2nd Wireless Communication Artificial Intelligence (AI) Competition (WAIC) which is hosted by IMT-2020(5G) Promotion Group 5G+AIWork Group, where the framework of the eigenvector-based channel state information (CSI) feedback problem is firstly provided. Then a basic Transformer backbone for CSI feedback referred to EVCsiNet-T is proposed. Moreover, a series of potential enhancements for deep learning based (DL-based) CSI feedback including i) data augmentation, ii) loss function design, iii) training strategy, and iv) model ensemble are introduced. The experimental results involving the comparison between EVCsiNet-T and traditional codebook methods over different channels are further provided, which show the advanced performance and a promising prospect of Transformer on DL-based CSI feedback problem.

eess.SP↗

arXiv2022

Support Vector Machines under Adversarial Label Contamination

Machine learning algorithms are increasingly being applied in security-related tasks such as spam and malware detection, although their security properties against deliberate attacks have not yet been widely understood. Intelligent and adaptive attackers may indeed exploit specific vulnerabilities exposed by machine learning techniques to violate system security. Being robust to adversarial data manipulation is thus an important, additional requirement for machine learning algorithms to successfully operate in adversarial settings. In this work, we evaluate the security of Support Vector Machines (SVMs) to well-crafted, adversarial label noise attacks. In particular, we consider an attacker that aims to maximize the SVM's classification error by flipping a number of labels in the training data. We formalize a corresponding optimal attack strategy, and solve it by means of heuristic approaches to keep the computational complexity tractable. We report an extensive experimental analysis on the effectiveness of the considered attacks against linear and non-linear SVMs, both on synthetic and real-world datasets. We finally argue that our approach can also provide useful insights for developing more secure SVM learning algorithms, and also novel techniques in a number of related research areas, such as semi-supervised and active learning.

cs.LG↗

arXiv2022

Variation between Antiferromagnetism and Ferrimagnetism in NiPS3 by Electron Doping

How to electrically control magnetic properties of a magnetic material is promising towards spintronic applications, where the investigation of carrier doping effects on antiferromagnetic (AFM) materials remains challenging due to their zero net magnetization. In this work, we found electron doping dependent variation of magnetic orders of a two-dimensional (2D) AFM insulator NiPS3, where doping concentration is tuned by intercalating various organic cations into the van der Waals gaps of NiPS3 without introduction of defects and impurity phases. The doped NiPS3 shows an AFM-ferrimagnetic (FIM) transition at doping level of 0.2-0.5 electrons/cell and a FIM-AFM transition at doping level of >= 0.6 electrons/cell. We propose that the found phenomenon is due to competition between Stoner exchange dominated inter-chain ferromagnetic order and super-exchange dominated inter-chain AFM order at different doping level. Our studies provide a viable way to exploit correlation between electronic structures and magnetic properties of 2D magnetic materials for realization of magnetoelectric effect.

cond-mat.mtrl-sci↗

arXiv2021

Excite atom-photon bound state inside the coupled-resonator waveguide coupled with a giant atom

It is of fundamental interest in controlling the light-matter interaction for a long time in the field of quantum information processing. However, usual excitation with the propagating photon can hardly excite a localized state of light while keeping the atom under a subradiant decay in an atom-waveguide system. Here, we propose a model of coupling between a giant atom and the dynamically-modulated coupled-resonator waveguide and find that a bound state, where the light shows the localization effect and atom exhibits a subradiant decay time, can be excited by a propagating photon. An analytical treatment based on the separation of the propagating states and localized states of light has been used and provides inspiring explanation of our finding, i.e., a propagating photon can be efficiently converted to the localized light through the light-atom interactions in three resonators at frequency difference precisely equivalent to external modulation frequency. Our work therefore provides an alternative method for actively localizing the photon in a modulated coupled-resonator waveguide system interacting with giant atom, and also points out a way to study the light-atom interaction in a synthetic frequency dimension that holds the similar Hamiltonian.

quant-ph↗

arXiv2021

Approximate Core Allocations for Multiple Partners Matching Games

The matching game is a cooperative game where the value of every coalition is the maximum revenue of players in the coalition can make by forming pairwise disjoint partners. The multiple partners matching game generalizes the matching game by allowing each player to have more than one possibly repeated partner. In this paper, we study profit-sharing in multiple partners matching games. A central concept for profit-sharing is the core which consists of all possible ways of distributing the profit among individual players such that the grand coalition remains intact. The core of multiple partners matching games may be empty [Deng et al., Algorithmic aspects of the core of combinatorial optimization games, Math. Oper. Res., 1999.]; even when the core is non-empty, the core membership problem is intractable in general [Biro et al., The stable fixtures problem with payments, Games Econ. Behav., 2018]. Thus we study approximate core allocations upon which a coalition may be paid less than the profit it makes by seceding from the grand coalition. We provide an LP-based mechanism guaranteeing that no coalition is paid less than $2/3$ times the profit it makes on its own. We also show that $2/3$ is the best possible factor relative to the underlying LP-relaxation. Our result generalizes the work of Vazirani [Vazirani, The general graph matching game: approximate core, arXiv, 2021] from matching games to multiple partners matching games.

cs.GT↗

arXiv2021

Multi-linear Tensor Autoregressive Models

Contemporary time series analysis has seen more and more tensor type data, from many fields. For example, stocks can be grouped according to Size, Book-to-Market ratio, and Operating Profitability, leading to a 3-way tensor observation at each month. We propose an autoregressive model for the tensor-valued time series, with autoregressive terms depending on multi-linear coefficient matrices. Comparing with the traditional approach of vectoring the tensor observations and then applying the vector autoregressive model, the tensor autoregressive model preserves the tensor structure and admits corresponding interpretations. We introduce three estimators based on projection, least squares, and maximum likelihood. Our analysis considers both fixed dimensional and high dimensional settings. For the former we establish the central limit theorems of the estimators, and for the latter we focus on the convergence rates and the model selection. The performance of the model is demonstrated by simulated and real examples.

stat.ME↗

arXiv2021

Generalizable Mixed-Precision Quantization via Attribution Rank Preservation

In this paper, we propose a generalizable mixed-precision quantization (GMPQ) method for efficient inference. Conventional methods require the consistency of datasets for bitwidth search and model deployment to guarantee the policy optimality, leading to heavy search cost on challenging largescale datasets in realistic applications. On the contrary, our GMPQ searches the mixed-quantization policy that can be generalized to largescale datasets with only a small amount of data, so that the search cost is significantly reduced without performance degradation. Specifically, we observe that locating network attribution correctly is general ability for accurate visual analysis across different data distribution. Therefore, despite of pursuing higher model accuracy and complexity, we preserve attribution rank consistency between the quantized models and their full-precision counterparts via efficient capacity-aware attribution imitation for generalizable mixed-precision quantization strategy search. Extensive experiments show that our method obtains competitive accuracy-complexity trade-off compared with the state-of-the-art mixed-precision networks in significantly reduced search cost. The code is available at https://github.com/ZiweiWangTHU/GMPQ.git.

cs.CV↗

arXiv2021

AI Enlightens Wireless Communication: Analyses, Solutions and Opportunities on CSI Feedback

In this paper, we give a systematic description of the 1st Wireless Communication Artificial Intelligence (AI) Competition (WAIC) which is hosted by IMT-2020(5G) Promotion Group 5G+AI Work Group. Firstly, the framework of full channel state information (F-CSI) feedback problem and its corresponding channel dataset are provided. Then the enhancing schemes for DL-based F-CSI feedback including i) channel data analysis and preprocessing, ii) neural network design and iii) quantization enhancement are elaborated. The final competition results composed of different enhancing schemes are presented. Based on the valuable experience of 1st WAIC, we also list some challenges and potential study areas for the design of AI-based wireless communication systems.

eess.SP↗

arXiv2021

Generalized Autoregressive Moving Average Models with GARCH Errors

One of the important and widely used classes of models for non-Gaussian time series is the generalized autoregressive model average models (GARMA), which specifies an ARMA structure for the conditional mean process of the underlying time series. However, in many applications one often encounters conditional heteroskedasticity. In this paper we propose a new class of models, referred to as GARMA-GARCH models, that jointly specify both the conditional mean and conditional variance processes of a general non-Gaussian time series. Under the general modeling framework, we propose three specific models, as examples, for proportional time series, nonnegative time series, and skewed and heavy-tailed financial time series. Maximum likelihood estimator (MLE) and quasi Gaussian MLE (GMLE) are used to estimate the parameters. Simulation studies and three applications are used to demonstrate the properties of the models and the estimation procedures.

stat.ME↗

arXiv2021

Population Monotonicity in Matching Games

A matching game is a cooperative profit game defined on an edge-weighted graph, where the players are the vertices and the profit of a coalition is the maximum weight of matchings in the subgraph induced by the coalition. A population monotonic allocation scheme is a collection of rules defining how to share the profit among players in each coalition such that every player is better off when the coalition expands. In this paper, we study matching games and provide a necessary and sufficient characterization for the existence of population monotonic allocation schemes. Our characterization also implies that whether a matching game admits population monotonic allocation schemes can be determined efficiently.

cs.GT↗

arXiv2020

Matrix Completion using Kronecker Product Approximation

A matrix completion problem is to recover the missing entries in a partially observed matrix. Most of the existing matrix completion methods assume a low rank structure of the underlying complete matrix. In this paper, we introduce an alternative and more general form of the underlying complete matrix, which assumes a low Kronecker rank instead of a low regular rank, but includes the latter as a special case. The extra flexibility allows for a much more parsimonious representation of the underlying matrix, but also raises the challenge of determining the proper Kronecker product configuration to be used. We find that the configuration can be identified using the mean squared error criterion as well as a modified cross-validation criterion. We establish the consistency of this procedure under suitable conditions on the signal-to-noise ratio. A aggregation procedure is also proposed to deal with special missing patterns and complex underlying structures. Both numerical and empirical studies are carried out to demonstrate the performance of the new method.

stat.ML↗

arXiv2020

On the Convexity of Independent Set Games

Independent set games are cooperative games defined on graphs, where players are edges and the value of a coalition is the maximum cardinality of independent sets in the subgraph defined by the coalition. In this paper, we investigate the convexity of independent set games, as convex games possess many nice properties both economically and computationally. For independent set games introduced by Deng et al. (Math. Oper. Res., 24:751-766, 1999), we provide a necessary and sufficient characterization for the convexity, i.e., every non-pendant edge is incident to a pendant edge in the underlying graph. Our characterization immediately yields a polynomial time algorithm for recognizing convex instances of independent set games. Besides, we introduce a new class of independent set games and provide an efficient characterization for the convexity.

cs.DM↗

arXiv2020

KoPA: Automated Kronecker Product Approximation

We consider the problem of matrix approximation and denoising induced by the Kronecker product decomposition. Specifically, we propose to approximate a given matrix by the sum of a few Kronecker products of matrices, which we refer to as the Kronecker product approximation (KoPA). Because the Kronecker product is an extension of the outer product from vectors to matrices, KoPA extends the low rank matrix approximation, and includes it as a special case. Comparing with the latter, KoPA also offers a greater flexibility, since it allows the user to choose the configuration, which are the dimensions of the two smaller matrices forming the Kronecker product. On the other hand, the configuration to be used is usually unknown, and needs to be determined from the data in order to achieve the optimal balance between accuracy and parsimony. We propose to use extended information criteria to select the configuration. Under the paradigm of high dimensional analysis, we show that the proposed procedure is able to select the true configuration with probability tending to one, under suitable conditions on the signal-to-noise ratio. We demonstrate the superiority of KoPA over the low rank approximations through numerical studies, and several benchmark image examples.

math.ST↗