arXiv subjects

Xiao Zhang

Publications and source records attributed to Xiao Zhang.

At least 415 records · Page 23Linked to original sources

arXiv2021

To the Point: Efficient 3D Object Detection in the Range Image with Graph Convolution Kernels

3D object detection is vital for many robotics applications. For tasks where a 2D perspective range image exists, we propose to learn a 3D representation directly from this range image view. To this end, we designed a 2D convolutional network architecture that carries the 3D spherical coordinates of each pixel throughout the network. Its layers can consume any arbitrary convolution kernel in place of the default inner product kernel and exploit the underlying local geometry around each pixel. We outline four such kernels: a dense kernel according to the bag-of-words paradigm, and three graph kernels inspired by recent graph neural network advances: the Transformer, the PointNet, and the Edge Convolution. We also explore cross-modality fusion with the camera image, facilitated by operating in the perspective range image view. Our method performs competitively on the Waymo Open Dataset and improves the state-of-the-art AP for pedestrian detection from 69.7% to 75.5%. It is also efficient in that our smallest model, which still outperforms the popular PointPillars in quality, requires 180 times fewer FLOPS and model parameters

cs.CV↗

arXiv2021

Universal Adversarial Perturbations for CNN Classifiers in EEG-Based BCIs

Multiple convolutional neural network (CNN) classifiers have been proposed for electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, CNN models have been found vulnerable to universal adversarial perturbations (UAPs), which are small and example-independent, yet powerful enough to degrade the performance of a CNN model, when added to a benign example. This paper proposes a novel total loss minimization (TLM) approach to generate UAPs for EEG-based BCIs. Experimental results demonstrated the effectiveness of TLM on three popular CNN classifiers for both target and non-target attacks. We also verified the transferability of UAPs in EEG-based BCI systems. To our knowledge, this is the first study on UAPs of CNN classifiers in EEG-based BCIs. UAPs are easy to construct, and can attack BCIs in real-time, exposing a potentially critical security concern of BCIs.

cs.LG↗

arXiv2021

Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examples

Training images with data transformations have been suggested as contrastive examples to complement the testing set for generalization performance evaluation of deep neural networks (DNNs). In this work, we propose a practical framework ContRE (The word "contre" means "against" or "versus" in French.) that uses Contrastive examples for DNN geneRalization performance Estimation. Specifically, ContRE follows the assumption in contrastive learning that robust DNN models with good generalization performance are capable of extracting a consistent set of features and making consistent predictions from the same image under varying data transformations. Incorporating with a set of randomized strategies for well-designed data transformations over the training set, ContRE adopts classification errors and Fisher ratios on the generated contrastive examples to assess and analyze the generalization performance of deep models in complement with a testing set. To show the effectiveness and the efficiency of ContRE, extensive experiments have been done using various DNN models on three open source benchmark datasets with thorough ablation studies and applicability analyses. Our experiment results confirm that (1) behaviors of deep models on contrastive examples are strongly correlated to what on the testing set, and (2) ContRE is a robust measure of generalization performance complementing to the testing set in various settings.

cs.LG↗

arXiv2021

Utility Optimal Thread Assignment and Resource Allocation in Multi-Server Systems

Achieving high performance in many multi-server systems requires finding a good assignment of worker threads to servers and also effectively allocating each server's resources to its assigned threads. The assignment and allocation components of this problem have been studied extensively but largely separately in the literature. In this paper, we introduce the assign and allocate (AA) problem, which seeks to simultaneously find an assignment and allocation that maximizes the total utility of the threads. Assigning and allocating the threads together can result in substantially better overall utility than performing the steps separately, as is traditionally done. We model each thread by a utility function giving its performance as a function of its assigned resources. We first prove that the AA problem is NP-hard. We then present a $2 (\sqrt{2}-1) > 0.828$ factor approximation algorithm for concave utility functions, which runs in $O(mn^2 + n (\log mC)^2)$ time for $n$ threads and $m$ servers with $C$ amount of resources each. We also give a faster algorithm with the same approximation ratio and $O(n (\log mC)^2)$ time complexity. We then extend the problem to two more general settings. First, we consider threads with nonconcave utility functions, and give a 1/2 factor approximation algorithm. Next, we give an algorithm for threads using multiple types of resources, and show the algorithm achieves good empirical performance. We conduct extensive experiments to test the performance of our algorithms on threads with both synthetic and realistic utility functions, and find that they achieve over 92\% of the optimal utility on average. We also compare our algorithms with a number of practical heuristics, and find that our algorithms achieve up to 9 times higher total utility.

cs.DC↗

arXiv2021

Rethink the Connections among Generalization, Memorization and the Spectral Bias of DNNs

Over-parameterized deep neural networks (DNNs) with sufficient capacity to memorize random noise can achieve excellent generalization performance, challenging the bias-variance trade-off in classical learning theory. Recent studies claimed that DNNs first learn simple patterns and then memorize noise; some other works showed a phenomenon that DNNs have a spectral bias to learn target functions from low to high frequencies during training. However, we show that the monotonicity of the learning bias does not always hold: under the experimental setup of deep double descent, the high-frequency components of DNNs diminish in the late stage of training, leading to the second descent of the test error. Besides, we find that the spectrum of DNNs can be applied to indicating the second descent of the test error, even though it is calculated from the training set only.

cs.LG↗

arXiv2021

Optimization Variance: Exploring Generalization Properties of DNNs

Unlike the conventional wisdom in statistical learning theory, the test error of a deep neural network (DNN) often demonstrates double descent: as the model complexity increases, it first follows a classical U-shaped curve and then shows a second descent. Through bias-variance decomposition, recent studies revealed that the bell-shaped variance is the major cause of model-wise double descent (when the DNN is widened gradually). This paper investigates epoch-wise double descent, i.e., the test error of a DNN also shows double descent as the number of training epoches increases. By extending the bias-variance analysis to epoch-wise double descent of the zero-one loss, we surprisingly find that the variance itself, without the bias, varies consistently with the test error. Inspired by this result, we propose a novel metric, optimization variance (OV), to measure the diversity of model updates caused by the stochastic gradients of random training batches drawn in the same iteration. OV can be estimated using samples from the training set only but correlates well with the (unknown) \emph{test} error, and hence early stopping may be achieved without using a validation set.

cs.LG↗

arXiv2021

Near-Room-Temperature Ferromagnetic Behavior of Single-Atom-Thick 2D Iron in Nanolaminated Ternary MAX Phases

Two dimensional (2D) ferromagnetic materials have attracted much attention in the fields of condensed matter physics and materials science, but their synthesis is still a challenge given their limitations on structural stability and susceptibility to oxidization. MAX phases nanolaminated ternary carbides or nitrides possess a unique crystal structure in which single-atom-thick A sublayers are interleaved by two dimensional MX slabs, providing nanostructured templates for designing 2D ferromagnetic materials if the non-magnetic A sublayers can be substituted replaced by magnetic elements. Here, we report three new ternary magnetic MAX phases (Ta2FeC, Ti2FeN and Nb2FeC) with A sublayers of single-atom-thick 2D iron through an isomorphous replacement reaction of MAX precursors (Ta2AlC, Ti2AlN and Nb2AlC) with a Lewis acid salts (FeCl2). All these MAX phases exhibit ferromagnetic (FM) behavior. The Curie temperature (Tc) of Ta2FeC and Nb2FeC MAX phase are 281 K and 291 K, respectively, i.e. close to room temperature. The saturation magnetization of these ternary magnetic MAX phases is almost two orders of magnitude higher than that of V2(Sn,Fe)C MAX phase whose A-site is partial substituted by Fe. Theoretical calculations on magnetic orderings of spin moments of Fe atoms in these nanolaminated magnetic MAX phases reveal that the magnetism can be mainly ascribed to intralayer exchange interaction of the 2D Fe atomic layers. Owning to the richness in composition of MAX phases, there is a large compositional space for constructing functional single-atom-thick 2D layers in materials using these nanolaminated templates.

cond-mat.mtrl-sci↗

arXiv2021

Quantum k-uniform states for heterogeneous systems from irredundant mixed orthogonal arrays

Quantum multipartite entangled states play significant roles in quantum information processing. By using difference schemes and orthogonal partitions, we construct a series of infinite classes of irredundant mixed orthogonal arrays (IrMOAs) and thus provide positive answers to two open problems. The first is the extension of the method for constructing homogeneous systems from orthogonal arrays (OAs) to heterogeneous multipartite systems with different individual levels. The second is the existence of $k$-uniform states in heterogeneous quantum systems. We present explicit constructions of two and three-uniform states for arbitrary heterogeneous multipartite systems with coprime individual levels, and characterize the entangled states in heterogeneous systems consisting of subsystems with nonprime power dimensions as well. Moreover, we obtain infinite classes of $k$-uniform states for heterogeneous multipartite systems for any $k\geq2$. The non-existence of a class of IrMOAs is also proved.

quant-ph↗

arXiv2021

Design of Robust and Efficient Edge Server Placement and Server Scheduling Policies: Extended Version

We study how to design edge server placement and server scheduling policies under workload uncertainty for 5G networks. We introduce a new metric called resource pooling factor to handle unexpected workload bursts. Maximizing this metric offers a strong enhancement on top of robust optimization against workload uncertainty. Using both real traces and synthetic traces, we show that the proposed server placement and server scheduling policies not only demonstrate better robustness against workload uncertainty than existing approaches, but also significantly reduce the cost of service providers. Specifically, in order to achieve close-to-zero workload rejection rate, the proposed server placement policy reduces the number of required edge servers by about 25% compared with the state-of-the-art approach; the proposed server scheduling policy reduces the energy consumption of edge servers by about 13% without causing much impact on the service quality.

cs.NI↗

arXiv2021

Improved Estimation of Concentration Under $\ell_p$-Norm Distance Metrics Using Half Spaces

Concentration of measure has been argued to be the fundamental cause of adversarial vulnerability. Mahloujifar et al. presented an empirical way to measure the concentration of a data distribution using samples, and employed it to find lower bounds on intrinsic robustness for several benchmark datasets. However, it remains unclear whether these lower bounds are tight enough to provide a useful approximation for the intrinsic robustness of a dataset. To gain a deeper understanding of the concentration of measure phenomenon, we first extend the Gaussian Isoperimetric Inequality to non-spherical Gaussian measures and arbitrary $\ell_p$-norms ($p \geq 2$). We leverage these theoretical insights to design a method that uses half-spaces to estimate the concentration of any empirical dataset under $\ell_p$-norm distance metrics. Our proposed algorithm is more efficient than Mahloujifar et al.'s, and our experiments on synthetic datasets and image benchmarks demonstrate that it is able to find much tighter intrinsic robustness bounds. These tighter estimates provide further evidence that rules out intrinsic dataset concentration as a possible explanation for the adversarial vulnerability of state-of-the-art classifiers.

cs.LG↗

arXiv2021

Tidal surface states as fingerprints of non-Hermitian nodal knot metals

Non-Hermitian nodal knot metals (NKMs) contains intricate complex-valued energy bands gives rise to knotted exceptional loops and new topological surface states. We introduce a formalism that connects the algebraic, geometric, and topological aspects of these surface states with their parent knots, and provide an optimized constructive ansatz for tight-binding models for non-Hermitian NKMs of arbitrary knot complexity and minimal hybridization range. Specifically, various representative non-Hermitian torus knots Hamiltonians are constructed in real-space, and their nodal topologies studied via winding numbers that avoid the explicit construction of generalized Brillouin zones. In particular, we identify the surface state boundaries as "tidal" intersections of the complex band structure in a marine landscape analogy. Beyond topological quantities based on Berry phases, we further find these tidal surface states to be intimately connected to the band vorticity and the layer structure of their dual Seifert surface, and as such provide a fingerprint for non-Hermitian NKMs.

cond-mat.mes-hall↗

arXiv2021

Non-Abelian charged nodal links in dielectric photonic crystal

A nodal link is a special form of a line degeneracy (a nodal line) between adjacent bands in the momentum space of a three-dimensional topological crystal. Unlike nodal chains or knots, a nodal link consists of two or more mutually-linked rings that do not touch each other. Recent studies on non-Abelian band topology revealed that the topological charges of the nodal links can have the properties of quaternions. However, a photonic crystal that has a nodal link with non-Abelian charges has not been reported. Here, we propose dielectric photonic crystals in forms of double diamond structures which realize the nodal links in the momentum space. By examining the evolution of the eigenstate correlations along the closed loops which enclose the nodal line(s) of the links, their non-Abelian topological charges are also analyzed. The proposed design scheme and theoretical approach in this work will allow for experimental observation of photonic non-Abelian charges in purely dielectric materials and facilitate the control of the degeneracy in complex photonic structures.

physics.optics↗

arXiv2021

Characterization of the magnetic field through the three-body loss near a narrow Feshbach resonance

The narrow s-wave Feshbach resonance of a $^6$Li Fermi gas shows strong three-body loss, which is proposed to be used to measure the minute change of a magnetic field around the resonance. However, the eddy current will cause ultracold atom experiencing a magnetic field delayed to the desired magnetic field from the current of the magnetic coils. The elimination of the eddy current effect will play a key role in any experiments that motivated to measure the magnetic field to the precision of a part per million stability. Here, we apply a method to correct the eddy current effect for precision measurement of the magnetic field. We first record the three-body loss influenced by the effect of induced eddy current, then use a certain model to obtain the time constant of the actual magnetic field by fitting the atom loss. This precisely determines the actual magnetic field according to the time response of the three-body loss. After that, we implement the desired magnetic field to the atoms so that we can analyze the three-body loss across the whole narrow Feshbach resonance. The results show that the three-body recombination is the dominated loss mechanism near the resonance. We expect this practical method of correcting the eddy current error of the magnetic field can be further applied to the future studies of quantum few- and many-body physics near a narrow Feshbach resonance.

cond-mat.quant-gas↗

arXiv2021

Multi-Task Time Series Forecasting With Shared Attention

Time series forecasting is a key component in many industrial and business decision processes and recurrent neural network (RNN) based models have achieved impressive progress on various time series forecasting tasks. However, most of the existing methods focus on single-task forecasting problems by learning separately based on limited supervised objectives, which often suffer from insufficient training instances. As the Transformer architecture and other attention-based models have demonstrated its great capability of capturing long term dependency, we propose two self-attention based sharing schemes for multi-task time series forecasting which can train jointly across multiple tasks. We augment a sequence of paralleled Transformer encoders with an external public multi-head attention function, which is updated by all data of all tasks. Experiments on a number of real-world multi-task time series forecasting tasks show that our proposed architectures can not only outperform the state-of-the-art single-task forecasting baselines but also outperform the RNN-based multi-task forecasting method.

cs.LG↗

arXiv2021

Modeling Heterogeneous Relations across Multiple Modes for Potential Crowd Flow Prediction

Potential crowd flow prediction for new planned transportation sites is a fundamental task for urban planners and administrators. Intuitively, the potential crowd flow of the new coming site can be implied by exploring the nearby sites. However, the transportation modes of nearby sites (e.g. bus stations, bicycle stations) might be different from the target site (e.g. subway station), which results in severe data scarcity issues. To this end, we propose a data driven approach, named MOHER, to predict the potential crowd flow in a certain mode for a new planned site. Specifically, we first identify the neighbor regions of the target site by examining the geographical proximity as well as the urban function similarity. Then, to aggregate these heterogeneous relations, we devise a cross-mode relational GCN, a novel relation-specific transformation model, which can learn not only the correlations but also the differences between different transportation modes. Afterward, we design an aggregator for inductive potential flow representation. Finally, an LTSM module is used for sequential flow prediction. Extensive experiments on real-world data sets demonstrate the superiority of the MOHER framework compared with the state-of-the-art algorithms.

cs.LG↗

arXiv2020

Guidance Module Network for Video Captioning

Video captioning has been a challenging and significant task that describes the content of a video clip in a single sentence. The model of video captioning is usually an encoder-decoder. We find that the normalization of extracted video features can improve the final performance of video captioning. Encoder-decoder model is usually trained using teacher-enforced strategies to make the prediction probability of each word close to a 0-1 distribution and ignore other words. In this paper, we present a novel architecture which introduces a guidance module to encourage the encoder-decoder model to generate words related to the past and future words in a caption. Based on the normalization and guidance module, guidance module net (GMNet) is built. Experimental results on commonly used dataset MSVD show that proposed GMNet can improve the performance of the encoder-decoder model on video captioning tasks.

cs.CV↗

arXiv2020

Self-Supervised Visual Representation Learning from Hierarchical Grouping

We create a framework for bootstrapping visual representation learning from a primitive visual grouping capability. We operationalize grouping via a contour detector that partitions an image into regions, followed by merging of those regions into a tree hierarchy. A small supervised dataset suffices for training this grouping primitive. Across a large unlabeled dataset, we apply this learned primitive to automatically predict hierarchical region structure. These predictions serve as guidance for self-supervised contrastive feature learning: we task a deep network with producing per-pixel embeddings whose pairwise distances respect the region hierarchy. Experiments demonstrate that our approach can serve as state-of-the-art generic pre-training, benefiting downstream tasks. We additionally explore applications to semantic region search and video-based object instance tracking.

cs.CV↗

arXiv2020

Measuring Information Transfer in Neural Networks

Quantifying the information content in a neural network model is essentially estimating the model's Kolmogorov complexity. Recent success of prequential coding on neural networks points to a promising path of deriving an efficient description length of a model. We propose a practical measure of the generalizable information in a neural network model based on prequential coding, which we term Information Transfer ($L_{IT}$). Theoretically, $L_{IT}$ is an estimation of the generalizable part of a model's information content. In experiments, we show that $L_{IT}$ is consistently correlated with generalizable information and can be used as a measure of patterns or "knowledge" in a model or a dataset. Consequently, $L_{IT}$ can serve as a useful analysis tool in deep learning. In this paper, we apply $L_{IT}$ to compare and dissect information in datasets, evaluate representation models in transfer learning, and analyze catastrophic forgetting and continual learning algorithms. $L_{IT}$ provides an information perspective which helps us discover new insights into neural network learning.

cs.LG↗