arXiv Science⌕ Search

arXiv subjects

Xiao-Wen Chang

Publications and source records attributed to Xiao-Wen Chang.

At least 55 records · Page 3Linked to original sources
arXiv2022

Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Neural Networks

The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filters the neighborhood information of nodes. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN models for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks.

cs.LG↗
arXiv2022

Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks

The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filters the neighborhood node information. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN methods for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks.

cs.LG↗
arXiv2022

Revisiting Heterophily For Graph Neural Networks

Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using graph structures based on the relational inductive bias (homophily assumption). While GNNs have been commonly believed to outperform NNs in real-world tasks, recent work has identified a non-trivial set of datasets where their performance compared to NNs is not satisfactory. Heterophily has been considered the main cause of this empirical observation and numerous works have been put forward to address it. In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph-label consistency is a shortcoming. Then, we study heterophily from the perspective of post-aggregation node similarity and define new homophily metrics, which are potentially advantageous compared to existing ones. Based on this investigation, we prove that some harmful cases of heterophily can be effectively addressed by local diversification operation. Then, we propose the Adaptive Channel Mixing (ACM), a framework to adaptively exploit aggregation, diversification and identity channels node-wisely to extract richer localized information for diverse node heterophily situations. ACM is more powerful than the commonly used uni-channel framework for node classification tasks on heterophilic graphs and is easy to be implemented in baseline GNN layers. When evaluated on 10 benchmark node classification tasks, ACM-augmented baselines consistently achieve significant performance gain, exceeding state-of-the-art GNNs on most tasks without incurring significant computational burden.

cs.LG↗
arXiv2022

Sharper Bounds on Four Lattice Constants

The Korkine--Zolotareff (KZ) reduction, and its generalisations, are widely used lattice reduction strategies in communications and cryptography. The KZ constant and Schnorr's constant were defined by Schnorr in 1987. The KZ constant can be used to quantify some useful properties of KZ reduced matrices. Schnorr's constant can be used to characterize the output quality of his block $2k$-reduction and is used to define his semi block $2k$-reduction, which was also developed in 1987. Hermite's constant, which is a fundamental constant lattices, has many applications, such as bounding the length of the shortest nonzero lattice vector and the orthogonality defect of lattices. Rankin's constant was introduced by Rankin in 1953 as a generalization of Hermite's constant. It plays an important role in characterizing the output quality of block-Rankin reduction, proposed by Gama et al. in 2006. In this paper, we first develop a linear upper bound on Hermite's constant and then use it to develop an upper bound on the KZ constant. These upper bounds are sharper than those obtained recently by the authors, and the ratio of the new linear upper bound to the nonlinear upper bound, developed by Blichfeldt in 1929, on Hermite's constant is asymptotically 1.0047. Furthermore, we develop lower and upper bounds on Schnorr's constant. The improvement to the lower bound over the sharpest existing one developed by Gama et al. is around 1.7 times asymptotically, and the improvement to the upper bound over the sharpest existing one which was also developed by Gama et al. is around 4 times asymptotically. Finally, we develop lower and upper bounds on Rankin's constant. The improvements of the bounds over the sharpest existing ones, also developed by Gama et al., are exponential in the parameter defining the constant.

cs.IT↗
arXiv2021

Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?

Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using the graph structures based on the relational inductive bias (homophily assumption). Though GNNs are believed to outperform NNs in real-world tasks, performance advantages of GNNs over graph-agnostic NNs seem not generally satisfactory. Heterophily has been considered as a main cause and numerous works have been put forward to address it. In this paper, we first show that not all cases of heterophily are harmful for GNNs with aggregation operation. Then, we propose new metrics based on a similarity matrix which considers the influence of both graph structure and input features on GNNs. The metrics demonstrate advantages over the commonly used homophily metrics by tests on synthetic graphs. From the metrics and the observations, we find some cases of harmful heterophily can be addressed by diversification operation. With this fact and knowledge of filterbanks, we propose the Adaptive Channel Mixing (ACM) framework to adaptively exploit aggregation, diversification and identity channels in each GNN layer to address harmful heterophily. We validate the ACM-augmented baselines with 10 real-world node classification tasks. They consistently achieve significant performance gain and exceed the state-of-the-art GNNs on most of the tasks without incurring significant computational burden.

cs.LG↗
arXiv2020

META-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation

Temporal-Difference (TD) learning is a standard and very successful reinforcement learning approach, at the core of both algorithms that learn the value of a given policy, as well as algorithms which learn how to improve policies. TD-learning with eligibility traces provides a way to boost sample efficiency by temporal credit assignment, i.e. deciding which portion of a reward should be assigned to predecessor states that occurred at different previous times, controlled by a parameter $λ$. However, tuning this parameter can be time-consuming, and not tuning it can lead to inefficient learning. For better sample efficiency of TD-learning, we propose a meta-learning method for adjusting the eligibility trace parameter, in a state-dependent manner. The adaptation is achieved with the help of auxiliary learners that learn distributional information about the update targets online, incurring roughly the same computational complexity per step as the usual value learner. Our approach can be used both in on-policy and off-policy learning. We prove that, under some assumptions, the proposed method improves the overall quality of the update targets, by minimizing the overall target error. This method can be viewed as a plugin to assist prediction with function approximation by meta-learning feature (observation)-based $λ$ online, or even in the control case to assist policy improvement. Our empirical evaluation demonstrates significant performance improvements, as well as improved robustness of the proposed algorithm to learning rate variation.

cs.LG↗
arXiv2019

Revisit Policy Optimization in Matrix Form

In tabular case, when the reward and environment dynamics are known, policy evaluation can be written as $\bm{V}_{\bmπ} = (I - γP_{\bmπ})^{-1} \bm{r}_{\bmπ}$, where $P_{\bmπ}$ is the state transition matrix given policy ${\bmπ}$ and $\bm{r}_{\bmπ}$ is the reward signal given ${\bmπ}$. What annoys us is that $P_{\bmπ}$ and $\bm{r}_{\bmπ}$ are both mixed with ${\bmπ}$, which means every time when we update ${\bmπ}$, they will change together. In this paper, we leverage the notation from \cite{wang2007dual} to disentangle ${\bmπ}$ and environment dynamics which makes optimization over policy more straightforward. We show that policy gradient theorem \cite{sutton2018reinforcement} and TRPO \cite{schulman2015trust} can be put into a more general framework and such notation has good potential to be extended to model-based reinforcement learning.

cs.LG↗
arXiv2019

Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks

Recently, neural network based approaches have achieved significant improvement for solving large, complex, graph-structured problems. However, their bottlenecks still need to be addressed, and the advantages of multi-scale information and deep architectures have not been sufficiently exploited. In this paper, we theoretically analyze how existing Graph Convolutional Networks (GCNs) have limited expressive power due to the constraint of the activation functions and their architectures. We generalize spectral graph convolution and deep GCN in block Krylov subspace forms and devise two architectures, both with the potential to be scaled deeper but each making use of the multi-scale information in different ways. We further show that the equivalence of these two architectures can be established under certain conditions. On several node classification tasks, with or without the help of validation, the two new architectures achieve better performance compared to many state-of-the-art methods.

cs.LG↗
arXiv2019

Improved Upper Bounds on the Hermite and KZ Constants

The Korkine-Zolotareff (KZ) reduction is a widely used lattice reduction strategy in communications and cryptography. The Hermite constant, which is a vital constant of lattice, has many applications, such as bounding the length of the shortest nonzero lattice vector and orthogonality defect of lattices. The KZ constant can be used in quantifying some useful properties of KZ reduced matrices. In this paper, we first develop a linear upper bound on the Hermite constant and then use the bound to develop an upper bound on the KZ constant. These upper bounds are sharper than those obtained recently by the first two authors. Some examples on the applications of the improved upper bounds are also presented.

cs.IT↗
arXiv2018

On the KZ Reduction

The Korkine-Zolotareff (KZ) reduction is one of the often used reduction strategies for lattice decoding. In this paper, we first investigate some important properties of KZ reduced matrices. Specifically, we present a linear upper bound on the Hermit constant which is around $\frac{7}{8}$ times of the existing sharpest linear upper bound, and an upper bound on the KZ constant which is {\em polynomially} smaller than the existing sharpest one. We also propose upper bounds on the lengths of the columns of KZ reduced matrices, and an upper bound on the orthogonality defect of KZ reduced matrices which are even {\em polynomially and exponentially} smaller than those of boosted KZ reduced matrices, respectively. Then, we derive upper bounds on the magnitudes of the entries of any solution of a shortest vector problem (SVP) when its basis matrix is LLL reduced. These upper bounds are useful for analyzing the complexity and understanding numerical stability of the basis expansion in a KZ reduction algorithm. Finally, we propose a new KZ reduction algorithm by modifying the commonly used Schnorr-Euchner search strategy for solving SVPs and the basis expansion method proposed by Zhang {\em et al.} Simulation results show that the new KZ reduction algorithm is much faster and more numerically reliable than the KZ reduction algorithm proposed by Zhang {\em et al.}, especially when the basis matrix is ill conditioned.

cs.IT↗
arXiv2017

A Novel Approach for Ellipsoidal Outer-Approximation of the Intersection Region of Ellipses in the Plane

In this paper, a novel technique for tight outer-approximation of the intersection region of a finite number of ellipses in 2-dimensional (2D) space is proposed. First, the vertices of a tight polygon that contains the convex intersection of the ellipses are found in an efficient manner. To do so, the intersection points of the ellipses that fall on the boundary of the intersection region are determined, and a set of points is generated on the elliptic arcs connecting every two neighbouring intersection points. By finding the tangent lines to the ellipses at the extended set of points, a set of half-planes is obtained, whose intersection forms a polygon. To find the polygon more efficiently, the points are given an order and the intersection of the half-planes corresponding to every two neighbouring points is calculated. If the polygon is convex and bounded, these calculated points together with the initially obtained intersection points will form its vertices. If the polygon is non-convex or unbounded, we can detect this situation and then generate additional discrete points only on the elliptical arc segment causing the issue, and restart the algorithm to obtain a bounded and convex polygon. Finally, the smallest area ellipse that contains the vertices of the polygon is obtained by solving a convex optimization problem. Through numerical experiments, it is illustrated that the proposed technique returns a tighter outer-approximation of the intersection of multiple ellipses, compared to conventional techniques, with only slightly higher computational cost.

cs.CG↗
arXiv2017

On the Success Probability of the Box-Constrained Rounding and Babai Detectors

In communications, one frequently needs to detect a parameter vector $\hbx$ in a box from a linear model. The box-constrained rounding detector $\x^\sBR$ and Babai detector $\x^\sBB$ are often used to detect $\hbx$ due to their high probability of correct detection, which is referred to as success probability, and their high efficiency of implimentation. It is generally believed that the success probability $P^\sBR$ of $\x^\sBR$ is not larger than the success probability $P^\sBB$ of $\x^\sBB$. In this paper, we first present formulas for $P^\sBR$ and $P^\sBB$ for two different situations: $\hbx$ is deterministic and $\hbx$ is uniformly distributed over the constraint box. Then, we give a simple example to show that $P^\sBR$ may be strictly larger than $P^\sBB$ if $\hbx$ is deterministic, while we rigorously show that $P^\sBR\leq P^\sBB$ always holds if $\hbx$ is uniformly distributed over the constraint box.

cs.IT↗
arXiv2016

Success probability of the Babai estimators for box-constrained integer linear models

In many applications including communications, one may encounter a linear model where the parameter vector $\hbx$ is an integer vector in a box. To estimate $\hbx$, a typical method is to solve a box-constrained integer least squares (BILS) problem. However, due to its high complexity, the box-constrained Babai integer point $\x^\sBB$ is commonly used as a suboptimal solution. In this paper, we first derive formulas for the success probability $P^\sBB$ of $\x^\sBB$ and the success probability $P^\sOB$ of the ordinary Babai integer point $\x^\sOB$ when $\hbx$ is uniformly distributed over the constraint box. Some properties of $P^\sBB$ and $P^\sOB$ and the relationship between them are studied. Then, we investigate the effects of some column permutation strategies on $¶^\sBB$. In addition to V-BLAST and SQRD, we also consider the permutation strategy involved in the LLL lattice reduction, to be referred to as LLL-P. On the one hand, we show that when the noise is relatively small, LLL-P always increases $P^\sBB$ and argue why both V-BLAST and SQRD often increase $P^\sBB$; and on the other hand, we show that when the noise is relatively large, LLL-P always decreases $P^\sBB$ and argue why both V-BLAST and SQRD often decrease $P^\sBB$. We also derive a column permutation invariant bound on $P^\sBB$, which is an upper bound and a lower bound under these two opposite conditions, respectively. Numerical results demonstrate our findings. Finally, we consider a conjecture concerning $\x^\sOB$ proposed by Ma et al. We first construct an example to show that the conjecture does not hold in general, and then show that it does hold under some conditions.

cs.IT↗
arXiv2016

An Efficient Algorithm for Optimally Solving a Shortest Vector Problem in Compute-and-Forward Protocol Design

We consider the problem of finding the optimal coefficient vector that maximizes the computation rate at a relay in the compute-and-forward scheme. Based on the idea of sphere decoding, we propose a highly efficient algorithm that finds the optimal coefficient vector. First, we derive a novel algorithm to transform the original quadratic form optimization problem into a shortest vector problem (SVP) using the Cholesky factorization. Instead of computing the Cholesky factor explicitly, the proposed algorithm realizes the Cholesky factorization with only $\bigO(n)$ flops by taking advantage of the structure of the Gram matrix in the quadratic form. Then, we propose some conditions that can be checked with $\bigO(n)$ flops, under which a unit vector is the optimal coefficient vector. Finally, by taking into account some useful properties of the optimal coefficient vector, we modify the Schnorr-Euchner search algorithm to solve the SVP. We show that the estimated average complexity of our new algorithm is $\bigO(n^{1.5}P^{0.5})$ flops for i.i.d. Gaussian channel entries with SNR $P$ based on the Gaussian heuristic. Simulations show that our algorithm is not only much more efficient than the existing ones that give the optimal solution, but also faster than some best known suboptimal methods. Besides, we show that our algorithm can be readily adapted to output a list of $L$ best candidate vectors for use in the compute-and-forward design. The estimated average complexity of the resultant list-output algorithm is $\bigO\left(n^{1.5}P^{0.5}\log L + nL\right)$ flops for i.i.d. Gaussian channel entries.

cs.IT↗
arXiv2015

A Modified KZ Reduction Algorithm

The Korkine-Zolotareff (KZ) reduction has been used in communications and cryptography. In this paper, we modify a very recent KZ reduction algorithm proposed by Zhang et al., resulting in a new algorithm, which can be much faster and more numerically reliable, especially when the basis matrix is ill conditioned.

cs.IT↗
arXiv2014

Effects of the LLL reduction on the success probability of the Babai point and on the complexity of sphere decoding

The common method to estimate an unknown integer parameter vector in a linear model is to solve an integer least squares (ILS) problem. A typical approach to solving an ILS problem is sphere decoding. To make a sphere decoder faster, the well-known LLL reduction is often used as preprocessing. The Babai point produced by the Babai nearest plan algorithm is a suboptimal solution of the ILS problem. First we prove that the success probability of the Babai point as a lower bound on the success probability of the ILS estimator is sharper than the lower bound given by Hassibi and Boyd [1]. Then we show rigorously that applying the LLL reduction algorithm will increase the success probability of the Babai point. Finally we show rigorously that applying the LLL reduction algorithm will also reduce the computational complexity of sphere decoders, which is measured approximately by the number of nodes in the search tree in the literature

cs.IT↗
arXiv2014

A Joint Localization and Clock Bias Estimation Technique Using Time-of-Arrival at Multiple Antenna Receivers

In this work, a system scheme is proposed for tracking a radio emitting target moving in two-dimensional space. The localization is based on the use of biased time-of-arrival (TOA) measurements obtained at two asynchronous receivers, each equipped with two closely spaced antennas. By exploiting the multi-antenna configuration and using all the TOA measurements up to current time step, the relative clock bias at each receiver and the target position are jointly estimated by solving a nonlinear least-squares (NLS) problem. To this end, a novel time recursive algorithm is proposed which fully takes advantage of the problem structure to achieve computational efficiency while using orthogonal transformations to ensure numerical reliability. Simulations show that the mean-squared error (MSE) of the proposed method is much smaller than that of existing methods with the same antenna scheme, and approaches the Cramer-Rao lower bound (CRLB) closely.

eess.SY↗