arXiv subjects

Ping Li

Publications and source records attributed to Ping Li.

At least 451 records · Page 25Linked to original sources

arXiv2012

Improving Compressed Counting

Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath frequency moments as a -> 1. Monitoring Shannon entropy for anomaly detection (e.g., DDoS attacks) in large networks is an important task. This paper presents a new algorithm for improving CC. The improvement is most substantial when a -> 1--. For example, when a = 0:99, the new algorithm reduces the estimation variance roughly by 100-fold. This new algorithm would make CC considerably more practical for estimating Shannon entropy. Furthermore, the new algorithm is statistically optimal when a = 0.5.

cs.DS↗

arXiv2012

Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost

Logitboost is an influential boosting algorithm for classification. In this paper, we develop robust logitboost to provide an explicit formulation of tree-split criterion for building weak learners (regression trees) for logitboost. This formulation leads to a numerically stable implementation of logitboost. We then propose abc-logitboost for multi-class classification, by combining robust logitboost with the prior work of abc-boost. Previously, abc-boost was implemented as abc-mart using the mart algorithm. Our extensive experiments on multi-class classification compare four algorithms: mart, abcmart, (robust) logitboost, and abc-logitboost, and demonstrate the superiority of abc-logitboost. Comparisons with other learning methods including SVM and deep learning are also available through prior publications.

cs.LG↗

arXiv2012

Approximating Higher-Order Distances Using Random Projections

We provide a simple method and relevant theoretical analysis for efficiently estimating higher-order lp distances. While the analysis mainly focuses on l4, our methodology extends naturally to p = 6,8,10..., (i.e., when p is even). Distance-based methods are popular in machine learning. In large-scale applications, storing, computing, and retrieving the distances can be both space and time prohibitive. Efficient algorithms exist for estimating lp distances if 0 < p <= 2. The task for p > 2 is known to be difficult. Our work partially fills this gap.

cs.LG↗

arXiv2011

On an algebraic formula and applications to group action on manifolds

We consider a purely algebraic result. Then given a circle or cyclic group of prime order action on a manifold, we will use it to estimate the lower bound of the number of fixed points. We also give an obstruction to the existence of $\mathbb{Z}_p$ action on manifolds with isolated fixed points when $p$ is a prime.

math.AT↗

arXiv2011

User-level sentiment analysis incorporating social networks

We show that information about social relationships can be used to improve user-level sentiment analysis. The main motivation behind our approach is that users that are somehow "connected" may be more likely to hold similar opinions; therefore, relationship information can complement what we can extract about a user's viewpoints from their utterances. Employing Twitter as a source for our experimental data, and working within a semi-supervised framework, we propose models that are induced either from the Twitter follower/followee network or from the network in Twitter formed by users referring to each other using "@" mentions. Our transductive learning results reveal that incorporating social-network information can indeed lead to statistically significant sentiment-classification improvements over the performance of an approach based on Support Vector Machines having access only to textual features.

cs.CL↗

arXiv2011

Training Logistic Regression and SVM on 200GB Data Using b-Bit Minwise Hashing and Comparisons with Vowpal Wabbit (VW)

We generated a dataset of 200 GB with 10^9 features, to test our recent b-bit minwise hashing algorithms for training very large-scale logistic regression and SVM. The results confirm our prior work that, compared with the VW hashing algorithm (which has the same variance as random projections), b-bit minwise hashing is substantially more accurate at the same storage. For example, with merely 30 hashed values per data point, b-bit minwise hashing can achieve similar accuracies as VW with 2^14 hashed values per data point. We demonstrate that the preprocessing cost of b-bit minwise hashing is roughly on the same order of magnitude as the data loading time. Furthermore, by using a GPU, the preprocessing cost can be reduced to a small fraction of the data loading time. Minwise hashing has been widely used in industry, at least in the context of search. One reason for its popularity is that one can efficiently simulate permutations by (e.g.,) universal hashing. In other words, there is no need to store the permutation matrix. In this paper, we empirically verify this practice, by demonstrating that even using the simplest 2-universal hashing does not degrade the learning performance.

cs.LG↗

arXiv2011

Accurate Estimators for Improving Minwise Hashing and b-Bit Minwise Hashing

Minwise hashing is the standard technique in the context of search and databases for efficiently estimating set (e.g., high-dimensional 0/1 vector) similarities. Recently, b-bit minwise hashing was proposed which significantly improves upon the original minwise hashing in practice by storing only the lowest b bits of each hashed value, as opposed to using 64 bits. b-bit hashing is particularly effective in applications which mainly concern sets of high similarities (e.g., the resemblance >0.5). However, there are other important applications in which not just pairs of high similarities matter. For example, many learning algorithms require all pairwise similarities and it is expected that only a small fraction of the pairs are similar. Furthermore, many applications care more about containment (e.g., how much one object is contained by another object) than the resemblance. In this paper, we show that the estimators for minwise hashing and b-bit minwise hashing used in the current practice can be systematically improved and the improvements are most significant for set pairs of low resemblance and high containment.

stat.ML↗

arXiv2011

Optimizing and controlling functions of complex networks by manipulating rich-club connections

Traditionally, there is no evidence suggesting that there are strong ties between the rich-club property and the function of complex networks. In this study, we find that whether a very small portion of rich nodes connected to each other or not can strongly affect the frequency of occurrence of basic building blocks (motif) within networks, and therefore the function, of a heterogeneous network. Conversely whether a homogeneous network has a rich-club property or not generally has no significant effect on its structure and function. These findings open the possibility to optimize and control the function of complex networks by manipulating rich-club connections. Furthermore, based on the subgraph ratio profile, we develop a more rigorous approach to judge whether a network has a rich-club or not. The new method does not calculate how many links there are among rich nodes but depends on how the links among rich nodes can affect the overall structure as well as function of a given network. These results can also help us to understand the evolution of dynamical networks and design new models for characterizing real-world networks.

physics.soc-ph↗

arXiv2011

Hashing Algorithms for Large-Scale Learning

In this paper, we first demonstrate that b-bit minwise hashing, whose estimators are positive definite kernels, can be naturally integrated with learning algorithms such as SVM and logistic regression. We adopt a simple scheme to transform the nonlinear (resemblance) kernel into linear (inner product) kernel; and hence large-scale problems can be solved extremely efficiently. Our method provides a simple effective solution to large-scale learning in massive and extremely high-dimensional datasets, especially when data do not fit in memory. We then compare b-bit minwise hashing with the Vowpal Wabbit (VW) algorithm (which is related the Count-Min (CM) sketch). Interestingly, VW has the same variances as random projections. Our theoretical and empirical comparisons illustrate that usually $b$-bit minwise hashing is significantly more accurate (at the same storage) than VW (and random projections) in binary data. Furthermore, $b$-bit minwise hashing can be combined with VW to achieve further improvements in terms of training speed, especially when $b$ is large.

stat.ML↗

arXiv2011

b-Bit Minwise Hashing for Large-Scale Linear SVM

In this paper, we propose to (seamlessly) integrate b-bit minwise hashing with linear SVM to substantially improve the training (and testing) efficiency using much smaller memory, with essentially no loss of accuracy. Theoretically, we prove that the resemblance matrix, the minwise hashing matrix, and the b-bit minwise hashing matrix are all positive definite matrices (kernels). Interestingly, our proof for the positive definiteness of the b-bit minwise hashing kernel naturally suggests a simple strategy to integrate b-bit hashing with linear SVM. Our technique is particularly useful when the data can not fit in memory, which is an increasingly critical issue in large-scale machine learning. Our preliminary experimental results on a publicly available webspam dataset (350K samples and 16 million dimensions) verified the effectiveness of our algorithm. For example, the training time was reduced to merely a few seconds. In addition, our technique can be easily extended to many other linear and nonlinear machine learning applications such as logistic regression.

cs.LG↗

arXiv2011

Some remarks on circle action on manifolds

This paper contains several results concerning circle action on almost-complex and smooth manifolds. More precisely, we show that, for an almost-complex manifold $M^{2mn}$(resp. a smooth manifold $N^{4mn}$), if there exists a partition $λ=(λ_{1},...,λ_{u})$ of weight $m$ such that the Chern number $(c_{λ_{1}}... c_{λ_{u}})^{n}[M]$ (resp. Pontrjagin number $(p_{λ_{1}}... p_{λ_{u}})^{n}[N]$) is nonzero, then \emph{any} circle action on $M^{2mn}$ (resp. $N^{4mn}$) has at least $n+1$ fixed points. When an even-dimensional smooth manifold $N^{2n}$ admits a semi-free action with isolated fixed points, we show that $N^{2n}$ bounds, which generalizes a well-known fact in the free case. We also provide a topological obstruction, in terms of the first Chern class, to the existence of semi-free circle action with \emph{nonempty} isolated fixed points on almost-complex manifolds. The main ingredients of our proofs are Bott's residue formula and rigidity theorem.

math.AT↗

arXiv2011

The rigidity of Dolbeault-type operators and symplectic circle actions

Following the idea of Lusztig, Atiyah-Hirzebruch and Kosniowski, we note that the Dolbeault-type operators on compact, almost-complex manifolds are rigid. When the circle action has isolated fixed points, this rigidity result will produce many identities concerning the weights on the fixed points. In particular, it gives a criterion to detemine whether or not a symplectic circle action with isolated fixed points is Hamiltonian. As applications, we simplify the proofs of some known results related to symplectic circle actions, due to Godinho, Tolman-Weitsman and Pelayo-Tolman, and generalize some of them to more general cases.

math.SG↗

arXiv2010

Chern numbers and the indices of some elliptic differential operators

Libgober and Wood proved that the Chern number $c_{1}c_{n-1}$ of a $n$-dimensional compact complex manifold can be determined by its Hirzebruch $χ_{y}$-genus. Inspired by the idea of their proof, we show that, for compact, spin, almost-complex manifolds, more Chern numbers can be determined by the indices of some twisted Dirac and signature operators. As a byproduct, we get a divisibility result of certain characteristic number for such manifolds. Using our method, we give a direct proof of Libgober-Wood's result, which was originally proved by induction.

math.DG↗

arXiv2010

Circle action and some vanishing results on manifolds

Kawakubo and Uchida showed that, if a closed oriented $4k$-dimensional manifold $M$ admits a semi-free circle action such that the dimension of the fixed point set is less than $2k$, then the signature of $M$ vanishes. In this note, by using $G$-signature theorem and the rigidity of the signature operator, we generalize this result to more general circle actions. Combining the same idea with the remarkable Witten-Taubes-Bott rigidity theorem, we explore more vanishing results on spin manifolds admitting such circle actions. Our results are closely related to some earlier results of Conner-Floyd, Landweber-Stong and Hirzebruch-Slodowy.

math.AT↗

arXiv2010

Fast ABC-Boost for Multi-Class Classification

Abc-boost is a new line of boosting algorithms for multi-class classification, by utilizing the commonly used sum-to-zero constraint. To implement abc-boost, a base class must be identified at each boosting step. Prior studies used a very expensive procedure based on exhaustive search for determining the base class at each boosting step. Good testing performances of abc-boost (implemented as abc-mart and abc-logitboost) on a variety of datasets were reported. For large datasets, however, the exhaustive search strategy adopted in prior abc-boost algorithms can be too prohibitive. To overcome this serious limitation, this paper suggests a heuristic by introducing Gaps when computing the base class during training. That is, we update the choice of the base class only for every $G$ boosting steps (i.e., G=1 in prior studies). We test this idea on large datasets (Covertype and Poker) as well as datasets of moderate sizes. Our preliminary results are very encouraging. On the large datasets, even with G=100 (or larger), there is essentially no loss of test accuracy. On the moderate datasets, no obvious loss of test accuracy is observed when G<= 20~50. Therefore, aided by this heuristic, it is promising that abc-boost will be a practical tool for accurate multi-class classification.

cs.LG↗

arXiv2010

On Practical Algorithms for Entropy Estimation and the Improved Sample Complexity of Compressed Counting

Estimating the p-th frequency moment of data stream is a very heavily studied problem. The problem is actually trivial when p = 1, assuming the strict Turnstile model. The sample complexity of our proposed algorithm is essentially O(1) near p=1. This is a very large improvement over the previously believed O(1/eps^2) bound. The proposed algorithm makes the long-standing problem of entropy estimation an easy task, as verified by the experiments included in the appendix.

cs.DS↗

arXiv2010

An Empirical Evaluation of Four Algorithms for Multi-Class Classification: Mart, ABC-Mart, Robust LogitBoost, and ABC-LogitBoost

This empirical study is mainly devoted to comparing four tree-based boosting algorithms: mart, abc-mart, robust logitboost, and abc-logitboost, for multi-class classification on a variety of publicly available datasets. Some of those datasets have been thoroughly tested in prior studies using a broad range of classification algorithms including SVM, neural nets, and deep learning. In terms of the empirical classification errors, our experiment results demonstrate: 1. Abc-mart considerably improves mart. 2. Abc-logitboost considerably improves (robust) logitboost. 3. Robust) logitboost} considerably improves mart on most datasets. 4. Abc-logitboost considerably improves abc-mart on most datasets. 5. These four boosting algorithms (especially abc-logitboost) outperform SVM on many datasets. 6. Compared to the best deep learning methods, these four boosting algorithms (especially abc-logitboost) are competitive.

cs.LG↗

arXiv2009

b-Bit Minwise Hashing

This paper establishes the theoretical framework of b-bit minwise hashing. The original minwise hashing method has become a standard technique for estimating set similarity (e.g., resemblance) with applications in information retrieval, data management, social networks and computational advertising. By only storing the lowest $b$ bits of each (minwise) hashed value (e.g., b=1 or 2), one can gain substantial advantages in terms of computational efficiency and storage space. We prove the basic theoretical results and provide an unbiased estimator of the resemblance for any b. We demonstrate that, even in the least favorable scenario, using b=1 may reduce the storage space at least by a factor of 21.3 (or 10.7) compared to using b=64 (or b=32), if one is interested in resemblance > 0.5.

cs.DS↗