arXiv subjects

Wei Gao

Publications and source records attributed to Wei Gao.

328 records · Page 19Linked to original sources

arXiv2013

On the Doubt about Margin Explanation of Boosting

Margin theory provides one of the most popular explanations to the success of \texttt{AdaBoost}, where the central point lies in the recognition that \textit{margin} is the key for characterizing the performance of \texttt{AdaBoost}. This theory has been very influential, e.g., it has been used to argue that \texttt{AdaBoost} usually does not overfit since it tends to enlarge the margin even after the training error reaches zero. Previously the \textit{minimum margin bound} was established for \texttt{AdaBoost}, however, \cite{Breiman1999} pointed out that maximizing the minimum margin does not necessarily lead to a better generalization. Later, \cite{Reyzin:Schapire2006} emphasized that the margin distribution rather than minimum margin is crucial to the performance of \texttt{AdaBoost}. In this paper, we first present the \textit{$k$th margin bound} and further study on its relationship to previous work such as the minimum margin bound and Emargin bound. Then, we improve the previous empirical Bernstein bounds \citep{Maurer:Pontil2009,Audibert:Munos:Szepesvari2009}, and based on such findings, we defend the margin-based explanation against Breiman's doubts by proving a new generalization error bound that considers exactly the same factors as \cite{Schapire:Freund:Bartlett:Lee1998} but is sharper than \cite{Breiman1999}'s minimum margin bound. By incorporating factors such as average margin and variance, we present a generalization error bound that is heavily related to the whole margin distribution. We also provide margin distribution bounds for generalization error of voting classifiers in finite VC-dimension space.

cs.LG↗

arXiv2013

Online dictionary learning for kernel LMS. Analysis and forward-backward splitting algorithm

Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This dramatically increases the computational burden and memory requirement. A variety of strategies based on dictionary learning have been proposed to overcome this severe drawback. Few, if any, of these works analyze the problem of updating the dictionary in a time-varying environment. In this paper, we present an analytical study of the convergence behavior of the Gaussian least-mean-square algorithm in the case where the statistics of the dictionary elements only partially match the statistics of the input data. This allows us to emphasize the need for updating the dictionary in an online way, by discarding the obsolete elements and adding appropriate ones. We introduce a kernel least-mean-square algorithm with L1-norm regularization to automatically perform this task. The stability in the mean of this method is analyzed, and its performance is tested with experiments.

stat.ML↗

arXiv2013

One-Pass AUC Optimization

AUC is an important performance measure and many algorithms have been devoted to AUC optimization, mostly by minimizing a surrogate convex loss on a training data set. In this work, we focus on one-pass AUC optimization that requires only going through the training data once without storing the entire training dataset, where conventional online learning algorithms cannot be applied directly because AUC is measured by a sum of losses defined over pairs of instances from different classes. We develop a regression-based algorithm which only needs to maintain the first and second order statistics of training data in memory, resulting a storage requirement independent from the size of training data. To efficiently handle high dimensional data, we develop a randomized algorithm that approximates the covariance matrices by low rank matrices. We verify, both theoretically and empirically, the effectiveness of the proposed algorithm.

cs.LG↗

arXiv2011

High-Speed Propulsion of Flexible Nanowire Motors: Theory and Experiments

Micro/nano-scale propulsion has attracted considerable recent attention due to its promise for biomedical applications such as targeted drug delivery. In this paper, we report on a new experimental design and theoretical modelling of high-speed fuel-free magnetically-driven propellers which exploit the flexibility of nanowires for propulsion. These readily prepared nanomotors display both high dimensional propulsion velocities (up to ~ 21 micrometer per second) and dimensionless speeds (in body lengths per revolution) when compared with natural microorganisms and other artificial propellers. Their propulsion characteristics are studied theoretically using an elastohydrodynamic model which takes into account the elasticity of the nanowire and its hydrodynamic interaction with the fluid medium. The critical role of flexibility in this mode of propulsion is illustrated by simple physical arguments, and is quantitatively investigated with the help of an asymptotic analysis for small-amplitude swimming. The theoretical predictions are then compared with experimental measurements and we obtain good agreement. Finally, we demonstrate the operation of these nanomotors in a real biological environment (human serum), emphasizing the robustness of their propulsion performance and their promise for biomedical applications.

physics.flu-dyn↗