arXiv Science⌕ Search

arXiv subjects

Dmitry Panchenko

Publications and source records attributed to Dmitry Panchenko.

63 records · Page 4Linked to original sources

Some extensions of an inequality of Vapnik and Chervonenkis

The inequality of Vapnik and Chervonenkis controls the expectation of the function by its sample average uniformly over a VC-major class of functions taking into account the size of the expectation. Using Talagrand's kernel method we prove a similar result for the classes of functions for which Dudley's uniform entropy integral or bracketing entropy integral is finite.

math.PR↗

Empirical margin distributions and bounding the generalization error of combined classifiers

We prove new probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifiers. Such combinations could be implemented by neural networks or by voting methods of combining the classifiers, such as boosting and bagging. The bounds are in terms of the empirical distribution of the margin of the combined classifier. They are based on the methods of the theory of Gaussian and empirical processes (comparison inequalities, symmetrization method, concentration inequalities) and they improve previous results of Bartlett (1998) on bounding the generalization error of neural networks in terms of l_1-norms of the weights of neurons and of Schapire, Freund, Bartlett and Lee (1998) on bounding the generalization error of boosting. We also obtain rates of convergence in Levy distance of empirical margin distribution to the true margin distribution uniformly over the classes of classifiers and prove the optimality of these rates.

math.PR↗

Bounding the generalization error of convex combinations of classifiers: balancing the dimensionality and the margins

A problem of bounding the generalization error of a classifier f in H, where H is a "base" class of functions (classifiers), is considered. This problem frequently occurs in computer learning, where efficient algorithms of combining simple classifiers into a complex one (such as boosting and bagging) have attracted a lot of attention. Using Talagrand's concentration inequalities for empirical processes, we obtain new sharper bounds on the generalization error of combined classifiers that take into account both the empirical distribution of "classification margins'' and an "approximate dimension" of the classifiers and study the performance of these bounds in several experiments with learning algorithms.

math.PR↗

Symmetrization approach to concentration inequalities for empirical processes

We introduce a symmetrization technique that allows us to translate a problem of controlling the deviation of some functionals on a product space from their mean into a problem of controlling the deviation between two independent copies of the functional. As an application we give a new easy proof of Talagrand's concentration inequality for empirical processes, where besides symmetrization we use only Talagrand's concentration inequality on the discrete cube {-1,+1}^n. As another application of this technique we prove new Vapnik-Chervonenkis type inequalities. For example, for VC-classes of functions we prove a classical inequality of Vapnik and Chervonenkis only with normalization by the sum of variance and sample variance.

math.PR↗

Deviation inequality for monotonic Boolean functions with application to a number of k-cycles in a random graph

Using Talagrand's concentration inequality on the discrete cube {0,1}^m we show that given a real-valued function Z(x)on {0,1}^m that satisfies certain monotonicity conditions one can control the deviations of Z(x) above its median by a local Lipschitz norm of Z(x) at the point x. As one application, we give a simple proof of a nearly optimal deviation inequality for the number of k-cycles in a random graph.

math.PR↗

Bounds for diluted mean-fields spin glass models

In an important recent paper, \cite{FL}, S. Franz and M. Leone prove rigorous lower bounds for the free energy of the diluted $p$-spin model and the $K$-sat model at any temperature. We show that the results for these two models are consequences of a single general principle. Our calculations are significantly simpler than those of \cite{FL}, even in the replica-symmetric case.

math.PR↗

A note on the free energy of the coupled system in the Sherrington-Kirkpatrick model

In this paper we consider a system of spins that consists of two configurations $\vsi^1,\vsi^2\inΣ_N=\{-1,+1\}^N$ with Gaussian Hamiltonians $H_N^1(\vsi^1)$ and $H_N^2(\vsi^2)$ correspondingly, and these configurations are coupled on the set where their overlap is fixed $\{R_{1,2}=N^{-1}\sum_{i=1}^N σ_i^1σ_i^2 = u_N\}.$ We prove the existence of the thermodynamic limit of the free energy of this system given that $\lim_{N\to\infty}u_N = u\in[-1,1]$ and give the analogue of the Aizenman-Sims-Starr variational principle that describes this limit via random overlap structures.

math.PR↗

Rademacher processes and bounding the risk of function learning

We construct data dependent bounds on the risk in function learning problems. The bounds are based on the local norms of the Rademacher process indexed by the underlying function class and they do not require prior knowledge about the distribution of the training examples or any specific properties of the function class.

math.PR↗