arXiv Science⌕ Search

arXiv · cs/9905012

Linear and Order Statistics Combiners for Pattern Classification

Abstract

Several researchers have experimentally shown that substantial improvements can be obtained in difficult pattern recognition problems by combining or integrating the outputs of multiple classifiers. This chapter provides an analytical framework to quantify the improvements in classification results due to combining. The results apply to both linear combiners and order statistics combiners. We first show that to a first order approximation, the error rate obtained over and above the Bayes error rate, is directly proportional to the variance of the actual decision boundaries around the Bayes optimum boundary. Combining classifiers in output space reduces this variance, and hence reduces the "added" error. If N unbiased classifiers are combined by simple averaging, the added error rate can be reduced by a factor of N if the individual errors in approximating the decision boundaries are uncorrelated. Expressions are then derived for linear combiners which are biased or correlated, and the effect of output correlations on ensemble performance is quantified. For order statistics based non-linear combiners, we derive expressions that indicate how much the median, the maximum and in general the ith order statistic can improve classifier performance. The analysis presented here facilitates the understanding of the relationships among error rates, classifier boundary distributions, and combining in output space. Experimental results on several public domain data sets are provided to illustrate the benefits of combining and to support the analytical results.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kagan Tumer, Joydeep Ghosh. 1999-05-20. Linear and Order Statistics Combiners for Pattern Classification. https://arxiv.science/abs/cs/9905012

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Relocating Nonlinearity: How a Downstream Learner Reshapes What Genetic Programming Must Evolve

Genetic programming was conceived as a way of evolving solutions: the program is the answer, and fitness is the error of its own output. A substantial line of work instead makes the program an input to a separate learner, so fitness measures the learner's output. Which learner to attach matters and depends on the problem, with no account of what decides it. What makes a target hard for genetic programming is how much nonlinearity the program must build by composing primitives; any nonlinearity the learner supplies, the program is spared. Here we show that how much the program must still build is decided by which learner is attached, and is measurable on the programs themselves. In Boolean domains, where a target's nonlinearity is exactly its Fourier degree, we tune that degree from one to six with everything else fixed. Conditioned on success, a linear learner forces the evolved program to the target's degree exactly, at one through six without exception over thirty runs per setting, while tree ensembles succeed with far simpler programs and four times as often. Where the target's structure is known, the learner can be chosen from that structure rather than from the target's reputation for difficulty: our control target has lower degree than the parity problems yet gains nothing from a nonlinear learner. Where it is not, the same quantities serve as diagnostics: the degree of the evolved program is free to compute during any run and shows whether the learner is contributing nothing or carrying the model; on two measured datasets the resulting procedure picks the right learner before any evolution is run. A warning follows: evolution internalises what the learner supplies in only four of sixty-three conditions and grows more dependent on it in forty-four. The more capable the learner, the less of the model is left legible in the program.

cs.NE↗

DeNA-SGN: Deterministic Neighborhood Aggregation for Verified Spiking Graph Inference

Graph learning on neuromorphic substrates requires an explicit spiking realization of neighborhood aggregation, supporting control and classification. We propose DeNA-SGN, which combines fixed OR and tie-inclusive Majority aggregation of binary node features with a learned signed-weight threshold classifier. A degree-dependent inhibitory construction implements Majority through shared control neurons, enabling a complete circuit compilation into SuperNeuroMAT. Across seven datasets and 41 mapped classifiers, native simulation matches the exported ANN on every evaluated class decision and output-spike pattern. On the new confirmatory Amazon Photo dataset, DeNA-SGN achieves 83.93+-0.93% current-readout accuracy and 71.61+-2.22% strict output-spike accuracy; Wiki-CS reaches 75.99+-0.54% over 20 public masks. In the matched Cora study, we achieve 74.90+-2.67% on current-readout and 68.57+-2.94% strict spike accuracy. Complete circuit and activity accounting supports analytical energy comparisons and identifies the cost of clocking inactive neurons in large deployments. Current argmax is decoded on the host, and energy is modeled rather than measured on hardware. These findings establish a reproducible route to useful graph classification with decision-matched spiking execution and explicit implementation costs.

cs.NE↗

Evolving Neural Solvers for Multi-objective Combinatorial Optimization

Multi-objective combinatorial optimization (MOCO) seeks high-quality solutions that represent different trade-offs among conflicting objectives. Existing neural methods typically train a dedicated model for each preference or condition a single model on different preferences. Training dedicated models becomes more costly as the number of preferences increases, while a shared model can produce duplicate solutions for different preferences, limiting the benefit of denser preference queries. We propose the Evolving Neural Solver (ENS), which trains only a small set of preference-specific models and takes advantage of evolutionary computation to evolve them into a diverse collection of solvers for better solution quality and Pareto front coverage. ENS first trains a model at an estimated knee-point preference to target a balanced compromise, then obtains the remaining initial models through parameter transfer toward the extremes, so as to preliminarily approximate the Pareto front. It then evolves these models to find more candidates that can generate non-dominated solutions through evolution strategies. Experimental results on classic MOCO problems show that ENS outperforms state-of-the-art neural MOCO methods in terms of hypervolume in most evaluated settings.

cs.NE↗