arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 0704.2725

Exploiting Heavy Tails in Training Times of Multilayer Perceptrons: A Case Study with the UCI Thyroid Disease Database

Manuel Cebrian·Ivan Cantador
View original source ↗Read source document ↗Export metadata ↓

Abstract

The random initialization of weights of a multilayer perceptron makes it possible to model its training process as a Las Vegas algorithm, i.e. a randomized algorithm which stops when some required training error is obtained, and whose execution time is a random variable. This modeling is used to perform a case study on a well-known pattern recognition benchmark: the UCI Thyroid Disease Database. Empirical evidence is presented of the training time probability distribution exhibiting a heavy tail behavior, meaning a big probability mass of long executions. This fact is exploited to reduce the training time cost by applying two simple restart strategies. The first assumes full knowledge of the distribution yielding a 40% cut down in expected time with respect to the training without restarts. The second, assumes null knowledge, yielding a reduction ranging from 9% to 23%.

Explore related subjects

cs.NE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Manuel Cebrian, Ivan Cantador. 2007-12-07. Exploiting Heavy Tails in Training Times of Multilayer Perceptrons: A Case Study with the UCI Thyroid Disease Database. https://arxiv.org/abs/0704.2725

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Organisms Write Their Own Evolution: Direct Epigenomic Inheritance Accelerates Learning on Compositional Tasks

What if organisms could write their own evolution? For 150 years, evolution has been understood as one-way: genes provide the rules, organisms respond to selection. But if organisms could learn during their lifetime, write that learned knowledge directly into their epigenome -- the heritable regulatory layer controlling gene expression -- and pass it to offspring, then knowledge would accumulate across generations instead of being rediscovered by each one. Such direct epigenomic inheritance would constitute genuine organismal agency: writing new information into heredity rather than passively receiving it. To test whether organisms actually benefit from this capability, we implement Phenopoiesis, a computational model in which organisms maintain two independent heritable layers: a genetic layer (inherited with mutation) and an epigenetic layer (inherited unchanged). We compare Phenopoiesis against Selection Only (no learning) and the Baldwin Effect (learning influences selection but is not inherited) across six learning tasks of increasing complexity: three simple (learn one pattern) and three compositional (learn two patterns simultaneously). On simple tasks all three mechanisms perform equivalently (p = 0.559): complexity is required for inheritance to matter. On learnable compositional tasks, Phenopoiesis outperforms the alternatives by 3-6x in fitness (F = 177-358, p < 0.001, Cohen's d = 3.44-4.89), with perfect reproducibility (s.d. = 0.0 on T+Plus across all 30 runs), indicating deterministic knowledge accumulation: offspring inherit their parents' learned solutions and add new learning on top. On a geometrically incompatible task all mechanisms fail equally, revealing an inherent limit. These results give the first direct computational test of Denis Noble's hypothesis that organisms possess read-write access to their own heredity, and show precisely when that access pays off.

cs.NE↗
arXiv2026

Convex Optimization of Spiking Neural Networks with Global and Approximation Guarantees

Spiking neural networks (SNNs) have been proposed as energy-efficient alternatives to deep neural networks. Their training is non-convex and involves a non-differentiable spike function, so existing methods offer no guarantee of reaching a global optimum. We address this issue by formulating SNN optimization as an equivalent convex optimization problem. Specifically, we show that the weight-decay-regularized training problem for a class of threshold RNNs, including leaky integrate-and-fire SNNs, has zero duality gap and the same optimal value as a convex program. Since the resulting convex programs can have exponentially many variables in the amount of data, we introduce a practical randomized surrogate and establish a high-probability guarantee on its approximation quality. Empirically, the best convex variant outperforms surrogate-gradient training on addition and first-last-XOR, matches its performance on sequential MNIST, and trails a reservoir baseline on base-3 and base-5 addition.

cs.NE↗
arXiv2026

Evolving Hybrid Quantum-Classical Architectures for Image Classification

Hybrid quantum classical neural networks integrate parameterized quantum circuits (PQCs) with established deep learning architectures, but their performance depends strongly on the choice of quantum circuit architecture, a choice that remains largely manual. Most existing approaches rely on hand-designed or fixed circuit ansätze, requiring circuit structure, gate composition, and qubit connectivity to be specified in advance with no guarantee that they suit the task. This limitation is especially acute in image classification, where quantum circuits must transform features extracted by classical networks while remaining compact enough for practical training, requirements that generic, task-agnostic ansätze are unlikely to satisfy simultaneously. We extend EXAQC, an evolutionary framework for automated quantum circuit discovery, to image classification. EXAQC evolves PQCs as intermediate processing modules while retaining classical feature-extraction and prediction layers. On MNIST, Fashion-MNIST, and CIFAR-10, EXAQC achieves 98.42%, 90.62%, and 85.47% accuracy, respectively, while using comparable gate counts to other quantum architecture-search methods. Against classical networks, evolved hybrid models maintain comparable accuracy with substantially fewer trainable parameters, reaching 85.68% on CIFAR-10 with over 25$\times$ fewer parameters than a 10-layer CNN. Encoding choice also matters: rotation-based encodings (RX, RY, U3) outperform amplitude encoding by 22-25 points on CIFAR-10. These results demonstrate that automated circuit discovery yields compact quantum modules that can replace larger classical components in vision architectures while retaining competitive accuracy.

cs.NE↗