arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2005.12826

BHN: A Brain-like Heterogeneous Network

Tao Liu
View original source ↗Read source document ↗Export metadata ↓

Abstract

The human brain works in an unsupervised way, and more than one brain region is essential for lighting up intelligence. Inspired by this, we propose a brain-like heterogeneous network (BHN), which can cooperatively learn a lot of distributed representations and one global attention representation. By optimizing distributed, self-supervised, and gradient-isolated objective functions in a minimax fashion, our model improves its representations, which are generated from patches of pictures or frames of videos in experiments.

Explore related subjects

cs.NEcs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Tao Liu. 2020-06-06. BHN: A Brain-like Heterogeneous Network. https://arxiv.org/abs/2005.12826

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Black-Box Optimization of Mixed Binary-Continuous Variables: Challenges and Opportunities in Evolutionary Model Merging

Model merging has emerged as a cost-effective paradigm for combining capabilities of multiple large language models (LLMs) without additional training. Among existing approaches, Data Flow Space (DFS) merging operates by selectively activating parameter groups across model layers, offering greater flexibility than naive parameter averaging. However, the optimization challenges underlying DFS merging remain poorly understood. In this paper, we formally characterize the DFS merging problem as a mixed binary-continuous black-box optimization problem, identifying three fundamental challenges: (1) mixed binary-continuous variable types, (2) high-dimensional search spaces with up to 291 parameter groups, and (3) conditional variable dependencies corresponding to Type-I interaction as characterized by Akimoto et al. [1]. We empirically evaluate seven merging methods on Mistral-7B models across official GSM8K (1,319 questions) and MMLU (14,042 questions, 57 subjects) benchmarks. Our structured approach achieves a 49.5% reduction in effective search space. Results reveal that all tested merging methods fail to surpass the stronger source model on official benchmarks, with TIES (13.72%) and DELLA (4.12%) performing catastrophically on GSM8K. Notably, DELLA scores below even the base model (6.67%), demonstrating that standard parameter space methods fail when merging base and instruction-tuned models. These findings directly validate the difficulty of the Type-I interaction structure we identify, and motivate the application of algorithms specifically designed for this challenge such as the concurrent IGBD work of Ong et al. [2].

cs.NE↗
arXiv2026

SpikeQ: Spike-Guided Post-Training Quantization for Spiking Neural Networks

On neuromorphic chips, spiking neural networks (SNNs) consume far less energy than artificial neural networks (ANNs), but the limited memory of these chips requires low-bit synaptic weights and membrane potentials. Existing post-training quantization (PTQ) methods provide them without retraining and, like those for ANNs, minimize numerical errors such as the mean squared error (MSE). In an SNN, however, a quantization error reaches later layers only through the spikes it flips. We find that errors of equal MSE can cause very different accuracy losses, whereas errors that flip the same fraction of spikes cause the same loss. We therefore propose SpikeQ, a spike-guided PTQ framework. For weights, we select the level of each weight by simulating the downstream neurons and minimizing their spike errors. For membrane potentials, most existing methods quantize the state after the reset, whereas neuromorphic chips compare the threshold with the membrane potential stored at a fixed bit width. We target this pre-fire quantization and build a reset-aware grid that places the threshold on a rounding boundary, so rounding never flips the spike at the current time step. We evaluate SpikeQ on seven benchmarks of image classification, semantic segmentation, and object detection. With 2-bit pre-fire membrane potentials, every existing grid collapses, whereas SpikeQ retains most of the full-precision accuracy. With both at 4 bits, the models stay within 2.59 points of full precision, shrink their memory footprint 7.8--7.9 times, and use 6.4--6.9 times less energy.

cs.NE↗
arXiv2026

First Mathematical Runtime Analyses of Evolutionary Multitask Multi-Objective Optimization

To facilitate the solving of complex problems, evolutionary multitask optimization is regarded as a promising technique due to its capacity for cross-task knowledge transfer. In contrast to empirical evidences, no mathematical runtime support exists for this technique within the domain of multi-objective optimization. In this work, we aim to fill this important gap. We analyze how the representative multiobjective multifactorial evolutionary algorithm (MO-MFEA) covers the Pareto fronts of two bi-objective problems simultaneously. For the two-task problem of maximizing the classic bi-objective OMM and the multimodal OJZJ benchmarks, the MO-MFEA requires only an expected runtime of $O(n^2 \log n)$---the same as optimizing the simple OMM alone---yet achieves a speedup by a factor of $Θ(n^{k-1}/\log n)$ compared to optimizing OJZJ alone (with gap size $k$). Similarly, for the task pair comprising the classic LOTZ and the multimodal DLTB benchmarks, a speedup of $Θ(n)$ is attained over optimizing DLTB alone, while the runtime matches that of optimizing LOTZ alone.

cs.NE↗