arXiv Science⌕ Search

arXiv · 2611.17455

TSGP-2: On the Non-Linearity of Symbolic Regression Problems and How to Exploit It with Model-based Semantic Genetic Programming

Abstract

Model-based variation operators such as the one used in Transformer Semantic Genetic Programming (TSGP) are affected by possible biases of their training data. TSGP is trained, in particular, on programs generated by Genetic Programming (GP) runs on linear target functions, which leads to a bias toward linear offspring. This bias can hinder search performance on problems that require non-linear solutions. This work has two goals: first, it investigates the linearity of standard test problems such as the Feynman and real-world benchmark problems. While most of the structure of real-world problems can be well approximated by linear structures, the Feynman problems exhibit varying levels of non-linear structure. Thus, a proper variation operator of a search method should be able to create both types of programs, linear as well as non-linear ones. Motivated by this observation, we propose TSGP-2, which follows the functionality of TSGP but uses as training data programs from GP runs on both linear and non-linear target functions. As a result, the training data includes a lower share of programs that behave linearly (the share drops from 83% to 37%). With a higher share of non-linear structures, TSGP-2 outperforms TSGP on problems with non-linear structure, while performing similarly on linear problems. In addition, TSGP-2 is less biased toward linear offspring and is able to propose a higher number of meaningful non-linear offspring earlier during search, which coincides with better solutions. The findings underscore the importance of training data for a learned variation operator and emphasize that the biases of search operators should be aligned with the problems they are applied to.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Philipp Anthes, Alina Geiger, Martin Briesch, Franz Rothlauf. 2026-11-12. TSGP-2: On the Non-Linearity of Symbolic Regression Problems and How to Exploit It with Model-based Semantic Genetic Programming. https://arxiv.science/abs/2611.17455

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Relocating Nonlinearity: How a Downstream Learner Reshapes What Genetic Programming Must Evolve

Genetic programming was conceived as a way of evolving solutions: the program is the answer, and fitness is the error of its own output. A substantial line of work instead makes the program an input to a separate learner, so fitness measures the learner's output. Which learner to attach matters and depends on the problem, with no account of what decides it. What makes a target hard for genetic programming is how much nonlinearity the program must build by composing primitives; any nonlinearity the learner supplies, the program is spared. Here we show that how much the program must still build is decided by which learner is attached, and is measurable on the programs themselves. In Boolean domains, where a target's nonlinearity is exactly its Fourier degree, we tune that degree from one to six with everything else fixed. Conditioned on success, a linear learner forces the evolved program to the target's degree exactly, at one through six without exception over thirty runs per setting, while tree ensembles succeed with far simpler programs and four times as often. Where the target's structure is known, the learner can be chosen from that structure rather than from the target's reputation for difficulty: our control target has lower degree than the parity problems yet gains nothing from a nonlinear learner. Where it is not, the same quantities serve as diagnostics: the degree of the evolved program is free to compute during any run and shows whether the learner is contributing nothing or carrying the model; on two measured datasets the resulting procedure picks the right learner before any evolution is run. A warning follows: evolution internalises what the learner supplies in only four of sixty-three conditions and grows more dependent on it in forty-four. The more capable the learner, the less of the model is left legible in the program.

cs.NE↗

DeNA-SGN: Deterministic Neighborhood Aggregation for Verified Spiking Graph Inference

Graph learning on neuromorphic substrates requires an explicit spiking realization of neighborhood aggregation, supporting control and classification. We propose DeNA-SGN, which combines fixed OR and tie-inclusive Majority aggregation of binary node features with a learned signed-weight threshold classifier. A degree-dependent inhibitory construction implements Majority through shared control neurons, enabling a complete circuit compilation into SuperNeuroMAT. Across seven datasets and 41 mapped classifiers, native simulation matches the exported ANN on every evaluated class decision and output-spike pattern. On the new confirmatory Amazon Photo dataset, DeNA-SGN achieves 83.93+-0.93% current-readout accuracy and 71.61+-2.22% strict output-spike accuracy; Wiki-CS reaches 75.99+-0.54% over 20 public masks. In the matched Cora study, we achieve 74.90+-2.67% on current-readout and 68.57+-2.94% strict spike accuracy. Complete circuit and activity accounting supports analytical energy comparisons and identifies the cost of clocking inactive neurons in large deployments. Current argmax is decoded on the host, and energy is modeled rather than measured on hardware. These findings establish a reproducible route to useful graph classification with decision-matched spiking execution and explicit implementation costs.

cs.NE↗

Evolving Neural Solvers for Multi-objective Combinatorial Optimization

Multi-objective combinatorial optimization (MOCO) seeks high-quality solutions that represent different trade-offs among conflicting objectives. Existing neural methods typically train a dedicated model for each preference or condition a single model on different preferences. Training dedicated models becomes more costly as the number of preferences increases, while a shared model can produce duplicate solutions for different preferences, limiting the benefit of denser preference queries. We propose the Evolving Neural Solver (ENS), which trains only a small set of preference-specific models and takes advantage of evolutionary computation to evolve them into a diverse collection of solvers for better solution quality and Pareto front coverage. ENS first trains a model at an estimated knee-point preference to target a balanced compromise, then obtains the remaining initial models through parameter transfer toward the extremes, so as to preliminarily approximate the Pareto front. It then evolves these models to find more candidates that can generate non-dominated solutions through evolution strategies. Experimental results on classic MOCO problems show that ENS outperforms state-of-the-art neural MOCO methods in terms of hypervolume in most evaluated settings.

cs.NE↗