Jiran Tao | arXiv Science

arXiv2026

Do Tabular Foundation Models Still Need Feature Engineering?

Feature engineering has long been a cornerstone of tabular machine learning. Tabular foundation models (TFMs) are pretrained on a wide range of tabular datasets and applied via in-context learning. Their rise raises a natural question: does manual feature construction still matter as these models become more capable? To answer this, we perform a controlled study across several versions of two major TFM families, testing a wide range of existing feature engineering techniques on benchmark datasets from TabArena. We find a consistent pattern: feature engineering gains are concentrated in earlier model generations and become negligible for the strongest models. These results suggest that stronger TFMs depend less on explicitly engineered input representations. In a complementary experiment, however, adding in-context information from related datasets still improves performance. Our findings indicate a shift in the source of performance gains for stronger TFMs: re-representing existing inputs becomes less effective, while providing additional task-relevant context remains beneficial.

cs.LG↗

arXiv2026

LLM-Enhanced Rumor Detection via Virtual Node Induced Edge Prediction

The rapid proliferation of rumors on social networks poses a significant threat to information integrity. While rumor dissemination forms complex structural patterns, existing detection methods often fail to capture the intricate interplay between textual coherence and propagation dynamics. Current approaches typically represent nodes through isolated textual embeddings, neglecting the semantic flow across the entire propagation path. To bridge this gap, we introduce a novel framework that integrates Large Language Models (LLMs) as a structural augmentation layer for graph-based rumor detection. Moving beyond conventional methods, our framework employs LLMs to evaluate information subchains and strategically introduce a virtual node into the graph. This structural modification converts latent semantic patterns into explicit topological features, effectively capturing the textual coherence that has historically been inaccessible to Graph Neural Networks (GNNs). To ensure reliability, we develop a structured prompt framework that mitigates inherent biases in LLMs while maintaining robust graph learning performance. Furthermore, our proposed framework is model-agnostic, meaning it is not constrained to any specific graph learning algorithm or LLMs. Its plug-and-play nature allows for seamless integration with further fine-tuned LLMs and graph techniques in the future, potentially enhancing predictive performance without the need to modify original algorithms.

cs.SI↗