arXiv Science⌕ Search

arXiv subjects

Yishuo Wang

Publications and source records attributed to Yishuo Wang.

5 recordsLinked to original sources

No Second Run: Does RSI Stop at Frontier Pre-training?

Pre-training is where every foundation model begins; bringing recursive self-improvement (RSI) there would let AI improve the process that creates AI. Yet public RSI work has so far targeted only post-training, which starts from an existing base model, runs trials costing GPU-hours, and returns verifiable scores. Pre-training breaks all three conditions at orders-of-magnitude higher cost: a run consumes hundreds of thousands of GPU-hours over months and cannot be repeated; its objective, post-training headroom, cannot be read off any single score; and what must improve is not a model but the whole production system, from data pipelines and training infrastructure to the teams that run them. To our knowledge, we are the first to apply RSI to an entire frontier pre-training run from scratch and to report it in detail. RSI does not stop at frontier pre-training, but it changes form: since no end-to-end loop can close over a single unrepeatable run, we decompose it into smaller loops that can, each repairing a step the full run breaks. The result is 801-30B-A3B-Base, a 30B-A3B MoE trained from scratch on 11.8T tokens in three months by a newly formed team. We release it as a base model; the lightweight SFT reported here (6B tokens, no RL) only probes post-training headroom. As a base model, 801-30B-A3B outperforms Qwen3-30B-A3B and Nemotron-3.5-Lightning-30B-A3B, trailing only the larger Qwen3.5-35B-A3B, and keeps this position under identical SFT; with lightweight SFT alone, it achieves the highest average on four agentic benchmarks against fully post-trained models, including Qwen3.6-35B-A3B. Since post-training RSI presupposes a strong base model, bringing RSI into pre-training acts on the foundation every later stage builds on. We open-source 801-30B-A3B-Base and the lightweight SFT data at https://github.com/GAIR-NLP/801-30B .

cs.AI↗

Asymptotic Flocking Behavior in Particle and Kinetic Systems with biological Memory Term

We investigate a Cucker-Smale type flocking model with a biological memory term and conclude that memory makes flocking harder. The model introduces $h(t)$ to describe memory, with dynamics $\frac{dh}{dt}=x(t)+v(t)-λh(t)$ reflecting path, velocity dependence and exponential decay. In microscopic level, the flocking phenomenon becomes conditional, requires communication kernel $ψ(t)$ has polynomial lower bound and memory deviation $H(t)$ decays exponentially. In mesoscopic level, flocking rates exhibit exponential or algebraic decay and the range of communication strength $β$ become smaller.

math.AP↗

CTP: A hybrid CNN-Transformer-PINN model for ocean front forecasting

This paper proposes CTP, a novel deep learning framework that integrates convolutional neural network(CNN), Transformer architectures, and physics-informed neural network(PINN) for ocean front prediction. Ocean fronts, as dynamic interfaces between distinct water masses, play critical roles in marine biogeochemical and physical processes. Existing methods such as LSTM, ConvLSTM, and AttentionConv often struggle to maintain spatial continuity and physical consistency over multi-step forecasts. CTP addresses these challenges by combining localized spatial encoding, long-range temporal attention, and physical constraint enforcement. Experimental results across south China sea(SCS) and Kuroshio(KUR) regions from 1993 to 2020 demonstrate that CTP achieves state-of-the-art(SOTA) performance in both single-step and multi-step predictions, significantly outperforming baseline models in accuracy, $F_1$ score, and temporal stability.

cs.LG↗

An ocean front detection and tracking algorithm

Existing ocean front detection methods--including histogram-based variance analysis, Lyapunov exponent, gradient thresholding, and machine learning--suffer from critical limitations: discontinuous outputs, over-detection, reliance on single-threshold decisions, and lack of open-source implementations. To address these challenges, this paper proposes the Bayesian Front Detection and Tracking framework with Metric Space Analysis (BFDT-MSA). The framework introduces three innovations: (1) a Bayesian decision mechanism that integrates gradient priors and field operators to eliminate manual threshold sensitivity; (2) morphological refinement algorithms for merging fragmented fronts, deleting spurious rings, and thinning frontal zones to pixel-level accuracy; and (3) a novel metric space definition for temporal front tracking, enabling systematic analysis of front evolution. Validated on global SST data (2022--2024), BFDT-MSA reduces over-detection by $73\%$ compared to histogram-based methods while achieving superior intensity ($0.16^\circ$C/km), continuity, and spatiotemporal coherence. The open-source release bridges a critical gap in reproducible oceanographic research.

cs.CV↗

Talk to Parallel LiDARs: A Human-LiDAR Interaction Method Based on 3D Visual Grounding

LiDAR sensors play a crucial role in various applications, especially in autonomous driving. Current research primarily focuses on optimizing perceptual models with point cloud data as input, while the exploration of deeper cognitive intelligence remains relatively limited. To address this challenge, parallel LiDARs have emerged as a novel theoretical framework for the next-generation intelligent LiDAR systems, which tightly integrate physical, digital, and social systems. To endow LiDAR systems with cognitive capabilities, we introduce the 3D visual grounding task into parallel LiDARs and present a novel human-computer interaction paradigm for LiDAR systems. We propose Talk2LiDAR, a large-scale benchmark dataset tailored for 3D visual grounding in autonomous driving. Additionally, we present a two-stage baseline approach and an efficient one-stage method named BEVGrounding, which significantly improves grounding accuracy by fusing coarse-grained sentence and fine-grained word embeddings with visual features. Our experiments on Talk2Car-3D and Talk2LiDAR datasets demonstrate the superior performance of BEVGrounding, laying a foundation for further research in this domain.

cs.CV↗