arXiv Science⌕ Search

arXiv subjects

Ni Yang

Publications and source records attributed to Ni Yang.

13 recordsLinked to original sources

GameRealm: Grounding Visual Learning in Interactive Games

Visual foundation models are central to physical intelligence, yet their development remains constrained by fragmented training datasets, inconsistent annotation quality, and limited multimodal supervision. To address these challenges, we introduce GameRealm, a large-scale, richly annotated dataset that grounds visual observations in synchronized, game-native supervision. GameRealm encompasses over 120 million RGB frames and 1,000 hours of gameplay footage across all views. Beyond its unprecedented scale, GameRealm provides rich multimodal supervision, including metric depth, camera parameters, synchronized audio, control inputs, and 3D point trajectories. Its diversity further extends to flexible view configurations, spanning single-view, stereo, multi-view, and panoramic recordings. To enable scalable acquisition across diverse game environments, we develop a unified data collection pipeline that combines human-operated interaction with agentic exploration. Extensive experiments across diverse visual learning tasks demonstrate that GameRealm consistently strengthens visual foundation models in both visual generation and geometric learning. These results establish GameRealm as a versatile and valuable data resource, unlocking new opportunities for advancing visual intelligence.

cs.CV↗

$π^4$: A Foundation Model for 4D Scene Geometry and Motion

$π^4$ is a unified foundation model for 4D scene perception. From diverse, uncalibrated RGB observations in the wild---a single image, an unordered collection, or a video sequence---it recovers dense geometry, camera poses, and metric scale, together with 2D/3D point tracks and moving-object masks for multi-image inputs. It couples a geometry branch with a geometry-conditioned motion branch, which predicts image-space correspondences that are lifted into 3D using reconstructed target-view geometry. Across diverse benchmarks, $π^4$ achieves the best overall average rank in geometry evaluation, alongside competitive performance in single- and multiview 2D/3D point tracking and moving-object segmentation. With a shared backbone, the full geometry-and-motion system contains approximately 0.5B parameters---less than half the size of most baselines. It produces joint reconstructions and tracks in one forward pass, including large-scale scenes with thousands of input images despite training with contexts of at most 64 views. We further present the data and training recipe underlying these capabilities, together with empirical studies of key design choices. Code and models are available at https://github.com/yyfz/Pi4.

cs.CV↗

InternW0-$Δ$: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate priors from large-scale pretrained models---including visual dynamics, scene semantics, geometry, and motion---into a unified framework for robot action generation. We introduce InternW0-$Δ$, a unified WAM pretrained on a heterogeneous corpus that outperforms prior methods across simulation benchmarks and real-robot platforms. InternW0-$Δ$ combines pretrained visual dynamics, scene-level semantics, 4D geometric and motion priors, and action generation within a Mixture-of-Transformers (MoT) framework. A pretrained video expert and an action expert interact under semantic guidance from a frozen VLM, while a pretrained 4D foundation model injects geometric and motion priors through training-only distillation. We further introduce Causal Imprint, which learns future-relevant scene changes from training-only future supervision and provides predictive representations directly to the action expert without future-video rollout at inference. For large-scale joint training, we construct a heterogeneous corpus of robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data, curated and aligned under a common state-action representation. The resulting corpus contains over 20K hours of processed training data, to our knowledge the largest open-source corpus of its kind. We pretrain InternW0-$Δ$ on this corpus and demonstrate strong performance across simulation benchmarks and real-robot platforms. We will open source the training code, model weights, infrastructure, data-processing pipeline, and processed data where licenses permit. Project page: https://internrobotics.github.io/InternW0-Delta/

cs.RO↗

Cross-lingual brain-language model alignment is robust but challenges hierarchical and computational accounts

Brain-language model alignment is often interpreted as evidence that transformer models implement computations similar to those of the human brain. This assumes that neural predictivity reflects internal computational properties of large language models (LLMs), such as hierarchical contextual processing, predictive coding, or representational compression. An alternative possibility is that brain scores primarily reflect stable lexical-semantic correspondences shared by language models and the brain. Here we tested these interpretations using whole-brain encoding models across Mandarin, English, and French. Across all three languages, transformer representations significantly predicted activity in a distributed network spanning classical language regions, transmodal cortical systems, and subcortical structures. These spatial patterns showed substantial cross-linguistic overlap and remained remarkably stable across layers, providing little evidence that model depth systematically maps onto cortical processing hierarchies. Likewise, contextual transformer embeddings did not consistently outperform static lexical embeddings, despite providing some unique predictive variance. Finally, neither surprisal nor intrinsic dimensionality reproduced the layer-wise profile of brain scores, arguing against prediction and information compression as primary explanations for brain-LLM alignment. Together, these findings suggest that brain-LLM alignment is more robust across languages, transformer depth, and model architectures than previously appreciated, but less informative about shared computational mechanisms. Our results are more consistent with neural predictivity reflecting stable representational structure preserved across model transformations than with a one-to-one correspondence between their underlying computations.

cs.CL↗

The grip of grammar on meaning uncertainty: cross-linguistic evidence, neural correlates, and clinical relevance

Isolated word meanings are inherently uncertain. This uncertainty reduces when they are combined and anchored in context. We propose that grammar compresses meaning uncertainty cross-linguistically, which is reflected in brain and selectively disrupted in disorders. Compression was operationalized as the relative difference between non-contextual surprisal estimated from lexical frequency, and contextual surprisal from grammar-sensitive models. In narratives from 20 languages, contextual surprisal reduced frequency-based surprisal. This reduction closely tracked the surprisal cost of reversing word order, and scaled with richer, non-redundant lexis as organized by more complex but optimal dependency structure. During fMRI, surprisal and its reduction explained BOLD activity for comprehension and production in overlapping but distinct regions. Uncertainty reduction was significantly attenuated in aphasia, dementia, and schizophrenia, but remained intact where primary deficit is not language. These findings position uncertainty reduction via grammar as a foundational concept that illuminates principles, brain basis, and disruptions of language.

cs.CL↗

Coherence in the brain unfolds across separable temporal regimes

To maintain coherence in language, the brain must satisfy key competing temporal demands: the gradual accumulation of meaning across extended context (drift) and the rapid reconfiguration of representations at event boundaries (shift). How these processes are implemented in the human brain during naturalistic listening remains unclear. Here, we tested whether both can be captured by annotation-free drift and shift signals and whether their neural expression shows distinct regional preferences across the brain. These signals were derived from a large language model (LLM) processing the narrative input. To enable high-precision voxelwise encoding models with stable parameter estimates, we densely sampled one healthy adult across more than 7 hours of listening to crime stories while collecting 7 Tesla fMRI data. We then modeled the feature-informed hemodynamic response using a regularized encoding framework validated on independent stories. Drift predictions were prevalent in default-mode network hubs, whereas shift predictions were evident bilaterally in the primary auditory cortex and language association cortex. Together, these findings show that coherence during language comprehension is implemented through distinct but co-expressed neural regimes of slow contextual integration and rapid event-driven reconfiguration, offering a mechanistic entry point for understanding disturbances of language coherence in psychiatric disorders.

q-bio.NC↗

When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning

Recent progress in multimodal large language models has led to strong performance on reasoning tasks, but these improvements largely rely on high-quality annotated data or teacher-model distillation, both of which are costly and difficult to scale. To address this, we propose an unsupervised self-evolution training framework for multimodal reasoning that achieves stable performance improvements without using human-annotated answers or external reward models. For each input, we sample multiple reasoning trajectories and jointly model their within group structure. We use the Actor's self-consistency signal as a training prior, and introduce a bounded Judge based modulation to continuously reweight trajectories of different quality. We further model the modulated scores as a group level distribution and convert absolute scores into relative advantages within each group, enabling more robust policy updates. Trained with Group Relative Policy Optimization (GRPO) on unlabeled data, our method consistently improves reasoning performance and generalization on five mathematical reasoning benchmarks, offering a scalable path toward self-evolving multimodal models. The code are available at https://github.com/OPPO-Mente-Lab/LLM-Self-Judge.

cs.CV↗

Modelling instrumental response for neutron scattering experiments at CSNS

Thermal neutron total scattering experiments of light and heavy water were reproduced using the CSNS in-house Monte Carlo thermal neutron transport code, Prompt, with a focus on the instrumental detector response and the accurate derivation of thermal neutron scattering cross-sections. In this work, a data reduction method is developed to process both the measured and simulated detector events for estimating angular, wavelength distributions, as well as angular differential cross sections. The reduction results of simulations and experiments show a high degree of consistency. The prominent inelasticity signatures observed in the experiments can be accurately reproduced in simulations. We discuss the cause of the inelasticity effects, and demonstrate the elimination of such effects when the inelastic scattering process is taken into account in simulations. In addition, multiple scattering in samples is analysed and discussed.

physics.ins-det↗

BzScope: an absolute cross section calculator for neutron-phonon scattering

BzScope is a Python package designed for efficiently calculating absolute cross sections of neutron-phonon inelastic scattering for crystalline powders in large phase spaces, addressing the limitations of traditional histogramming techniques in reproducing sharp structures and ensuring convergence. The package employs an adapted integral method and supports calculations of single- and two-phonon scattering functions in ideal crystalline powders, with numerical robustness up to a momentum transfer of 100 Ang^-1. Higher order scatterings up to several hundred orders are calculated by incoherent approximation in a well-established thermal neutron scattering physics package, NCrystal. In addition, a NCrystal plugin is made available for NCrystal-enabled Monte Carlo packages, facilitating direct comparison between the new physics and experimental data. Validation against NCrystal demonstrates good agreement in incoherent scattering for cubic systems Ni. In addition, it shows improved accuracy for low-symmetry materials $NiP_2$ by avoiding the isotropic atomic displacement approximations in NCrystal. Benchmarks the experimental differential cross section of LiH and total cross section of Be confirm its reliability. BzScope integrates with NCrystal via a plugin and therefore can be directly used in any NCrystal-enabled Monte Carlo package. This tool enhances the efficiency and accuracy of neutron scattering simulations, advancing the study of condensed matter dynamics.

physics.comp-ph↗

DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents

Mobile Phone Agents (MPAs) have emerged as a promising research direction due to their broad applicability across diverse scenarios. While Multimodal Large Language Models (MLLMs) serve as the foundation for MPAs, their effectiveness in handling multiple mobile phone tasks simultaneously remains limited. Although multitask supervised fine-tuning (SFT) is widely adopted for multitask learning, existing approaches struggle to determine optimal training data compositions for peak performance. To address this challenge, we propose DaMo (Data Mixture Optimizer) - a novel solution employing a trainable network that predicts optimal data mixtures by forecasting downstream task performance for any given dataset ratio. To support comprehensive evaluation, we introduce PhoneAgentBench, the first specialized benchmark to evaluate MLLMs on multimodal mobile phone tasks, comprising 1235 QA pairs spanning diverse real-world industrial mobile application scenarios. Demonstrating strong predictive capability (R^2=0.81) in small-scale pilot experiments, DaMo efficiently extrapolates optimal data mixing configurations. Our results show DaMo achieves a 3.38% performance improvement on PhoneAgentBench compared to alternative methods. Furthermore, extensive experiments across established benchmarks including BFCL-v3, MME-Reasoning, MME-Perception, and OCRBench reveal DaMo's superior generalization, outperforming other approaches by 2.57% in terms of average score. When used solely for MLLM optimization on the BFCL-v3 task, DaMo improves the metrics by 12.47% than other methods. Notably, DaMo maintains robust scalability, preserving its effectiveness when applied to other model architectures. The code and dataset are available at https://github.com/OPPO-Mente-Lab/DaMo.git

cs.CV↗

ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models

The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coherence in multi-turn dialogues. Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. To address this, we propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. The framework iteratively refines instructions by incorporating Reviewer feedback, enhancing dialogue diversity and difficulty. We construct a multi-turn dataset using the Alpaca dataset and fine-tune the LLaMA2-13B model. Evaluations on MT-Bench, MMLU-Pro, and Auto-Arena demonstrate significant improvements, achieving absolute gains of 2.9\% on MMLU-Pro and 2\% on MT-Bench compared to prior state-of-the-art models based on LLaMA2-13B. Ablation studies confirm the critical role of the Review stage and the use of multiple Reviewers in boosting instruction diversity and difficulty. Our work highlights the potential of review-driven, multi-agent frameworks for generating high-quality conversational data at scale.

cs.CL↗

The number of dissociation sets in connected graphs

Extremal problems related to the enumeration of graph substructures, such as independent sets, matchings, and induced matchings, have become a prominent area of research with the advancement of graph theory. A subset of vertices is called a dissociation set if it induces a subgraph with vertex degree at most $1$, making it a natural generalization of these previously studied substructures. In this paper, we present efficient tools to strictly increase the number of dissociation sets in a connected graph. Furthermore, we establish that the maximum number of dissociation sets among all connected graphs of order $n$ is given by \begin{align*} \begin{cases} 2^{n-1}+(n+3)\cdot 2^{\frac{n-5}{2}}, &~ {\rm if}~ n~{\rm is}~{\rm odd};\\ 2^{n-1}+(n+6)\cdot 2^{\frac{n-6}{2}}, &~ {\rm if}~ n~{\rm is}~{\rm even}. \end{cases} \end{align*} Additionally, we determine the achievable upper bound on the number of dissociation sets in a tree of order $n$ and characterize the corresponding extremal graphs as an intermediate result. Finally, we identify the unicyclic graph that is the candidate for having the second largest number of dissociation sets among all connected graphs.

math.CO↗

Prompt: Probability-Conserved Cross Section Biasing Monte Carlo Particle Transport System

An open source software package for simulating thermal neutron propagation in geometry is presented. In this system, neutron propagation can be treated by either the particle transport method or the ray-tracing method. Supported by an accurate backend scattering physics engine, this system is capable of reproducing neutron scattering experiments in complex geometries and is expected to be used in the areas of instrument characterisation, optimisation and data analysis. In this paper, the relevant theories are briefly introduced. The simulation flow and the user input syntax to control it are provided in detail. Five benchmarking simulations, focusing on different aspects of simulation and scattering techniques, are given to demonstrate the applications of this simulation system. They include an idealised total scattering instrument, a monochromatic powder diffractometer, a neutron guide, a chopper and an imaging setup for complex geometries. Simulated results are benchmarked against experimental data or well-established software packages when appropriate. Good agreements are observed.

physics.comp-ph↗