arXiv subjects

Yang Chen

Publications and source records attributed to Yang Chen.

At least 199 records · Page 11Linked to original sources

arXiv2025

On the Limitations and Capabilities of Position Embeddings for Length Generalization

In Transformers, Position Embeddings (PEs) significantly influence Length Generalization (LG) performance, yet their fundamental role remains unclear. In this work, we investigate the limitations and capabilities of PEs in achieving LG. We theoretically analyze PEs in Position-Only Linear Attentions (POLAs), introducing Linear Representation Complexity (LRC) to characterize when PEs enable LG. Our analysis shows that PEs do not expand computational capabilities but structure learned computations across positions. Extending to practical Transformers, we propose Sequential Representation Complexity (SRC) and conjecture that LG is possible if and only if SRC remains invariant across scales. We support this hypothesis with empirical evidence in various reasoning tasks. To enhance LG, we introduce Scale Hint, allowing flexible instance scaling, and a Learning-Based Position Embedding framework that automatically learns positional relations. Our work provides theoretical insights and practical strategies for improving LG in Transformers.

cs.LG↗

arXiv2025

The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback

Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a task is beyond its capability boundary and proactively requests external data to break through that limit. Extensive experiments on nine benchmarks showing a 53.8% relative improvement with less than 1.2% extra data demonstrate the efficacy of self-aware RL and underscore the promise of self-evolving agent training.

cs.CL↗

arXiv2025

Making high-order asymptotics practical: correcting goodness-of-fit test for astronomical count data

The C statistic is a widely used likelihood-ratio statistic for model fitting and goodness-of-fit assessments with Poisson data in high-energy physics and astrophysics. Although it enjoys convenient asymptotic properties, the statistic is routinely applied in cases where its nominal null distribution relies on unwarranted assumptions. Because researchers do not typically carry out robustness checks, their scientific findings are left vulnerable to misleading significance calculations. With an emphasis on low-count scenarios, we present a comprehensive study of the theoretical properties of C statistics and related goodness-of-fit algorithms. We focus on common ``plug-in'' algorithms where moments of C are obtained by assuming the true parameter equals its estimate. To correct such methods, we provide a suite of new principled user-friendly algorithms and well-calibrated p-values that are ready for immediate deployment in the (astro)physics data-analysis pipeline. Using both theoretical and numerical results, we show (a) standard $χ^2$-based goodness-of-fit assessments are invalid in low-count settings, (b) naive methods (e.g., vanilla bootstrap) result in biased null distributions, and (c) the corrected Z-test based on conditioning and high-order asymptotics gives the best precision with low computational cost. We illustrate our methods via a suite of simulations and applied astrophysical analyses. An open-source Python package is provided in a GitHub repository.

stat.ME↗

arXiv2025

Alternating Training-based Label Smoothing Enhances Prompt Generalization

Recent advances in pre-trained vision-language models have demonstrated remarkable zero-shot generalization capabilities. To further enhance these models' adaptability to various downstream tasks, prompt tuning has emerged as a parameter-efficient fine-tuning method. However, despite its efficiency, the generalization ability of prompt remains limited. In contrast, label smoothing (LS) has been widely recognized as an effective regularization technique that prevents models from becoming over-confident and improves their generalization. This inspires us to explore the integration of LS with prompt tuning. However, we have observed that the vanilla LS even weakens the generalization ability of prompt tuning. To address this issue, we propose the Alternating Training-based Label Smoothing (ATLaS) method, which alternately trains with standard one-hot labels and soft labels generated by LS to supervise the prompt tuning. Moreover, we introduce two types of efficient offline soft labels, including Class-wise Soft Labels (CSL) and Instance-wise Soft Labels (ISL), to provide inter-class or instance-class relationships for prompt tuning. The theoretical properties of the proposed ATLaS method are analyzed. Extensive experiments demonstrate that the proposed ATLaS method, combined with CSL and ISL, consistently enhances the generalization performance of prompt tuning. Moreover, the proposed ATLaS method exhibits high compatibility with prevalent prompt tuning methods, enabling seamless integration into existing methods.

cs.CV↗

arXiv2025

Saturation of the kinetic ballooning instability due to the electron parallel nonlinearity

The electron parallel nonlinearity (EPN) is implemented in the gyrokinetic Particle-in-Cell turbulence code GEM [Y. Chen and S. E. Parker, J. Comp. Phys. 220, 839 (2007)]. Application to the Cyclone Base Case reveals a strong effect of EPN on the saturated heat transport above the kinetic ballooning mode threshold. Evidence is provided to show that the strong effect is associated with the electron radial motion due to magnetic fluttering, which turns fine structures of the KBM eigenmode in radius into fine structures in velocity and increases the magnitude of the EPN term in the kinetic equation.

physics.plasm-ph↗

arXiv2025

Matrix Autoregressive Model with Vector Time Series Covariates for Spatio-Temporal Data

We develop a new methodology for forecasting matrix-valued time series with historical matrix data and auxiliary vector time series data. We focus on a time series of matrices defined on a static 2-D spatial grid and an auxiliary time series of non-spatial vectors. The proposed model, Matrix AutoRegression with Auxiliary Covariates (MARAC), contains an autoregressive component for the historical matrix predictors and an additive component that maps the auxiliary vector predictors to a matrix response via tensor-vector product. The autoregressive component adopts a bi-linear transformation framework following Chen et al. (2021), significantly reducing the number of parameters. The auxiliary component posits that the tensor coefficient, which maps non-spatial predictors to a spatial response, contains slices of spatially smooth matrix coefficients that are discrete evaluations of smooth functions on a spatial grid from a Reproducing Kernel Hilbert Space (RKHS). We propose to estimate the model parameters under a penalized maximum likelihood estimation framework coupled with an alternating minimization algorithm. We establish the joint asymptotics of the autoregressive and tensor parameters under fixed and high-dimensional regimes. Extensive simulations and a geophysical application for forecasting the global Total Electron Content (TEC) are conducted to validate the performance of MARAC.

stat.ME↗

arXiv2025

Absence of Radio Emission Reveals an Exceptionally Weak Explosion of the Putative Historical Supernova Pa 30

We present the first deep radio continuum observations of Pa 30, a nebula hosting a unique optical source driven by an ultrafast outflow with a velocity of 16,000 km s$^{-1}$. The nebula was proposed to be the remnant of a white dwarf merger that occurred in 1181CE. We report no detection of the radio diffuse emission from Pa 30 or radio emission from the central source, setting $3σ$ upper limit flux densities of $0.84\,\rm mJy$ and $0.29\,\rm mJy$ at 1.5 GHz and 6 GHz, respectively, for Pa 30. The radio surface brightness of Pa 30 is $\sim 3$ orders of magnitude smaller than that of typical supernova remnants (SNRs) with comparable angular size. If Pa 30 is an SNR, our observations show it to be the faintest known in the radio band. Considering that 10\% of the supernova (SN) kinetic energy is transferred to cosmic rays (CRs), the absence of radio synchrotron emission suggests that the SN kinetic energy $\lesssim3\times 10^{47}(B/10 μ\rm G)^{-1.65}$ erg, which is 3 to 4 orders of magnitude lower than that of typical SNRs and the lowest measured among Galactic SNRs. There is also an indication of inefficient CR acceleration for this source. The low SN kinetic energy either implies the potential existence of many more radio-faint, sub-energetic SNRs in our Galaxy or challenges the SNR interpretation of Pa 30.

astro-ph.HE↗

arXiv2025

Chain-of-Thought Re-ranking for Image Retrieval Tasks

Image retrieval remains a fundamental yet challenging problem in computer vision. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated strong reasoning capabilities, existing methods typically employ them only for evaluation, without involving them directly in the ranking process. As a result, their rich multimodal reasoning abilities remain underutilized, leading to suboptimal performance. In this paper, we propose a novel Chain-of-Thought Re-Ranking (CoTRR) method to address this issue. Specifically, we design a listwise ranking prompt that enables MLLM to directly participate in re-ranking candidate images. This ranking process is grounded in an image evaluation prompt, which assesses how well each candidate aligns with users query. By allowing MLLM to perform listwise reasoning, our method supports global comparison, consistent reasoning, and interpretable decision-making - all of which are essential for accurate image retrieval. To enable structured and fine-grained analysis, we further introduce a query deconstruction prompt, which breaks down the original query into multiple semantic components. Extensive experiments on five datasets demonstrate the effectiveness of our CoTRR method, which achieves state-of-the-art performance across three image retrieval tasks, including text-to-image retrieval (TIR), composed image retrieval (CIR) and chat-based image retrieval (Chat-IR). Our code is available at https://github.com/freshfish15/CoTRR .

cs.CV↗

arXiv2025

Fermi-LAT Detected Gamma-ray Emission Likely Associated with SNR Kes 78

We have analyzed the GeV gamma-ray emission in the region of the supernova remnant (SNR) Kes 78 using $\sim$16.7 years of Fermi-LAT observations and found that the catalog sources 4FGL J1852.4+0037e and 4FGL J1851.8$-$0007c are better represented as two extended sources modeled as `2Ext'. One of them, designated as E2, is located at R.A.$=282.86^\circ$, Dec.$=-0.11^\circ$ with the 68\% containment radius $R_{68} = 0.31^\circ$, and is detected with a significance of 15.2$σ$ in the 0.2--500 GeV energy range. The gamma-ray emission of source E2 is well described by a log-parabola (LogP) spectral model with spectral index $Γ$ = 1.2 and curvature $β$ = 0.3. The fitting with electron-proton number ratio $K_{\rm ep}=0.01$ indicates that the GeV emission of source E2 is dominated by hadronic emission. Given the dense molecular environment surrounding the middle-aged SNR Kes 78, the hadronic scenario provides a natural explanation for the observed GeV emission. The extended source E2 can also be replaced with two point sources. One of them, designated as PTS1, is coincident with the newly discovered PSR J1852$-$0002g within the 68\% positional uncertainty circle, indicating a possible gamma-ray contribution from this PSR. The gamma-ray spectrum of source PTS1 can be well described by a LogP spectral shape. The synchro-curvature radiation model provides a satisfactory spectral fit for source PTS1, suggesting that some of the GeV emission from the Kes 78 region might possibly originate from the magnetosphere of PSR J1852$-$0002g.

astro-ph.HE↗

arXiv2025

Kalman Filtering of Stationary Graph Signals

In this paper, we propose a novel definition of stationary graph signals, formulated with respect to a symmetric graph shift, such as the graph Laplacian. We show that stationary graph signals can be generated by transmitting white noise through polynomial graph channels, and that their stationarity is preserved under polynomial channel transmission. In this paper, we also investigate Kalman filtering to dynamical systems characterized by polynomial state and observation matrices. We demonstrate that Kalman filtering maintains the stationarity of graph signals, while effectively incorporating both system dynamics and noise structure. In comparison to the static inverse filtering method and naive zero-signal strategy, the Kalman filtering procedure yields more accurate and adaptive signal estimates, highlighting its robustness and versatility in graph signal processing.

eess.SP↗

arXiv2025

The Impact of Automation on Risk-Taking: The Role of Sense of Agency

Automation significantly alters human behavior, particularly risk-taking. Previous researches have paid limited attention to the underlying characteristics of automation and their mechanisms of influence on risk-taking. This study investigated how automation affects risk-taking and examined the role of sense of agency therein. By quantifying sense of agency through subjective ratings, this research explored the impact of automation level and reliability level on risk-taking. The results of three experiments indicated that automation reduced the level of risk-taking; higher automation level was associated with lower sense of agency and lower risk-taking, with sense of agency playing a complete mediating role; higher automation reliability was associated with higher sense of agency and higher risk-taking, with sense of agency playing a partial mediating role. The study concludes that automation influences risk-taking, such that higher automation level or lower reliability is associated with a lower likelihood of risk-taking. Sense of agency mediates the impact of automation on risk-taking, and automation level and reliability have different effects on risk-taking.

cs.HC↗

arXiv2025

MoPD: Mixture-of-Prompts Distillation for Vision-Language Models

Soft prompt learning methods are effective for adapting vision-language models (VLMs) to downstream tasks. Nevertheless, empirical evidence reveals a tendency of existing methods that they overfit seen classes and exhibit degraded performance on unseen classes. This limitation is due to the inherent bias in the training data towards the seen classes. To address this issue, we propose a novel soft prompt learning method, named Mixture-of-Prompts Distillation (MoPD), which can effectively transfer useful knowledge from hard prompts manually hand-crafted (a.k.a. teacher prompts) to the learnable soft prompt (a.k.a. student prompt), thereby enhancing the generalization ability of soft prompts on unseen classes. Moreover, the proposed MoPD method utilizes a gating network that learns to select hard prompts used for prompt distillation. Extensive experiments demonstrate that the proposed MoPD method outperforms state-of-the-art baselines especially on on unseen classes.

cs.CV↗

arXiv2025

GeV Gamma-Rays from Molecular Clouds Illuminated by Particles Diffusing from the Adjacent Supernova Remnant G335.2+0.1 Confined in an Expanding Bubble

We report the detection of GeV gamma-ray emission likely associated with supernova remnant (SNR) G335.2+0.1 and the finding of a molecular cloud ($\sim20$--$30^\prime$ in angular size) that is very likely in physical contact with the SNR and responsible for the gamma-ray emission. Using the 16.8 yr Fermi-LAT data, an extended emission, with a significance of 13.5 $σ$ and a radius 0.24° in 0.2--500 GeV in the uniform-disk model, was found to the adjacent east of the SNR. With archival Mopra CO-line data, a large molecular clump at local-standard-of-rest velocity $\sim-48$ to $-43$ km s$^{-1}$ was revealed appearing coincident with the gamma-ray source. The SNR was found located in a cavity encircled by a 'C'-shaped ring-like molecular shell at $-45$ to $-43$ km s$^{-1}$. This morphological agreement, together with the position-velociy diagrams made along lines cutting across the cavity, suggests that the SNR was evolving in the expanding molecular bubble created by the stellar wind of the progenitor with a mass $\gtrsim 15 M_{\mathrm{sun}}$. The giant molecular cloud, visible at around $-46$ km s$^{-1}$, and the associated SNR are thus estimated to lie at a kinematic distance of 3.1 kpc, with the HI absorption taken into account. We suggest that the SNR has entered the radiative phase after the blastwave recently struck the cavity wall. With the evolutionary scenario of the SNR, we demonstrate that the gamma-ray emission reported here can be naturally interpreted by the hadronic interaction between the accelerated protons that escaped from the SNR shock and the eastern large molecular clump.

astro-ph.HE↗

arXiv2025

Challenging Bug Prediction and Repair Models with Synthetic Bugs

Bugs are essential in software engineering; many research studies in the past decades have been proposed to detect, localize, and repair bugs in software systems. Effectiveness evaluation of such techniques requires complex bugs, i.e., those that are hard to detect through testing and hard to repair through debugging. From the classic software engineering point of view, a hard-to-repair bug differs from the correct code in multiple locations, making it hard to localize and repair. Hard-to-detect bugs, on the other hand, manifest themselves under specific test inputs and reachability conditions. These two objectives, i.e., generating hard-to-detect and hard-to-repair bugs, are mostly aligned; a bug generation technique can change multiple statements to be covered only under a specific set of inputs. However, these two objectives are conflicting for learning-based techniques: A bug should have a similar code representation to the correct code in the training data to challenge a bug prediction model to distinguish them. The hard-to-repair bug definition remains the same but with a caveat: the more a bug differs from the original code, the more distant their representations are and easier to be detected. We propose BugFarm, to transform arbitrary code into multiple complex bugs. BugFarm leverages LLMs to mutate code in multiple locations (hard-to-repair). To ensure that multiple modifications do not notably change the code representation, BugFarm analyzes the attention of the underlying model and instructs LLMs to only change the least attended locations (hard-to-detect). Our comprehensive evaluation of 435k+ bugs from over 1.9M mutants generated by BUGFARM and two alternative approaches demonstrates our superiority in generating bugs that are hard to detect by learning-based bug prediction approaches and hard-to-repair by state-of-the-art learning-based program repair technique.

cs.SE↗

arXiv2025

Conformalized Tensor Completion with Riemannian Optimization

Tensor data, or multi-dimensional arrays, is a data format popular in multiple fields such as social network analysis, recommender systems, and brain imaging. It is not uncommon to observe tensor data containing missing values, and tensor completion aims at estimating the missing values given the partially observed tensor. Sufficient efforts have been spared on devising scalable tensor completion algorithms, but few on quantifying the uncertainty of the estimator. In this paper, we nest the uncertainty quantification (UQ) of tensor completion under a split conformal prediction framework and establish the connection of the UQ problem to a problem of estimating the missing propensity of each tensor entry. We model the data missingness of the tensor with a tensor Ising model parameterized by a low-rank tensor parameter. We propose to estimate the tensor parameter by maximum pseudo-likelihood estimation (MPLE) with a Riemannian gradient descent algorithm. Extensive simulation studies have been conducted to justify the validity of the resulting conformal interval. We apply our method to the regional total electron content (TEC) reconstruction problem. Supplemental materials of the paper are available online.

stat.ME↗

arXiv2025

Mirages and Large TeV Halo-Pulsar Offsets from Cosmic Ray Propagation

The study of extended $γ$-ray sources usually assumes symmetric diffusion of cosmic rays. However, recent observations of multiple sources near single pulsars and significant offsets between TeV halo centroids and their parent pulsars suggest that this assumption is overly simplistic. In this Letter, we demonstrate that asymmetric propagation of cosmic rays near their accelerators may create multiple TeV sources instead of a single symmetric source. This mechanism also explains the large offsets between TeV halo centroids and their pulsars. We demonstrate that several perplexing detected sources can be naturally explained without invoking additional invisible accelerators.

astro-ph.HE↗

arXiv2025

Semantic-guided LoRA Parameters Generation

Low-Rank Adaptation (LoRA) has demonstrated strong generalization capabilities across a variety of tasks for efficiently fine-tuning AI models, especially on resource-constrained edges. However, in real-world applications, edge users often exhibit task-specific preferences that are difficult to handle with a unified model trained under a closed-world assumption, and the challenge may further increase when there are significant domain shifts between training and deployment. Meanwhile, retraining/fine-tuning models for each user is also impractical due to its cost-intensive nature and privacy concerns over raw data utilization from edges. To address these challenges, we propose Semantic-guided LoRA Parameter Generation (SG-LoRA), the first of its kind framework to efficiently produce user-specific LoRA parameters without any additional training on user tasks or access to user-specific data. Concretely, SG-LoRA uses task descriptions as the semantic bridge, measuring their proximity to a set of known expert tasks in a shared embedding space. Based on this semantic guidance, it models the target task's LoRA parameter distribution to generate high-performing parameters for novel tasks. SG-LoRA enables the real-time construction of LoRA models aligned with individual intents by distilling knowledge from prominent LoRA experts and, meanwhile, offering a privacy-preserving solution for personalized model adaptation in a novel zero-shot open-world setting proposed in this work. Extensive experiments on multiple challenging tasks confirm the superior performance and remarkable adaptability of SG-LoRA. Code is available at https://github.com/keepgoingjkg/SG-LoRA.

cs.LG↗

arXiv2025

Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables

Designing suitable reward functions for numerous interacting intelligent agents is challenging in real-world applications. Inverse reinforcement learning (IRL) in mean field games (MFGs) offers a practical framework to infer reward functions from expert demonstrations. While promising, the assumption of agent homogeneity limits the capability of existing methods to handle demonstrations with heterogeneous and unknown objectives, which are common in practice. To this end, we propose a deep latent variable MFG model and an associated IRL method. Critically, our method can infer rewards from different yet structurally similar tasks without prior knowledge about underlying contexts or modifying the MFG model itself. Our experiments, conducted on simulated scenarios and a real-world spatial taxi-ride pricing problem, demonstrate the superiority of our approach over state-of-the-art IRL methods in MFGs.

cs.LG↗