arXiv subjects

Runyu Wang

Publications and source records attributed to Runyu Wang.

6 recordsLinked to original sources

arXiv2026

RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons

Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.

cs.AI↗

arXiv2025

Putting a Price on Immobility: Food Deliveries and Pricing Approaches

Urban food delivery services have become an integral part of daily life, yet their mobility and environmental externalities remain poorly addressed by planners. Most studies neglect whether consumers pay enough to internalize the broader social costs of these services. This study quantifies the value of access to and use of food delivery services in Beijing, China, through two discrete choice experiments. The first measures willingness to accept compensation for giving up access, with a median value of CNY588 (approximately USD80). The second captures willingness to pay for reduced waiting time and improved reliability, showing valuations far exceeding typical delivery fees (e.g., CNY96.6/hour and CNY4.83/min at work). These results suggest a substantial consumer surplus and a clear underpricing problem. These findings highlight the need for urban planning to integrate digital service economies into pricing and mobility frameworks. We propose a quantity-based pricing model that targets delivery speed rather than order volume, addressing the primary source of externalities while maintaining net welfare gains. This approach offers a pragmatic, equity-conscious strategy to curb delivery-related congestion, emissions, and safety risks, especially in dense urban cores.

econ.GN↗

arXiv2025

LoKI: Low-damage Knowledge Implanting of Large Language Models

Fine-tuning adapts pretrained models for specific tasks but poses the risk of catastrophic forgetting (CF), where critical knowledge from pretraining is overwritten. To address the issue of CF in a general-purpose framework, we propose Low-damage Knowledge Implanting (LoKI), a parameter-efficient fine-tuning (PEFT) technique that utilizes recent mechanistic understanding of how knowledge is stored in transformer architectures. We compare LoKI against state-of-the-art PEFT methods in two real-world fine-tuning scenarios. The results show that LoKI demonstrates significantly better preservation of general capabilities. At the same time, its task-specific performance is comparable to or even surpasses that of full parameter fine-tuning and these PEFT methods across various model architectures. Our work bridges the mechanistic insights of LLMs' knowledge storage with practical fine-tuning objectives, enabling an effective balance between task-specific adaptation and the retention of general-purpose capabilities.

cs.CL↗

arXiv2024

Role Identification based Method for Cyberbullying Analysis in Social Edge Computing

Over the past few years, many efforts have been dedicated to studying cyberbullying in social edge computing devices, and most of them focus on three roles: victims, perpetrators, and bystanders. If we want to obtain a deep insight into the formation, evolution, and intervention of cyberbullying in devices at the edge of the Internet, it is necessary to explore more fine-grained roles. This paper presents a multi-level method for role feature modeling and proposes a differential evolution-assisted K-means (DEK) method to identify diverse roles. Our work aims to provide a role identification scheme for cyberbullying scenarios for social edge computing environments to alleviate the general safety issues that cyberbullying brings. The experiments on ten real-world datasets obtained from Weibo and five public datasets show that the proposed DEK outperforms the existing approaches on the method level. After clustering, we obtained nine roles and analyzed the characteristics of each role and their evolution trends under different cyberbullying scenarios. Our work in this paper can be placed in devices at the edge of the Internet, leading to better real-time identification performance and adapting to the broad geographic location and high mobility of mobile devices.

cs.HC↗

arXiv2022

Optically-generated focused ultrasound for noninvasive brain stimulation with ultrahigh precision

High precision neuromodulation is a powerful tool to decipher neurocircuits and treat neurological diseases. Current non-invasive neuromodulation methods offer limited precision at the millimeter level. Here, we report optically-generated focused ultrasound (OFUS) for non-invasive brain stimulation with ultrahigh precision. OFUS is generated by a soft optoacoustic pad (SOAP) fabricated through embedding candle soot nanoparticles in a curved polydimethylsiloxane film. SOAP generates a transcranial ultrasound focus at 15 MHz with an ultrahigh lateral resolution of 83 um, which is two orders of magnitude smaller than that of conventional transcranial-focused ultrasound (tFUS). Here, we show effective OFUS neurostimulation in vitro with a single ultrasound cycle. We demonstrate submillimeter transcranial stimulation of the mouse motor cortex in vivo. An acoustic energy of 0.6 mJ/cm^2, four orders of magnitude less than that of tFUS, is sufficient for successful OFUS neurostimulation. OFUS offers new capabilities for neuroscience studies and disease treatments by delivering a focus with ultrahigh precision non-invasively.

q-bio.NC↗

arXiv2021

SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition

End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervision. This could result in recognition errors due to similar-phoneme confusion or phoneme reduction. To alleviate this problem, we propose a novel framework based on Supervised Contrastive Learning (SCaLa) to enhance phonemic representation learning for end-to-end ASR systems. Specifically, we extend the self-supervised Masked Contrastive Predictive Coding (MCPC) to a fully-supervised setting, where the supervision is applied in the following way. First, SCaLa masks variable-length encoder features according to phoneme boundaries given phoneme forced-alignment extracted from a pre-trained acoustic model; it then predicts the masked features via contrastive learning. The forced-alignment can provide phoneme labels to mitigate the noise introduced by positive-negative pairs in self-supervised MCPC. Experiments on reading and spontaneous speech datasets show that our proposed approach achieves 2.8 and 1.4 points Character Error Rate (CER) absolute reductions compared to the baseline, respectively.

eess.AS↗