arXiv subjects

Yun Chen

Publications and source records attributed to Yun Chen.

At least 127 records · Page 7Linked to original sources

arXiv2025

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

Efficient finetuning of large language models (LLMs) aims to adapt the LLMs with reduced computational and memory cost. Previous LoRA-based approaches initialize the low-rank matrices with Gaussian distribution and zero values while keeping the original weight matrices frozen. However, the trainable model parameters optimized in an unguided subspace might interfere with the well-learned subspace of the pretrained weight matrices. In this paper, we propose MiLoRA, a simple yet effective LLM finetuning approach that only updates the minor singular components of the weight matrix while keeping the principal singular components frozen. It is observed that the minor matrix corresponds to the noisy or long-tail information, while the principal matrix contains important knowledge. The MiLoRA initializes the low-rank matrices within a subspace that is orthogonal to the principal matrix, thus the pretrained knowledge is expected to be well preserved. During finetuning, MiLoRA makes the most use of the less-optimized subspace for learning the labeled dataset. Extensive experiments on commonsense reasoning, math reasoning, instruction following and visual instruction following benchmarks present the superior performance of our method.

cs.CL↗

arXiv2025

SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters

The widespread applications of large language models (LLMs) have brought about concerns regarding their potential misuse. Although aligned with human preference data before release, LLMs remain vulnerable to various malicious attacks. In this paper, we adopt a red-teaming strategy to enhance LLM safety and introduce SeqAR, a simple yet effective framework to design jailbreak prompts automatically. The SeqAR framework generates and optimizes multiple jailbreak characters and then applies sequential jailbreak characters in a single query to bypass the guardrails of the target LLM. Different from previous work which relies on proprietary LLMs or seed jailbreak templates crafted by human expertise, SeqAR can generate and optimize the jailbreak prompt in a cold-start scenario using open-sourced LLMs without any seed jailbreak templates. Experimental results show that SeqAR achieves attack success rates of 88% and 60% in bypassing the safety alignment of GPT-3.5-1106 and GPT-4, respectively. Furthermore, we extensively evaluate the transferability of the generated templates across different LLMs and held-out malicious requests, while also exploring defense strategies against the jailbreak attack designed by SeqAR.

cs.CR↗

arXiv2025

LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy

Despite being pretrained on multilingual corpora, large language models (LLMs) exhibit suboptimal performance on low-resource languages. Recent approaches have leveraged multilingual encoders alongside LLMs by introducing trainable parameters connecting the two models. However, these methods typically focus on the encoder's output, overlooking valuable information from other layers. We propose \aname (\mname), a framework that integrates representations from all encoder layers, coupled with the \attaname mechanism to enable layer-wise interaction between the LLM and the multilingual encoder. Extensive experiments on multilingual reasoning tasks, along with analyses of learned representations, show that our approach consistently outperforms existing baselines.

cs.CL↗

arXiv2025

"An Image of Ourselves in Our Minds": How College-educated Online Dating Users Construct Profiles for Effective Self Presentation

Online dating is frequently used by individuals looking for potential relationships and intimate connections. Central to dating apps is the creation and refinement of a dating profile, which represents the way individuals desire to present themselves to potential mates, while hiding information they do not care to share. To investigate the way frequent users of dating apps construct their online profiles and perceive the effectiveness of strategies taken in making profiles, we conducted semi-structured interviews with 20 experienced users who are Chinese college-educated young adults and uncovered the processes and rationales by which they make profiles for online dating, particularly in selecting images for inclusion. We found that participants used idealized photos that exaggerated their positive personality traits, sometimes traits that they do not possess but perceive others to desire, and sometimes even traits they wish they had possessed. Users also strategically used photos that show personality and habits without showing themselves, and often hid certain identifying information to reduce privacy risks. This analysis signals potential factors that are key in building online dating profiles, providing design implications for systems that limit the use of inaccurate information while still promoting self-expression in relationship platforms.

cs.HC↗

arXiv2025

Atom-Molecule Superradiance and Entanglement with Cavity-Mediated Three-Body Interactions

Ultracold atoms coupled to optical cavities offer a powerful platform for studying strongly correlated many-body physics. Here, we propose an experimental scheme for creating biatomic molecules via cavity-enhanced photoassociation from an atomic condensate. This setup realizes long-range three-body interactions mediated by tripartite cavity-atom-molecule coupling. Beyond a critical pump strength, a self-organized square lattice phase for molecular condensate emerges, resulting in hybrid atom-molecule superradiance with spontaneous $U(1)$ symmetry breaking. Distinct from previously observed ultracold bosonic (fermionic) atomic superradiance, our findings demonstrate bosonic enhancement characterized by a cubic scaling of steady-state photon number with total atom number. Additionally, strong photon-matter entanglement is shown to effectively characterize superradiant quantum phase transition. Our findings deepen the understanding of quantum superchemistry and exotic many-body nonequilibrium dynamics in cavity-coupled quantum gases.

cond-mat.quant-gas↗

arXiv2024

Programmable Kondo Effect Formed by Landau Levels

Nanobubbles wield significant influence over the electronic properties of 2D materials, showing diverse applications ranging from flexible devices to strain sensors. Here, we reveal that a strongly-correlated phenomenon, i.e., Kondo resonance, naturally takes place as an intrinsic property of graphene nanobubbles. The localized strain within the nanobubbles engenders pseudo magnetic fields, driving pseudo Landau levels with degenerate Landau orbits. Under the Coulomb repulsion, the Landau orbits form an effective $\mathrm{SU}(N)$ pseudospin coupled to the bath via exchange interaction. This results in a new flavor screening mechanism that drives an exotic flavor-frozen Kondo effect, which is absent in conventional Kondo systems. The resonance here also exhibits an unparalleled tunability via strain engineering, establishing a versatile new platform to simulate novel correlated phenomena based on pseudo Landau levels.

cond-mat.str-el↗

arXiv2024

FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models

Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored, and their performance on complex tasks like financial agent remains unknown. This paper presents FinEval, a benchmark designed to evaluate LLMs' financial domain knowledge and practical abilities. The dataset contains 8,351 questions categorized into four different key areas: Financial Academic Knowledge, Financial Industry Knowledge, Financial Security Knowledge, and Financial Agent. Financial Academic Knowledge comprises 4,661 multiple-choice questions spanning 34 subjects such as finance and economics. Financial Industry Knowledge contains 1,434 questions covering practical scenarios like investment research. Financial Security Knowledge assesses models through 1,640 questions on topics like application security and cryptography. Financial Agent evaluates tool usage and complex reasoning with 616 questions. FinEval has multiple evaluation settings, including zero-shot, five-shot with chain-of-thought, and assesses model performance using objective and subjective criteria. Our results show that Claude 3.5-Sonnet achieves the highest weighted average score of 72.9 across all financial domain categories under zero-shot setting. Our work provides a comprehensive benchmark closely aligned with Chinese financial domain.

cs.CL↗

arXiv2024

Quantum delayed "choice" based on vectorially structured photon

Whether a photon exhibits wavelike or particlelike behaviour depends on the observation method, as clearly demonstrated by Wheeler's delayed choice (DC) experiments. A key aspect of such experiments is the random determination of the observation device's status, typically controlled by a random number generator or a quantum-controlling apparatus. Here, we propose a novel version of the quantum delayed choice (QDC) experiment by tailoring the quantum state of the single photon into an arbitrary polarization superposition. In this experiment, the "choice" can be considered as being made by the photon's state itself at the moment of observation, thereby violating classical causality. Additionally, we observe the morphing behaviour of the single photon between wavelike and particlelike characteristics, which challenges the classical picture of waves and particles. Utilizing the quantum state of the photon rather than the quantum-controlling devices not only facilitates the implementation of the QDC experiment but also helps deepen the understanding of Bohr's complementarity principle.

quant-ph↗

arXiv2024

Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective

Generative models, particularly diffusion models, have achieved remarkable success in density estimation for multimodal data, drawing significant interest from the reinforcement learning (RL) community, especially in policy modeling in continuous action spaces. However, existing works exhibit significant variations in training schemes and RL optimization objectives, and some methods are only applicable to diffusion models. In this study, we compare and analyze various generative policy training and deployment techniques, identifying and validating effective designs for generative policy algorithms. Specifically, we revisit existing training objectives and classify them into two categories, each linked to a simpler approach. The first approach, Generative Model Policy Optimization (GMPO), employs a native advantage-weighted regression formulation as the training objective, which is significantly simpler than previous methods. The second approach, Generative Model Policy Gradient (GMPG), offers a numerically stable implementation of the native policy gradient method. We introduce a standardized experimental framework named GenerativeRL. Our experiments demonstrate that the proposed methods achieve state-of-the-art performance on various offline-RL datasets, offering a unified and practical guideline for training and deploying generative policies.

cs.LG↗

arXiv2024

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models

Fine-tuning is a crucial process for adapting large language models (LLMs) to diverse applications. In certain scenarios, such as multi-tenant serving, deploying multiple LLMs becomes necessary to meet complex demands. Recent studies suggest decomposing a fine-tuned LLM into a base model and corresponding delta weights, which are then compressed using low-rank or low-bit approaches to reduce costs. In this work, we observe that existing low-rank and low-bit compression methods can significantly harm the model performance for task-specific fine-tuned LLMs (e.g., WizardMath for math problems). Motivated by the long-tail distribution of singular values in the delta weights, we propose a delta quantization approach using mixed-precision. This method employs higher-bit representation for singular vectors corresponding to larger singular values. We evaluate our approach on various fine-tuned LLMs, including math LLMs, code LLMs, chat LLMs, and even VLMs. Experimental results demonstrate that our approach performs comparably to full fine-tuned LLMs, surpassing both low-rank and low-bit baselines by a considerable margin. Additionally, we show that our method is compatible with various backbone LLMs, such as Llama-2, Llama-3, and Mistral, highlighting its generalizability.

cs.CL↗

arXiv2024

SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation

Out-of-distribution (OOD) detection is crucial for the safe deployment of neural networks. Existing CLIP-based approaches perform OOD detection by devising novel scoring functions or sophisticated fine-tuning methods. In this work, we propose SeTAR, a novel, training-free OOD detection method that leverages selective low-rank approximation of weight matrices in vision-language and vision-only models. SeTAR enhances OOD detection via post-hoc modification of the model's weight matrices using a simple greedy search algorithm. Based on SeTAR, we further propose SeTAR+FT, a fine-tuning extension optimizing model performance for OOD detection tasks. Extensive evaluations on ImageNet1K and Pascal-VOC benchmarks show SeTAR's superior performance, reducing the relatively false positive rate by up to 18.95% and 36.80% compared to zero-shot and fine-tuning baselines. Ablation studies further validate SeTAR's effectiveness, robustness, and generalizability across different model backbones. Our work offers a scalable, efficient solution for OOD detection, setting a new state-of-the-art in this area.

cs.CL↗

arXiv2024

Distract Large Language Models for Automatic Jailbreak Attack

Extensive efforts have been made before the public release of Large language models (LLMs) to align their behaviors with human values. However, even meticulously aligned LLMs remain vulnerable to malicious manipulations such as jailbreaking, leading to unintended behaviors. In this work, we propose a novel black-box jailbreak framework for automated red teaming of LLMs. We designed malicious content concealing and memory reframing with an iterative optimization algorithm to jailbreak LLMs, motivated by the research about the distractibility and over-confidence phenomenon of LLMs. Extensive experiments of jailbreaking both open-source and proprietary LLMs demonstrate the superiority of our framework in terms of effectiveness, scalability and transferability. We also evaluate the effectiveness of existing jailbreak defense methods against our attack and highlight the crucial need to develop more effective and practical defense strategies.

cs.CR↗

arXiv2024

TSI: A Multi-View Representation Learning Approach for Time Series Forecasting

As the growing demand for long sequence time-series forecasting in real-world applications, such as electricity consumption planning, the significance of time series forecasting becomes increasingly crucial across various domains. This is highlighted by recent advancements in representation learning within the field. This study introduces a novel multi-view approach for time series forecasting that innovatively integrates trend and seasonal representations with an Independent Component Analysis (ICA)-based representation. Recognizing the limitations of existing methods in representing complex and high-dimensional time series data, this research addresses the challenge by combining TS (trend and seasonality) and ICA (independent components) perspectives. This approach offers a holistic understanding of time series data, going beyond traditional models that often miss nuanced, nonlinear relationships. The efficacy of TSI model is demonstrated through comprehensive testing on various benchmark datasets, where it shows superior performance over current state-of-the-art models, particularly in multivariate forecasting. This method not only enhances the accuracy of forecasting but also contributes significantly to the field by providing a more in-depth understanding of time series data. The research which uses ICA for a view lays the groundwork for further exploration and methodological advancements in time series forecasting, opening new avenues for research and practical applications.

cs.LG↗

arXiv2024

G3R: Gradient Guided Generalizable Reconstruction

Large scale 3D scene reconstruction is important for applications such as virtual reality and simulation. Existing neural rendering approaches (e.g., NeRF, 3DGS) have achieved realistic reconstructions on large scenes, but optimize per scene, which is expensive and slow, and exhibit noticeable artifacts under large view changes due to overfitting. Generalizable approaches or large reconstruction models are fast, but primarily work for small scenes/objects and often produce lower quality rendering results. In this work, we introduce G3R, a generalizable reconstruction approach that can efficiently predict high-quality 3D scene representations for large scenes. We propose to learn a reconstruction network that takes the gradient feedback signals from differentiable rendering to iteratively update a 3D scene representation, combining the benefits of high photorealism from per-scene optimization with data-driven priors from fast feed-forward prediction methods. Experiments on urban-driving and drone datasets show that G3R generalizes across diverse large scenes and accelerates the reconstruction process by at least 10x while achieving comparable or better realism compared to 3DGS, and also being more robust to large view changes.

cs.CV↗

arXiv2024

SpikingRx: From Neural to Spiking Receiver

In this work, we propose an energy efficient neuromorphic receiver to replace multiple signal-processing blocks at the receiver by a Spiking Neural Network (SNN) based module, called SpikingRx. We propose a deep convolutional SNN with spike-element-wise ResNet layers which takes a whole OFDM grid compliant with 5G specifications and provides soft outputs for decoded bits that can be used as log-likelihood ratios. We propose to employ the surrogate gradient descent method for training the SpikingRx and focus on its generalizability and robustness to quantization. Moreover, the interpretability of the proposed SpikingRx is studied by a comprehensive ablation study. Our extensive numerical simulations show that SpikingRx is capable of achieving significant block error rate performance gain compared to conventional 5G receivers and similar performance compared to its traditional NN-based counterparts with approximately 9x less energy consumption.

cs.IT↗

arXiv2024

CSST Strong Lensing Preparation: Forecasting the galaxy-galaxy strong lensing population for the China Space Station Telescope

Galaxy-galaxy strong gravitational lensing (GGSL) is a powerful probe for the formation and evolution of galaxies and cosmology, while the sample size of GGSLs leads to considerable uncertainties and potential bias. The China Space Station Telescope (CSST, to be launched in late 2026) will conduct observations across 17,500 square degrees of the sky, capturing images in the $ugriz$ bands with a spatial resolution comparable to that of the Hubble Space Telescope. We ran a set of Monte Carlo simulations to predict that the CSST's wide-field survey will observe $\sim$160,000 galaxy-galaxy strong lenses over its lifespan, increasing the number of existing galaxy-galaxy strong lens samples by three orders of magnitude. This is comparable to the capabilities of the $\it Euclid$ telescope but with the added benefit of additional color information. Specifically, the CSST can detect strong lenses with Einstein radii about $0.64\pm0.42^{"}$, corresponding to the velocity dispersions of $217.19 \pm 50.55 \, \text{km/s}$. These lenses exhibit a median magnification of $\sim$5. The apparent magnitude of the unlensed sources in the g-band is $25.87 \pm 1.19$. The signal-to-noise ratio of the lensed images covers a range of $\sim 20$ to $\sim 1000$, allowing us to determine the Einstein radius with an accuracy ranging from $\sim 1 \%$ to $\sim 0.1 \%$, ignoring various modeling systematics. Our estimates indicate that CSST can observe rare systems like double source-plane and spiral galaxy lenses. The above selection functions of the CSST strong lensing observation help optimize the strategy of finding and modeling GGSLs.

astro-ph.GA↗

arXiv2024

Generation and characterization of customized perfect Laguerre-Gaussian beams with arbitrary profiles

We experimentally demonstrate the generation of customized perfect Laguerre-Gaussian (PLG) beams whose intensity maxima localized around any desired curves. The principle is to act appropriate algebraic functions on the angular spectra of PLG beams. We characterize the propagation properties of these beams and compare them with non-diffraction caustic beams possessing the same intensity profiles. The results manifest that the customized-PLG beams can maintain their profiles during propagation and suffer less energy loss than the non-diffraction caustic beams, and hence are able to propagate a longer distance. This new structure beam would have potential applications in areas such as optical communication, soliton routing and steering, optical tweezing and trapping, atom optics, etc.

physics.optics↗

arXiv2024

The synthetic gauge field and exotic vortex phase with spin-orbital-angular-momentum coupling

Ultracold atoms endowed with tunable spin-orbital-angular-momentum coupling (SOAMC) represent a promising avenue for delving into exotic quantum phenomena. Building on recent experimental advancements, we propose the generation of synthetic gauge fields ,and by including exotic vortex phases within spinor Bose-Einstein condensates, employing a combination of a running wave and Laguerre-Gaussian laser fields. We investigate the ground-state characteristics of the SOAMC condensate, revealing the emergence of exotic vortex states with controllable orbital angular momenta. It is shown that the interplay of the SOAMC and conventional spin-linear-momentum coupling induced by the running wave beam leads to the formation of a vortex state exhibiting a phase stripe hosting single multiply quantized singularity. The phase of the ground state will undergo the phase transition corresponding to the breaking of rotational symmetry while preserving the mirror symmetry. Importantly, the observed density distribution of the ground-state wavefunction, exhibiting broken rotational symmetry, can be well characterized by the synthetic magnetic field generated through light interaction with the dressed spin state. Our findings pave the way for further exploration into the rotational properties of stable exotic vortices with higher orbital angular momenta against splitting in the presence of synthetic gauge fields in ultracold quantum gases.

cond-mat.quant-gas↗