arXiv subjects

Wei Zhu

Publications and source records attributed to Wei Zhu.

At least 127 records · Page 7Linked to original sources

arXiv2024

Universal time evolution of string order parameter in quantum critical systems with boundary invertible or non-invertible symmetry breaking

The global symmetry, either invertible or non-invertible, has been extensively studied in two dimensional conformal field theories in recent years. When the theory is defined on a manifold with open boundaries, however, many interesting conformal boundary conditions will fully or partially break such global symmetry. In this work, we study the effect of symmetry-breaking boundaries or interfaces when the system is out of equilibrium. We show that the boundary or interface symmetry-breaking can be detected by the time evolution of string order parameters, which are constructed from the symmetry operators that implement the symmetry transformations. While the string order parameters are independent of time if the symmetry is preserved over the whole system, they evolve in time in a universal way if the boundary or interface breaks the symmetry. More explicitly, in the presence of boundary or interface symmetry-breaking, the string order parameters decay exponentially in time after a global quantum quench, and decay as a power-law in time after a local quantum quench. We also generalize our study to the case when the string order parameters are defined in a subsystem, which are related to the full counting statistics. It is found there are also universal features in the time evolution of string order parameters in this case. We verify our field theory results by studying the time evolution of these two different types of string order parameters in lattice models.

cond-mat.str-el↗

arXiv2024

Dynamics of Binary Planets within Star Clusters

We develop analytical tools and perform three-body simulations to investigate the orbital evolution and dynamical stability of binary planets within star clusters. Our analytical results show that the orbital stability of a planetary-mass binary against passing stars is mainly related to its orbital period. Critical flybys, defined as stellar encounters with energy kicks comparable to the binary binding energy, can efficiently produce a wide range of semimajor axes ($a$) and eccentricities ($e$) from a dominant population of primordially tight JuMBOs. The critical flyby criterion we derived offers an improvement over the commonly used tidal radius criterion, particularly in high-speed stellar encounters. Applying our results to the recently discovered Jupiter-Mass Binary Objects (JuMBOs) by the James Webb Space Telescope (JWST), our simulations suggest that to match the observed $\sim$9% wide binary fraction, an initial semimajor axis of $a_0 \sim$ 10-20 au and a density-weighted residence time of $χ\gtrsim 10^4$ Myr pc$^{-3}$ are favored. These results imply that the JWST JuMBOs probably formed as tight binaries near the cluster core.

astro-ph.EP↗

arXiv2024

An extremely low-density exoplanet spins slow

We present constraints on the shape of Kepler-51d, which is a super-puff with a mass $\sim6\,M_\oplus$ and a radius $\sim9\,R_\oplus$, based on detailed modeling of the transit light curve from JWST NIRSpec. The projected shape of this extremely low-density planet is consistent with being spherical, and a projected oblateness $f_\perp>0.2$ can be excluded regardless of the spin obliquity angles. If this is taken as the limit on the true shape of the planet, Kepler-51d is rotating at $\lesssim 50\%$ of its break-up spin rate, or its rotation period is $\gtrsim 33\,$hr. In the more plausible situation that the planetary spin is aligned with its orbital direction to within $30^\circ$, then its oblateness is $<0.08$, which corresponds to a dimensionless spin rate $\lesssim30\%$ of the break-up rotation and a dimensional rotation period $\gtrsim 53\,$hr. This seems to contradict the theoretical expectation that planets with such low masses may be spinning near break-up. We point out the usefulness of the stellar mean density and the orbital eccentricity in constraining the shape of the transiting planet, so planets with well-characterized host and orbital parameters are preferred in the detection of planetary oblateness with the JWST transit method.

astro-ph.EP↗

arXiv2024

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning

Low-rank adaptation (LoRA) and its mixture-of-experts (MOE) variants are highly effective parameter-efficient fine-tuning (PEFT) methods. However, they introduce significant latency in multi-tenant settings due to the LoRA modules and MOE routers added to multiple linear modules in the Transformer layer. To address this issue, we propose Mixture of Low-Rank Adaptation (MiLoRA), a novel and efficient LoRA variant. MiLoRA differs from previous MOE-style LoRA methods by considering each LoRA module as an expert and employing a prompt-aware routing mechanism. This mechanism calculates expert routing results once before generating the first new token and reuses these results for subsequent tokens, reducing latency. Extensive experiments and analysis on commonsense reasoning tasks, math reasoning tasks, and widely used LLM evaluation benchmarks demonstrate that MiLoRA consistently outperforms strong PEFT baselines with comparable tunable parameter budgets. Additionally, MiLoRA significantly reduces latency in multi-tenant settings compared to previous LoRA-based methods.

cs.CL↗

arXiv2024

Low-Rank Continual Pyramid Vision Transformer: Incrementally Segment Whole-Body Organs in CT with Light-Weighted Adaptation

Deep segmentation networks achieve high performance when trained on specific datasets. However, in clinical practice, it is often desirable that pretrained segmentation models can be dynamically extended to enable segmenting new organs without access to previous training datasets or without training from scratch. This would ensure a much more efficient model development and deployment paradigm accounting for the patient privacy and data storage issues. This clinically preferred process can be viewed as a continual semantic segmentation (CSS) problem. Previous CSS works would either experience catastrophic forgetting or lead to unaffordable memory costs as models expand. In this work, we propose a new continual whole-body organ segmentation model with light-weighted low-rank adaptation (LoRA). We first train and freeze a pyramid vision transformer (PVT) base segmentation model on the initial task, then continually add light-weighted trainable LoRA parameters to the frozen model for each new learning task. Through a holistically exploration of the architecture modification, we identify three most important layers (i.e., patch-embedding, multi-head attention and feed forward layers) that are critical in adapting to the new segmentation tasks, while retaining the majority of the pretrained parameters fixed. Our proposed model continually segments new organs without catastrophic forgetting and meanwhile maintaining a low parameter increasing rate. Continually trained and tested on four datasets covering different body parts of a total of 121 organs, results show that our model achieves high segmentation accuracy, closely reaching the PVT and nnUNet upper bounds, and significantly outperforms other regularization-based CSS methods. When comparing to the leading architecture-based CSS method, our model has a substantial lower parameter increasing rate while achieving comparable performance.

cs.CV↗

arXiv2024

Data-Driven Discovery of Conservation Laws from Trajectories via Neural Deflation

In an earlier work by a subset of the present authors, the method of the so-called neural deflation was introduced towards identifying a complete set of functionally independent conservation laws of a nonlinear dynamical system. Here, we extend by a significant step this proposal. Instead of using the explicit knowledge of the underlying equations of motion, we develop the method directly from system trajectories. This is crucial towards enhancing the practical implementation of the method in scenarios where solely data reflecting discrete snapshots of the system are available. We showcase the results of the method and the number of associated conservation laws obtained in a diverse range of examples including 1D and 2D harmonic oscillators, the Toda lattice, the Fermi-Pasta-Ulam-Tsingou lattice and the Calogero-Moser system.

nlin.PS↗

arXiv2024

PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdification

Due to their substantial sizes, large language models (LLMs) are typically deployed within a single-backbone multi-tenant framework. In this setup, a single instance of an LLM backbone must cater to multiple users or tasks through the application of various parameter-efficient fine-tuning (PEFT) models. Despite the availability of numerous effective PEFT techniques such as LoRA, there remains a need for a PEFT approach that achieves both high efficiency during inference and competitive performance on downstream tasks. In this research, we introduce a new and straightforward PEFT methodology named \underline{P}rompt D\underline{E}pen\underline{D}ent \underline{R}epresentation M\underline{O}dification (PEDRO). The proposed method involves integrating a lightweight vector generator into each Transformer layer, which generates vectors contingent upon the input prompts. These vectors then modify the hidden representations created by the LLM through a dot product operation, thereby influencing the semantic output and generated content of the model. Extensive experimentation across a variety of tasks indicates that: (a) PEDRO surpasses recent PEFT benchmarks when using a similar number of tunable parameters. (b) Under the single-backbone multi-tenant deployment model, PEDRO exhibits superior efficiency compared to LoRA, indicating significant industrial potential.

cs.CL↗

arXiv2024

CorrAdaptor: Adaptive Local Context Learning for Correspondence Pruning

In the fields of computer vision and robotics, accurate pixel-level correspondences are essential for enabling advanced tasks such as structure-from-motion and simultaneous localization and mapping. Recent correspondence pruning methods usually focus on learning local consistency through k-nearest neighbors, which makes it difficult to capture robust context for each correspondence. We propose CorrAdaptor, a novel architecture that introduces a dual-branch structure capable of adaptively adjusting local contexts through both explicit and implicit local graph learning. Specifically, the explicit branch uses KNN-based graphs tailored for initial neighborhood identification, while the implicit branch leverages a learnable matrix to softly assign neighbors and adaptively expand the local context scope, significantly enhancing the model's robustness and adaptability to complex image variations. Moreover, we design a motion injection module to integrate motion consistency into the network to suppress the impact of outliers and refine local context learning, resulting in substantial performance improvements. The experimental results on extensive correspondence-based tasks indicate that our CorrAdaptor achieves state-of-the-art performance both qualitatively and quantitatively. The code and pre-trained models are available at https://github.com/TaoWangzj/CorrAdaptor.

cs.CV↗

arXiv2024

OGLE-2015-BLG-0845L: A low-mass M dwarf from the microlensing parallax and xallarap effects

We present the analysis of the microlensing event OGLE-2015-BLG-0845, which was affected by both the microlensing parallax and xallarap effects. The former was detected via the simultaneous observations from the ground and Spitzer, and the latter was caused by the orbital motion of the source star in a relatively close binary. The combination of these two effects led to a mass measurement of the lens object, revealing a low-mass ($0.14 \pm 0.05 M_{\odot}$) M-dwarf at the bulge distance ($7.6 \pm 1.0$ kpc). The source binary consists of a late F-type subgiant and a K-type dwarf of $\sim1.2 M_{\odot}$ and $\sim 0.9 M_{\odot}$, respectively, and the orbital period is $70 \pm 10$ days. OGLE-2015-BLG-0845 is the first single-lens event in which the lens mass is measured via the binarity of the source. Given the abundance of binary systems as potential microlensing sources, the xallarap effect may not be a rare phenomenon. Our work thus highlights the application of the xallarap effect in the mass determination of microlenses, and the same method can be used to identify isolated dark lenses.

astro-ph.SR↗

arXiv2024

Analysis of the full Spitzer microlensing sample I: Dark remnant candidates and Gaia predictions

In the pursuit of understanding the population of stellar remnants within the Milky Way, we analyze the sample of $\sim 950$ microlensing events observed by the Spitzer Space Telescope between 2014 and 2019. In this study we focus on a sub-sample of nine microlensing events, selected based on their long timescales, small microlensing parallaxes and joint observations by the Gaia mission, to increase the probability that the chosen lenses are massive and the mass is measurable. Among the selected events we identify lensing black holes and neutron star candidates, with potential confirmation through forthcoming release of the Gaia time-series astrometry in 2026. Utilizing Bayesian analysis and Galactic models, along with the Gaia Data Release 3 proper motion data, four good candidates for dark remnants were identified: OGLE-2016-BLG-0293, OGLE-2018-BLG-0483, OGLE-2018-BLG-0662, and OGLE-2015-BLG-0149, with lens masses of $2.98^{+1.75}_{-1.28}~M_{\odot}$, $4.65^{+3.12}_{-2.08}~M_{\odot}$, $3.15^{+0.66}_{-0.64}~M_{\odot}$ and $1.4^{+0.75}_{-0.55}~M_{\odot}$, respectively. Notably, the first two candidates are expected to exhibit astrometric microlensing signals detectable by Gaia, offering the prospect of validating the lens masses. The methodologies developed in this work will be applied to the full Spitzer microlensing sample, populating and analyzing the time-scale ($t_{\rm E}$) vs. parallax ($π_{\rm E}$) diagram to derive constraints on the population of lenses in general and massive remnants in particular.

astro-ph.GA↗

arXiv2024

UltraFeedback: Boosting Language Models with Scaled AI Feedback

Learning from human feedback has become a pivot technique in aligning large language models (LLMs) with human preferences. However, acquiring vast and premium human feedback is bottlenecked by time, labor, and human capability, resulting in small sizes or limited topics of current datasets. This further hinders feedback learning as well as alignment research within the open-source community. To address this issue, we explore how to go beyond human feedback and collect high-quality \textit{AI feedback} automatically for a scalable alternative. Specifically, we identify \textbf{scale and diversity} as the key factors for feedback data to take effect. Accordingly, we first broaden instructions and responses in both amount and breadth to encompass a wider range of user-assistant interactions. Then, we meticulously apply a series of techniques to mitigate annotation biases for more reliable AI feedback. We finally present \textsc{UltraFeedback}, a large-scale, high-quality, and diversified AI feedback dataset, which contains over 1 million GPT-4 feedback for 250k user-assistant conversations from various aspects. Built upon \textsc{UltraFeedback}, we align a LLaMA-based model by best-of-$n$ sampling and reinforcement learning, demonstrating its exceptional performance on chat benchmarks. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models, serving as a solid foundation for future feedback learning research. Our data and models are available at https://github.com/thunlp/UltraFeedback.

cs.CL↗

arXiv2024

Quantum Monte Carlo Simulation of the 3D Ising Transition on the Fuzzy Sphere

We present a numerical quantum Monte Carlo (QMC) method for simulating the 3D phase transition on the recently proposed fuzzy sphere [Phys. Rev. X 13, 021009 (2023)]. By introducing an additional $SU(2)$ layer degree of freedom, we reformulate the model into a form suitable for sign-problem-free QMC simulation. From the finite-size-scaling, we show that this QMC-friendly model undergoes a quantum phase transition belonging to the 3D Ising universality class, and at the critical point we compute the scaling dimensions from the state-operator correspondence, which largely agrees with the prediction from the conformal field theory. These results pave the way to construct sign-problem-free models for QMC simulations on the fuzzy sphere, which could advance the future study on more sophisticated criticalities.

cond-mat.str-el↗

arXiv2024

Efficient LLM inference solution on Intel GPU

Transformer based Large Language Models (LLMs) have been widely used in many fields, and the efficiency of LLM inference becomes hot topic in real applications. However, LLMs are usually complicatedly designed in model structure with massive operations and perform inference in the auto-regressive mode, making it a challenging task to design a system with high efficiency. In this paper, we propose an efficient LLM inference solution with low latency and high throughput. Firstly, we simplify the LLM decoder layer by fusing data movement and element-wise operations to reduce the memory access frequency and lower system latency. We also propose a segment KV cache policy to keep key/value of the request and response tokens in separate physical memory for effective device memory management, helping enlarge the runtime batch size and improve system throughput. A customized Scaled-Dot-Product-Attention kernel is designed to match our fusion policy based on the segment KV cache solution. We implement our LLM inference solution on Intel GPU and publish it publicly. Compared with the standard HuggingFace implementation, the proposed solution achieves up to 7x lower token latency and 27x higher throughput for some popular LLMs on Intel GPU.

cs.AR↗

arXiv2024

IAPT: Instruction-Aware Prompt Tuning for Large Language Models

Soft prompt tuning is a widely studied parameter-efficient fine-tuning method. However, it has a clear drawback: many soft tokens must be inserted into the input sequences to guarantee downstream performance. As a result, soft prompt tuning is less considered than Low-rank adaptation (LoRA) in the large language modeling (LLM) era. In this work, we propose a novel prompt tuning method, Instruction-Aware Prompt Tuning (IAPT), that requires only four soft tokens. First, we install a parameter-efficient soft prompt generator at each Transformer layer to generate idiosyncratic soft prompts for each input instruction. The generated soft prompts can be seen as a semantic summary of the input instructions and can effectively guide the output generation. Second, the soft prompt generators are modules with a bottleneck architecture consisting of a self-attention pooling operation, two linear projections, and an activation function. Pilot experiments show that prompt generators at different Transformer layers require different activation functions. Thus, we propose to learn the idiosyncratic activation functions for prompt generators automatically with the help of rational functions. We have conducted experiments on various tasks, and the experimental results demonstrate that (a) our IAPT method can outperform the recent baselines with comparable tunable parameters. (b) Our IAPT method is more efficient than LoRA under the single-backbone multi-tenant setting.

cs.CL↗

arXiv2024

Identification of moment equations via data-driven approaches in nonlinear schrodinger models

The moment quantities associated with the nonlinear Schrodinger equation offer important insights towards the evolution dynamics of such dispersive wave partial differential equation (PDE) models. The effective dynamics of the moment quantities is amenable to both analytical and numerical treatments. In this paper we present a data-driven approach associated with the Sparse Identification of Nonlinear Dynamics (SINDy) to numerically capture the evolution behaviors of such moment quantities. Our method is applied first to some well-known closed systems of ordinary differential equations (ODEs) which describe the evolution dynamics of relevant moment quantities. Our examples are, progressively, of increasing complexity and our findings explore different choices within the SINDy library. We also consider the potential discovery of coordinate transformations that lead to moment system closure. Finally, we extend considerations to settings where a closed analytical form of the moment dynamics is not available.

nlin.PS↗

arXiv2024

TCMBench: A Comprehensive Benchmark for Evaluating Large Language Models in Traditional Chinese Medicine

Large language models (LLMs) have performed remarkably well in various natural language processing tasks by benchmarking, including in the Western medical domain. However, the professional evaluation benchmarks for LLMs have yet to be covered in the traditional Chinese medicine(TCM) domain, which has a profound history and vast influence. To address this research gap, we introduce TCM-Bench, an comprehensive benchmark for evaluating LLM performance in TCM. It comprises the TCM-ED dataset, consisting of 5,473 questions sourced from the TCM Licensing Exam (TCMLE), including 1,300 questions with authoritative analysis. It covers the core components of TCMLE, including TCM basis and clinical practice. To evaluate LLMs beyond accuracy of question answering, we propose TCMScore, a metric tailored for evaluating the quality of answers generated by LLMs for TCM related questions. It comprehensively considers the consistency of TCM semantics and knowledge. After conducting comprehensive experimental analyses from diverse perspectives, we can obtain the following findings: (1) The unsatisfactory performance of LLMs on this benchmark underscores their significant room for improvement in TCM. (2) Introducing domain knowledge can enhance LLMs' performance. However, for in-domain models like ZhongJing-TCM, the quality of generated analysis text has decreased, and we hypothesize that their fine-tuning process affects the basic LLM capabilities. (3) Traditional metrics for text generation quality like Rouge and BertScore are susceptible to text length and surface semantic ambiguity, while domain-specific metrics such as TCMScore can further supplement and explain their evaluation results. These findings highlight the capabilities and limitations of LLMs in the TCM and aim to provide a more profound assistance to medical research.

cs.CL↗

arXiv2024

Revealing mysteries in gamma-ray bursts: the role of gluon condensation

We use a newly recognized gluon distribution in the nucleon, which was predicted by a QCD evolution equation to consistently explain several intriguing phenomena associated with gamma-ray bursts. They are the GeV-TeV spectra of GRB 221009A, the remarkably symmetrical explosion cloud in kilonova AT2017gfo, and the absence of a very high-energy gamma-ray signature in GRB 170817A. We find that these occurrences can be attributed to the gluon condensation within nucleons, i.e., a significant number of soft gluons within nucleons are condensed at a critical momentum, resulting in the emergence of a steep and high peak in the gluon distributions. Through this profound connection between microscopic and macroscopic phenomena, we have not only expanded the applications of the hadronic scenario in cosmic gamma-ray emissions but also presented new evidence for the existence of gluon condensation.

astro-ph.HE↗

arXiv2024

The Kinematic and Chemical Properties of the Close-in Planet Host Star 8 UMi

A recent study by Hon et al. reported that a close-in planet around the red clump star, 8 UMi, should have been engulfed during the expansion phase of its parent star's evolution. They explained the survival of this exoplanet through a binary-merger channel for 8 UMi. The key to testing this formation scenario is to derive the true age of this star: is it an old "imposter" resulting from a binary merger, or a genuinely young red clump giant? To accomplish this, we derive kinematic and chemical properties for 8 UMi using astrometric data from {\it Gaia} DR3 and the element-abundance pattern measured from a high-resolution ($R \sim 75,000$) spectrum taken by SOPHIE. Our analysis shows that 8 UMi is a normal thin-disk star with orbital rotation speed of $\it{V}_\mathrmϕ=\mathrm{244.96 km s^{-1}}$, and possesses a Solar metallicity ([Fe/H] $= -0.05 \pm 0.07$) and $α$-element abundance ratio ([$α$/Fe] $= +0.01 \pm 0.03$). By adopting well-established relationships between age and space velocities/elemental abundances, we estimate a kinematic age of $3.50^{+3.00}_{-2.00}$ Gyr, and a chemical age of $3.25^{+2.50}_{-1.50}$ Gyr from [C/N] and $3.47 \pm 1.96$ Gyr from [Y/Mg] for 8 UMi, respectively. These estimates are consistent with the isochrone-fitting age ($1.90^{+1.15}_{-0.30}$ Gyr) of 8 UMi, but are all much younger than the timescale required in a binary-merger scenario. This result challenges the binary-merger model; the existence of such a closely orbiting exoplanet around a giant star remains a mystery yet to be resolved.

astro-ph.EP↗