arXiv subjects

Yi Luo

Publications and source records attributed to Yi Luo.

At least 73 records · Page 4Linked to original sources

arXiv2024

Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architectures

This study systematically analyzes the vulnerability of 36 large language models (LLMs) to various prompt injection attacks, a technique that leverages carefully crafted prompts to elicit malicious LLM behavior. Across 144 prompt injection tests, we observed a strong correlation between model parameters and vulnerability, with statistical analyses, such as logistic regression and random forest feature analysis, indicating that parameter size and architecture significantly influence susceptibility. Results revealed that 56 percent of tests led to successful prompt injections, emphasizing widespread vulnerability across various parameter sizes, with clustering analysis identifying distinct vulnerability profiles associated with specific model configurations. Additionally, our analysis uncovered correlations between certain prompt injection techniques, suggesting potential overlaps in vulnerabilities. These findings underscore the urgent need for robust, multi-layered defenses in LLMs deployed across critical infrastructure and sensitive industries. Successful prompt injection attacks could result in severe consequences, including data breaches, unauthorized access, or misinformation. Future research should explore multilingual and multi-step defenses alongside adaptive mitigation strategies to strengthen LLM security in diverse, real-world environments.

cs.CR↗

arXiv2024

Xinyu: An Efficient LLM-based System for Commentary Generation

Commentary provides readers with a deep understanding of events by presenting diverse arguments and evidence. However, creating commentary is a time-consuming task, even for skilled commentators. Large language models (LLMs) have simplified the process of natural language generation, but their direct application in commentary creation still faces challenges due to unique task requirements. These requirements can be categorized into two levels: 1) fundamental requirements, which include creating well-structured and logically consistent narratives, and 2) advanced requirements, which involve generating quality arguments and providing convincing evidence. In this paper, we introduce Xinyu, an efficient LLM-based system designed to assist commentators in generating Chinese commentaries. To meet the fundamental requirements, we deconstruct the generation process into sequential steps, proposing targeted strategies and supervised fine-tuning (SFT) for each step. To address the advanced requirements, we present an argument ranking model for arguments and establish a comprehensive evidence database that includes up-to-date events and classic books, thereby strengthening the substantiation of the evidence with retrieval augmented generation (RAG) technology. To evaluate the generated commentaries more fairly, corresponding to the two-level requirements, we introduce a comprehensive evaluation metric that considers five distinct perspectives in commentary generation. Our experiments confirm the effectiveness of our proposed system. We also observe a significant increase in the efficiency of commentators in real-world scenarios, with the average time spent on creating a commentary dropping from 4 hours to 20 minutes. Importantly, such an increase in efficiency does not compromise the quality of the commentaries.

cs.CL↗

arXiv2024

Microwave-assisted unidirectional superconductivity in Al-InAs nanowire-Al junctions under magnetic fields

Under certain symmetry-breaking conditions, a superconducting system exhibits asymmetric critical currents, dubbed the ``superconducting diode effect". Recently, systems with the ideal superconducting diode efficiency or unidirectional superconductivity have received considerable interest. In this work, we report the study of Al-InAs nanowire-Al Josephson junctions under microwave irradiation and magnetic fields. We observe an enhancement of superconducting diode effect under microwave driving, featured by a horizontal offset of the zero-voltage step in the voltage-current characteristic that increases with microwave power. Devices reach the unidirectional superconductivity regime at sufficiently high driving amplitudes. The offset changes sign with the reversal of the magnetic field direction. Meanwhile, the offset magnitude exhibits a roughly linear response to the microwave power in dBm when both the power and the magnetic field are large. The signatures observed are reminiscent of a recent theoretical proposal using the resistively shunted junction (RSJ) model. However, the experimental results are not fully explained by the RSJ model, indicating a new mechanism for unidirectional superconductivity that is possibly related to non-equilibrium dynamics or dissipation in periodically driven superconducting systems.

cond-mat.mes-hall↗

arXiv2024

One-dimensional quantum dot array integrated with charge sensors in an InAs nanowire

We report an experimental study of a one-dimensional quintuple-quantum-dot array integrated with two quantum dot charge sensors in an InAs nanowire. The device is studied by measuring double quantum dots formed consecutively in the array and corresponding charge stability diagrams are revealed with both direct current measurements and charge sensor signals. The one-dimensional quintuple-quantum-dot array are then tuned up and its charge configurations are fully mapped out with the two charge sensors. The energy level of each dot in the array can be controlled individually by using a compensated gate architecture (i.e., "virtual gate"). After that, four dots in the array are selected to form two double quantum dots and ultra strong inter-double-dot interaction is obtained. A theoretical simulation based on a 4-dimensional Hamiltonian confirms the strong coupling strength between the two double quantum dots. The highly controllable one-dimensional quantum dot array achieved in this work is expected to be valuable for employing InAs nanowires to construct advanced quantum hardware in the future.

cond-mat.mes-hall↗

arXiv2024

RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization

In the realm of resource-constrained mobile vision tasks, the pursuit of efficiency and performance consistently drives innovation in lightweight Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). While ViTs excel at capturing global context through self-attention mechanisms, their deployment in resource-limited environments is hindered by computational complexity and latency. Conversely, lightweight CNNs are favored for their parameter efficiency and low latency. This study investigates the complementary advantages of CNNs and ViTs to develop a versatile vision backbone tailored for resource-constrained applications. We introduce RepNeXt, a novel model series integrates multi-scale feature representations and incorporates both serial and parallel structural reparameterization (SRP) to enhance network depth and width without compromising inference speed. Extensive experiments demonstrate RepNeXt's superiority over current leading lightweight CNNs and ViTs, providing advantageous latency across various vision benchmarks. RepNeXt-M4 matches RepViT-M1.5's 82.3\% accuracy on ImageNet within 1.5ms on an iPhone 12, outperforms its AP$^{box}$ by 1.3 on MS-COCO, and reduces parameters by 0.7M. Codes and models are available at https://github.com/suous/RepNeXt.

cs.CV↗

arXiv2024

Gull: A Generative Multifunctional Audio Codec

We introduce Gull, a generative multifunctional audio codec. Gull is a general purpose neural audio compression and decompression model which can be applied to a wide range of tasks and applications such as real-time communication, audio super-resolution, and codec language models. The key components of Gull include (1) universal-sample-rate modeling via subband modeling schemes motivated by recent progress in audio source separation, (2) gain-shape representations motivated by traditional audio codecs, (3) improved residual vector quantization modules, (4) elastic decoder network that enables user-defined model size and complexity during inference time, (5) built-in ability for audio super-resolution without the increase of bitrate. We compare Gull with existing traditional and neural audio codecs and show that Gull is able to achieve on par or better performance across various sample rates, bitrates and model complexities in both subjective and objective evaluation metrics.

eess.AS↗

arXiv2024

NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism

We present NewsBench, a novel evaluation framework to systematically assess the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism. Our constructed benchmark dataset is focused on four facets of writing proficiency and six facets of safety adherence, and it comprises manually and carefully designed 1,267 test samples in the types of multiple choice questions and short answer questions for five editorial tasks in 24 news domains. To measure performances, we propose different GPT-4 based automatic evaluation protocols to assess LLM generations for short answer questions in terms of writing proficiency and safety adherence, and both are validated by the high correlations with human evaluations. Based on the systematic evaluation framework, we conduct a comprehensive analysis of ten popular LLMs which can handle Chinese. The experimental results highlight GPT-4 and ERNIE Bot as top performers, yet reveal a relative deficiency in journalistic safety adherence in creative writing tasks. Our findings also underscore the need for enhanced ethical guidance in machine-generated journalistic content, marking a step forward in aligning LLMs with journalistic standards and safety considerations.

cs.CL↗

arXiv2024

Rapidly Achieving Chemical Accuracy with Quantum Computing Enforced Language Model

Finding accurate ground state energy of a many-body system has been a major challenge in quantum chemistry. The integration of classic and quantum computers has shed new light on resolving this outstanding problem. Here we propose QiankunNet-VQE, a transformer based language models enforced with quantum computing to learn and generate quantum states. It has been implemented using up to 12 qubits and attaining an accuracy level competitive with state-of-the-art classical methods. By leveraging both quantum and classical resources, this scheme overcomes the limitations of variational quantum eigensolver(VQE) without the need for cumbersome error mitigation. Moreover, QiankunNet-VQE provides a different route to achieve a practical quantum advantage for solving many-electron Schrödinger equation without requiring extremely precise preparation and measurement of the ground-state wavefunction on quantum computer.

quant-ph↗

arXiv2024

Gate-defined quantum point contacts in a germanium quantum well

We report an experimental study of quantum point contacts defined in a high-quality strained germanium quantum well with layered electric gates. At zero magnetic field, we observe quantized conductance plateaus in units of 2$e^2/h$. Bias-spectroscopy measurements reveal that the energy spacing between successive one-dimensional subbands ranges from 1.5 to 5\,meV as a consequence of the small effective mass of the holes and the narrow gate constrictions. At finite magnetic fields perpendicular to the device plane, the edges of the conductance plateaus get splitted due to the Zeeman effect and Landé $g$ factors are estimated to be $\sim6.6$ for the holes in the germanium quantum well. We demonstrate that all quantum point contacts in the same device have comparable performances, indicating a reliable and reproducible device fabrication process. Thus, our work lays a foundation for investigating multiple forefronts of physics in germanium-based quantum devices that require quantum point contacts as a building block.

cond-mat.mes-hall↗

arXiv2024

Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation

Domain generalized semantic segmentation is an essential computer vision task, for which models only leverage source data to learn the capability of generalized semantic segmentation towards the unseen target domains. Previous works typically address this challenge by global style randomization or feature regularization. In this paper, we argue that given the observation that different local semantic regions perform different visual characteristics from the source domain to the target domain, methods focusing on global operations are hard to capture such regional discrepancies, thus failing to construct domain-invariant representations with the consistency from local to global level. Therefore, we propose the Semantic-Rearrangement-based Multi-Level Alignment (SRMA) to overcome this problem. SRMA first incorporates a Semantic Rearrangement Module (SRM), which conducts semantic region randomization to enhance the diversity of the source domain sufficiently. A Multi-Level Alignment module (MLA) is subsequently proposed with the help of such diversity to establish the global-regional-local consistent domain-invariant representations. By aligning features across randomized samples with domain-neutral knowledge at multiple levels, SRMA provides a more robust way to handle the source-target domain gap. Extensive experiments demonstrate the superiority of SRMA over the current state-of-the-art works on various benchmarks.

cs.CV↗

arXiv2024

The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track

This paper summarizes the music demixing (MDX) track of the Sound Demixing Challenge (SDX'23). We provide a summary of the challenge setup and introduce the task of robust music source separation (MSS), i.e., training MSS models in the presence of errors in the training data. We propose a formalization of the errors that can occur in the design of a training dataset for MSS systems and introduce two new datasets that simulate such errors: SDXDB23_LabelNoise and SDXDB23_Bleeding. We describe the methods that achieved the highest scores in the competition. Moreover, we present a direct comparison with the previous edition of the challenge (the Music Demixing Challenge 2021): the best performing system achieved an improvement of over 1.6dB in signal-to-distortion ratio over the winner of the previous competition, when evaluated on MDXDB21. Besides relying on the signal-to-distortion ratio as objective metric, we also performed a listening test with renowned producers and musicians to study the perceptual quality of the systems and report here the results. Finally, we provide our insights into the organization of the competition and our prospects for future editions.

eess.AS↗

arXiv2024

The Sound Demixing Challenge 2023 $\unicode{x2013}$ Cinematic Demixing Track

This paper summarizes the cinematic demixing (CDX) track of the Sound Demixing Challenge 2023 (SDX'23). We provide a comprehensive summary of the challenge setup, detailing the structure of the competition and the datasets used. Especially, we detail CDXDB23, a new hidden dataset constructed from real movies that was used to rank the submissions. The paper also offers insights into the most successful approaches employed by participants. Compared to the cocktail-fork baseline, the best-performing system trained exclusively on the simulated Divide and Remaster (DnR) dataset achieved an improvement of 1.8 dB in SDR, whereas the top-performing system on the open leaderboard, where any data could be used for training, saw a significant improvement of 5.7 dB. A significant source of this improvement was making the simulated data better match real cinematic audio, which we further investigate in detail.

eess.AS↗

arXiv2024

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Large language models (LLMs), such as GPT3.5, GPT4 and LLAMA2 perform surprisingly well and outperform human experts on many tasks. However, in many domain-specific evaluations, these LLMs often suffer from hallucination problems due to insufficient training of relevant corpus. Furthermore, fine-tuning large models may face problems such as the LLMs are not open source or the construction of high-quality domain instruction is difficult. Therefore, structured knowledge databases such as knowledge graph can better provide domain background knowledge for LLMs and make full use of the reasoning and analysis capabilities of LLMs. In some previous works, LLM was called multiple times to determine whether the current triplet was suitable for inclusion in the subgraph when retrieving subgraphs through a question. Especially for the question that require a multi-hop reasoning path, frequent calls to LLM will consume a lot of computing power. Moreover, when choosing the reasoning path, LLM will be called once for each step, and if one of the steps is selected incorrectly, it will lead to the accumulation of errors in the following steps. In this paper, we integrated and optimized a pipeline for selecting reasoning paths from KG based on LLM, which can reduce the dependency on LLM. In addition, we propose a simple and effective subgraph retrieval method based on chain of thought (CoT) and page rank which can returns the paths most likely to contain the answer. We conduct experiments on three datasets: GenMedGPT-5k [14], WebQuestions [2], and CMCQA [21]. Finally, RoK can demonstrate that using fewer LLM calls can achieve the same results as previous SOTAs models.

cs.CL↗

arXiv2024

Ensuring Safe and High-Quality Outputs: A Guideline Library Approach for Language Models

Large Language Models (LLMs) exhibit impressive capabilities but also present risks such as biased content generation and privacy issues. One of the current alignment techniques includes principle-driven integration, but it faces challenges arising from the imprecision of manually crafted rules and inadequate risk perception in models without safety training. To address these, we introduce Guide-Align, a two-stage approach. Initially, a safety-trained model identifies potential risks and formulates specific guidelines for various inputs, establishing a comprehensive library of guidelines and a model for input-guidelines retrieval. Subsequently, the retrieval model correlates new inputs with relevant guidelines, which guide LLMs in response generation to ensure safe and high-quality outputs, thereby aligning with human values. An additional optional stage involves fine-tuning a model with well-aligned datasets generated through the process implemented in the second stage. Our method customizes guidelines to accommodate diverse inputs, thereby enhancing the fine-grainedness and comprehensiveness of the guideline library. Furthermore, it incorporates safety expertise from a safety-trained LLM through a lightweight retrieval model. We evaluate our approach on three benchmarks, demonstrating significant improvements in LLM security and quality. Notably, our fine-tuned model, Labrador, even at 13 billion parameters, outperforms GPT-3.5-turbo and surpasses GPT-4 in alignment capabilities.

cs.CL↗

arXiv2024

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models

Large language models (LLMs) can achieve highly effective performance on various reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting as demonstrations. However, the reasoning chains of demonstrations generated by LLMs are prone to errors, which can subsequently lead to incorrect reasoning during inference. Furthermore, inappropriate exemplars (overly simplistic or complex), can affect overall performance among varying levels of difficulty. We introduce Iter-CoT (Iterative bootstrapping in Chain-of-Thoughts Prompting), an iterative bootstrapping approach for selecting exemplars and generating reasoning chains. By utilizing iterative bootstrapping, our approach enables LLMs to autonomously rectify errors, resulting in more precise and comprehensive reasoning chains. Simultaneously, our approach selects challenging yet answerable questions accompanied by reasoning chains as exemplars with a moderate level of difficulty, which enhances the LLMs' generalizability across varying levels of difficulty. Experimental results indicate that Iter-CoT exhibits superiority, achieving competitive performance across three distinct reasoning tasks on ten datasets.

cs.CL↗

arXiv2024

Quantum Light Generation based on GaN Microring towards Fully On-chip Source

Integrated quantum light source is increasingly desirable in large-scale quantum information processing.~Despite recent remarkable advances, new material platform is constantly being explored for the fully on-chip integration of quantum light generation, active and passive manipulation, and detection. Here, for the first time, we demonstrate a gallium nitride (GaN) microring based quantum light generation in the telecom C-band, which has potential towards the monolithic integration of quantum light source.~In our demonstration, the GaN microring has a free spectral range of 330 GHz and a near-zero anomalous dispersion region of over 100 nm. The generation of energy-time entangled photon pair is demonstrated with a typical raw two-photon interference visibility of 95.5$\pm$6.5%, which is further configured to generate heralded single photon with a typical heralded second-order auto-correlation $g^{(2)}_{H}(0)$ of 0.045$\pm$0.001. Our results pave the way for developing chip-scale quantum photonic circuit.

quant-ph↗

arXiv2024

Distributed Information-theoretical Secure Protocols for Quantum Key Distribution Networks against Malicious Nodes

Quantum key distribution (QKD) networks are expected to enable information-theoretical secure (ITS) communication over a large-scale network. Most researches on relay-based QKD network assume that all relays or nodes are completely trustworthy. However, the malicious behavior of any single node can undermine security of QKD networks. Current research on QKD networks primarily addresses passive attacks conducted by malicious nodes such as eavesdropping. We suggest a novel paradigm, inspired by distributed systems, to address the active attack by collaborate malicious nodes in QKD networks. Firstly, regarding security, we introduce the ITS distributed authentication scheme, which additionally offers two crucial security properties to QKD networks: identity unforgeability and non-repudiation. Secondly, concerning correctness, our ITS fault-tolerant consensus method, ensures ITS and global consistency with fixed classical broadcast rounds, contrasting with the exponentially message-intensive Byzantine agreement method. Through our simulation, we have shown that our scheme exhibits a significantly lower growth trend in authentication key consumption compared to the original end-to-end pre-shared keys scheme.

quant-ph↗

arXiv2023

SECap: Speech Emotion Captioning with Large Language Model

Speech emotions are crucial in human communication and are extensively used in fields like speech synthesis and natural language understanding. Most prior studies, such as speech emotion recognition, have categorized speech emotions into a fixed set of classes. Yet, emotions expressed in human speech are often complex, and categorizing them into predefined groups can be insufficient to adequately represent speech emotions. On the contrary, describing speech emotions directly by means of natural language may be a more effective approach. Regrettably, there are not many studies available that have focused on this direction. Therefore, this paper proposes a speech emotion captioning framework named SECap, aiming at effectively describing speech emotions using natural language. Owing to the impressive capabilities of large language models in language comprehension and text generation, SECap employs LLaMA as the text decoder to allow the production of coherent speech emotion captions. In addition, SECap leverages HuBERT as the audio encoder to extract general speech features and Q-Former as the Bridge-Net to provide LLaMA with emotion-related speech features. To accomplish this, Q-Former utilizes mutual information learning to disentangle emotion-related speech features and speech contents, while implementing contrastive learning to extract more emotion-related speech features. The results of objective and subjective evaluations demonstrate that: 1) the SECap framework outperforms the HTSAT-BART baseline in all objective evaluations; 2) SECap can generate high-quality speech emotion captions that attain performance on par with human annotators in subjective mean opinion score tests.

cs.SD↗