arXiv subjects

Haiyang Zhang

Publications and source records attributed to Haiyang Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2024

End-to-End Beam Retrieval for Multi-Hop Question Answering

Multi-hop question answering (QA) involves finding multiple relevant passages and step-by-step reasoning to answer complex questions, indicating a retrieve-and-read paradigm. However, previous retrievers were customized for two-hop questions, and most of them were trained separately across different hops, resulting in a lack of supervision over the entire multi-hop retrieval process and leading to poor performance in complicated scenarios beyond two hops. In this work, we introduce Beam Retrieval, an end-to-end beam retrieval framework for multi-hop QA. This approach models the multi-hop retrieval process in an end-to-end manner by jointly optimizing an encoder and two classification heads across all hops. Moreover, Beam Retrieval maintains multiple partial hypotheses of relevant passages at each step, expanding the search space and reducing the risk of missing relevant passages. To establish a complete QA system, we incorporate a supervised reader or a large language model (LLM). Experimental results demonstrate that Beam Retrieval achieves a nearly 50% improvement compared with baselines on challenging MuSiQue-Ans, and it also surpasses all previous retrievers on HotpotQA and achieves 99.9% precision on 2WikiMultiHopQA. Providing high-quality context, Beam Retrieval helps our supervised reader achieve new state-of-the-art performance and substantially improves the few-shot QA performance of LLMs.

cs.CL↗

arXiv2024

Document Set Expansion with Positive-Unlabelled Learning Using Intractable Density Estimation

The Document Set Expansion (DSE) task involves identifying relevant documents from large collections based on a limited set of example documents. Previous research has highlighted Positive and Unlabeled (PU) learning as a promising approach for this task. However, most PU methods rely on the unrealistic assumption of knowing the class prior for positive samples in the collection. To address this limitation, this paper introduces a novel PU learning framework that utilizes intractable density estimation models. Experiments conducted on PubMed and Covid datasets in a transductive setting showcase the effectiveness of the proposed method for DSE. Code is available from https://github.com/Beautifuldog01/Document-set-expansion-puDE.

cs.IR↗

arXiv2024

Prompt Engineering for Healthcare: Methodologies and Applications

Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important in the healthcare domain. However, there is a lack of comprehensive reviews specifically focusing on prompt engineering in the medical field. This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. First, we will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications such as question-answering systems, text summarization, and machine translation. With the continuous improvement of general large language models, the importance of prompt engineering in the healthcare domain is becoming increasingly prominent. The aim of this article is to provide useful resources and bridges for healthcare natural language processing researchers to better explore the application of prompt engineering in this field. We hope that this review can provide new ideas and inspire for research and application in medical natural language processing.

cs.AI↗

arXiv2024

Hierarchical Query Classification in E-commerce Search

E-commerce platforms typically store and structure product information and search data in a hierarchy. Efficiently categorizing user search queries into a similar hierarchical structure is paramount in enhancing user experience on e-commerce platforms as well as news curation and academic research. The significance of this task is amplified when dealing with sensitive query categorization or critical information dissemination, where inaccuracies can lead to considerable negative impacts. The inherent complexity of hierarchical query classification is compounded by two primary challenges: (1) the pronounced class imbalance that skews towards dominant categories, and (2) the inherent brevity and ambiguity of search queries that hinder accurate classification. To address these challenges, we introduce a novel framework that leverages hierarchical information through (i) enhanced representation learning that utilizes the contrastive loss to discern fine-grained instance relationships within the hierarchy, called ''instance hierarchy'', and (ii) a nuanced hierarchical classification loss that attends to the intrinsic label taxonomy, named ''label hierarchy''. Additionally, based on our observation that certain unlabeled queries share typographical similarities with labeled queries, we propose a neighborhood-aware sampling technique to intelligently select these unlabeled queries to boost the classification performance. Extensive experiments demonstrate that our proposed method is better than state-of-the-art (SOTA) on the proprietary Amazon dataset, and comparable to SOTA on the public datasets of Web of Science and RCV1-V2. These results underscore the efficacy of our proposed solution, and pave the path toward the next generation of hierarchy-aware query classification systems.

cs.IR↗

arXiv2024

Label-free Node Classification on Graphs with Large Language Models (LLMS)

In recent years, there have been remarkable advancements in node classification achieved by Graph Neural Networks (GNNs). However, they necessitate abundant high-quality labels to ensure promising performance. In contrast, Large Language Models (LLMs) exhibit impressive zero-shot proficiency on text-attributed graphs. Yet, they face challenges in efficiently processing structural data and suffer from high inference costs. In light of these observations, this work introduces a label-free node classification on graphs with LLMs pipeline, LLM-GNN. It amalgamates the strengths of both GNNs and LLMs while mitigating their limitations. Specifically, LLMs are leveraged to annotate a small portion of nodes and then GNNs are trained on LLMs' annotations to make predictions for the remaining large portion of nodes. The implementation of LLM-GNN faces a unique challenge: how can we actively select nodes for LLMs to annotate and consequently enhance the GNN training? How can we leverage LLMs to obtain annotations of high quality, representativeness, and diversity, thereby enhancing GNN performance with less cost? To tackle this challenge, we develop an annotation quality heuristic and leverage the confidence scores derived from LLMs to advanced node selection. Comprehensive experimental results validate the effectiveness of LLM-GNN. In particular, LLM-GNN can achieve an accuracy of 74.9% on a vast-scale dataset \products with a cost less than 1 dollar.

cs.LG↗

arXiv2024

Document Set Expansion with Positive-Unlabeled Learning: A Density Estimation-based Approach

Document set expansion aims to identify relevant documents from a large collection based on a small set of documents that are on a fine-grained topic. Previous work shows that PU learning is a promising method for this task. However, some serious issues remain unresolved, i.e. typical challenges that PU methods suffer such as unknown class prior and imbalanced data, and the need for transductive experimental settings. In this paper, we propose a novel PU learning framework based on density estimation, called puDE, that can handle the above issues. The advantage of puDE is that it neither constrained to the SCAR assumption and nor require any class prior knowledge. We demonstrate the effectiveness of the proposed method using a series of real-world datasets and conclude that our method is a better alternative for the DSE task.

cs.LG↗

arXiv2024

Building confidence in state-of-the-art ab initio calculations of the density virial coefficients of B and C of helium-4: Part 1. Indirect evaluation methods using the results of SPRIGT

In this work, we propose indirect evaluation methods to check the accuracy and the uncertainty of ab initio calculated virial coefficients. To this end, we have used single-pressure refractive-index gas thermometry (SPRIGT) to estimate the impact of the second density virial coefficient B of helium-4 on temperature measurements between 5 K to 25 K. Our results, in good agreement with values of B obtained from recent ab initio calculations by Czachorowski et al. [Phys. Rev. A 102, 042810 (2020)], suggest uncertainties u(B) estimated previously by other authors were too conservative. Concerning the value of the third density virial coefficient C of helium-4, calculated ab initio by Garberoglio et al. [J. Chem. Phys. 134, 134106 (2011)], our results suggest their uncertainty u(C) is between 1.5 and 10.2 times too high.

physics.ins-det↗

arXiv2024

Building confidence in state-of-the-art ab initio calculations of the density virial coefficients B and C of helium-4: Part 2. Direct evaluation by high accuracy experimental data using RIGT

In our previous work [1], using indirect evaluation methods we concluded that the uncertainties of the second and the third density virial coefficient, B and C, of helium-4 at 5 K calculated by various authors had been overestimated. To check the reliability of these values and appraisal of uncertainties from ab initio calculations still further, a refractive-index gas thermometry method was developed to determine simultaneously thermodynamic temperatures and density virial coefficients. Using this technique, high accuracy experimental values of B and C of helium-4 and new values of T-T90 were obtained for the range 5 K to 25 K. A direct comparison with the ab initio calculation density virial coefficients was made. Results support the conclusion of our previous work, i.e., the ab initio calculation uncertainties u(B) [J. Chem. Phys. 136, 224303 (2012)] and u(C) [J. Chem. Phys. 134, 134106 (2011)] of helium-4 were overestimated by a factor of severalfold.

physics.ins-det↗

arXiv2023

A Novel Field-Free SOT Magnetic Tunnel Junction With Local VCMA-Induced Switching

By integrating the local voltage-controlled magnetic anisotropy (VCMA) effect, Dzyaloshinskii-Moriya interaction (DMI) effect, and spin-orbit torque (SOT) effect, we propose a novel device structure for field-free magnetic tunnel junction (MTJ). Micromagnetic simulation shows that the device utilizes the chiral symmetry breaking caused by the DMI effect to induce a non-collinear spin texture under the influence of SOT current. This, combined with the perpendicular magnetic anisotropy (PMA) gradient generated by the local VCMA effect, enables deterministic switching of the MTJ state without an external field. The impact of variations in DMI strength and PMA gradient on the magnetization dynamics is analyzed.

eess.SP↗

arXiv2023

Near-Field Wideband Secure Communications: An Analog Beamfocusing Approach

In the rapidly advancing landscape of 6G, characterized by ultra-high-speed wideband transmission in millimeter-wave and terahertz bands, our paper addresses the pivotal task of enhancing physical layer security (PLS) within near-field wideband communications. We introduce true-time delayer (TTD)-incorporated analog beamfocusing techniques designed to address the interplay between near-field propagation and wideband beamsplit, an uncharted domain in existing literature. Our approach to maximizing secrecy rates involves formulating an optimization problem for joint power allocation and analog beamformer design, employing a two-stage process encompassing a semi-digital solution and analog approximation. This problem is efficiently solved through a combination of alternating optimization, fractional programming, and block successive upper-bound minimization techniques. Additionally, we present a low-complexity beamsplit-aware beamfocusing strategy, capitalizing on geometric insights from near-field wideband propagation, which can also serve as a robust initial value for the optimization-based approach. Numerical results substantiate the efficacy of the proposed methods, clearly demonstrating their superiority over TTD-free approaches in fortifying wideband PLS, as well as the advantageous secrecy energy efficiency achieved by leveraging low-cost analog devices.

eess.SP↗

arXiv2023

Channel Estimation with Dynamic Metasurface Antennas via Model-Based Learning

Dynamic Metasurface Antenna (DMA) is a cutting-edge antenna technology offering scalable and sustainable solutions for large antenna arrays. The effectiveness of DMAs stems from their inherent configurable analog signal processing capabilities, which facilitate cost-limited implementations. However, when DMAs are used in multiple input multiple output (MIMO) communication systems, they pose challenges in channel estimation due to their analog compression. In this paper, we propose two model-based learning methods to overcome this challenge. Our approach starts by casting channel estimation as a compressed sensing problem. Here, the sensing matrix is formed using a random DMA weighting matrix combined with a spatial gridding dictionary. We then employ the learned iterative shrinkage and thresholding algorithm (LISTA) to recover the sparse channel parameters. LISTA unfolds the iterative shrinkage and thresholding algorithm into a neural network and trains the neural network into a highly efficient channel estimator fitting with the previous channel. As the sensing matrix is crucial to the accuracy of LISTA recovery, we introduce another data-aided method, LISTA-sensing matrix optimization (LISTA-SMO), to jointly optimize the sensing matrix. LISTA-SMO takes LISTA as a backbone and embeds the sensing matrix optimization layers in LISTA's neural network, allowing for the optimization of the sensing matrix along with the training of LISTA. Furthermore, we propose a self-supervised learning technique to tackle the difficulty of acquiring noise-free data. Our numerical results demonstrate that LISTA outperforms traditional sparse recovery methods regarding channel estimation accuracy and efficiency. Besides, LISTA-SMO achieves better channel accuracy than LISTA, demonstrating the effectiveness in optimizing the sensing matrix.

cs.IT↗

arXiv2023

Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generation

Modeling customer shopping intentions is a crucial task for e-commerce, as it directly impacts user experience and engagement. Thus, accurately understanding customer preferences is essential for providing personalized recommendations. Session-based recommendation, which utilizes customer session data to predict their next interaction, has become increasingly popular. However, existing session datasets have limitations in terms of item attributes, user diversity, and dataset scale. As a result, they cannot comprehensively capture the spectrum of user behaviors and preferences. To bridge this gap, we present the Amazon Multilingual Multi-locale Shopping Session Dataset, namely Amazon-M2. It is the first multilingual dataset consisting of millions of user sessions from six different locales, where the major languages of products are English, German, Japanese, French, Italian, and Spanish. Remarkably, the dataset can help us enhance personalization and understanding of user preferences, which can benefit various existing tasks as well as enable new tasks. To test the potential of the dataset, we introduce three tasks in this work: (1) next-product recommendation, (2) next-product recommendation with domain shifts, and (3) next-product title generation. With the above tasks, we benchmark a range of algorithms on our proposed dataset, drawing new insights for further research and practice. In addition, based on the proposed dataset and tasks, we hosted a competition in the KDD CUP 2023 and have attracted thousands of users and submissions. The winning solutions and the associated workshop can be accessed at our website https://kddcup23.github.io/.

cs.IR↗

arXiv2023

Hybrid Reconfigurable Intelligent Metasurfaces: Enabling Simultaneous Tunable Reflections and Sensing for 6G Wireless Communications

The latest discussions on the upcoming sixth Generation (6G) of wireless communications are envisioning future networks as a unified communications, sensing, and computing platform. The recently conceived concept of the smart radio environment, enabled by Reconfigurable Intelligent Surfaces (RISs), contributes towards this vision offering programmable propagation of information-bearing signals. Typical RIS implementations include metasurfaces with almost passive unit elements capable of reflecting their incident waves in controllable ways. However, this solely reflective operation induces significant challenges for the RIS optimization from the wireless network orchestrator. For example, RISs lack information to locally tune their reflection pattern, which can only be acquired by other network entities, and then shared with the RIS controller. Furthermore, channel estimation, which is essential for coherent RIS-empowered communications, is challenging with the available RIS designs. This article reviews the emerging concept of Hybrid reflecting and sensing RISs (HRISs), which enables metasurfaces to reflect the impinging signal in a controllable manner, while simultaneously sensing a portion of it. The sensing capability of HRISs facilitates various network management functionalities, including channel parameter estimation and localization, while giving rise to potentially computationally autonomous and self-configuring metasurfaces. We discuss a hardware design for HRISs and detail a full-wave electromagnetic proof of concept. The distinctive properties of HRISs, in comparison to their solely reflective counterparts, are highlighted and a simulation study evaluating their capability for performing full and parametric channel estimation is presented. Future research challenges and opportunities arising from the HRIS concept are also included.

cs.IT↗

arXiv2023

A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning

An efficient urban bus control system has the potential to significantly reduce travel delays and streamline the allocation of transportation resources, thereby offering enhanced and user-friendly transit services to passengers. However, bus operation efficiency can be impacted by bus bunching. This problem is notably exacerbated when the bus system operates along a signalized corridor with unpredictable travel demand. To mitigate this challenge, we introduce a multi-strategy fusion approach for the longitudinal control of connected and automated buses. The approach is driven by a physics-informed deep reinforcement learning (DRL) algorithm and takes into account a variety of traffic conditions along urban signalized corridors. Taking advantage of connected and autonomous vehicle (CAV) technology, the proposed approach can leverage real-time information regarding bus operating conditions and road traffic environment. By integrating the aforementioned information into the DRL-based bus control framework, our designed physics-informed DRL state fusion approach and reward function efficiently embed prior physics and leverage the merits of equilibrium and consensus concepts from control theory. This integration enables the framework to learn and adapt multiple control strategies to effectively manage complex traffic conditions and fluctuating passenger demands. Three control variables, i.e., dwell time at stops, speed between stations, and signal priority, are formulated to minimize travel duration and ensure bus stability with the aim of avoiding bus bunching. We present simulation results to validate the effectiveness of the proposed approach, underlining its superior performance when subjected to sensitivity analysis, specifically considering factors such as traffic volume, desired speed, and traffic signal conditions.

eess.SY↗

arXiv2023

Machine learning for predicting fatigue properties of additively manufactured materials

Fatigue properties of additively manufactured (AM) materials depend on many factors such as AM processing parameter, microstructure, residual stress, surface roughness, porosities, post-treatments, etc. Their evaluation inevitably requires these factors combined as many as possible, thus resulting in low efficiency and high cost. In recent years, their assessment by leveraging the power of machine learning (ML) has gained increasing attentions. Here, we present a comprehensive overview on the state-of-the-art progress of applying ML strategies to predict fatigue properties of AM materials, as well as their dependence on AM processing and post-processing parameters such as laser power, scanning speed, layer height, hatch distance, built direction, post-heat temperature, etc. A few attempts in employing feedforward neural network (FNN), convolutional neural network (CNN), adaptive network-based fuzzy system (ANFS), support vector machine (SVM) and random forest (RF) to predict fatigue life and RF to predict fatigue crack growth rate are summarized. The ML models for predicting AM materials' fatigue properties are found intrinsically similar to the commonly used ones, but are modified to involve AM features. Finally, an outlook for challenges (i.e., small dataset, multifarious features, overfitting, low interpretability, unable extension from AM material data to structure life) and potential solutions for the ML prediction of AM materials' fatigue properties is provided.

cond-mat.mtrl-sci↗

arXiv2023

Hardware Implementation of Task-based Quantization in Multi-user Signal Recovery

Quantization plays a critical role in digital signal processing systems, allowing the representation of continuous amplitude signals with a finite number of bits. However, accurately representing signals requires a large number of quantization bits, which causes severe cost, power consumption, and memory burden. A promising way to address this issue is task-based quantization. By exploiting the task information for the overall system design, task-based quantization can achieve satisfying performance with low quantization costs. In this work, we apply task-based quantization to multi-user signal recovery and present a hardware prototype implementation. The prototype consists of a tailored configurable combining board, and a software-based processing and demonstration system. Through experiments, we verify that with proper design, the task-based quantization achieves a reduction of 25 fold in memory by reducing from 16 receivers with 16 bits each to 2 receivers with 5 bits each, without compromising signal recovery performance.

eess.SP↗

arXiv2022

Near-Field Sparse Channel Representation and Estimation in 6G Wireless Communications

The employment of extremely large antenna arrays and high-frequency signaling makes future 6G wireless communications likely to operate in the near-field region. In this case, the spherical wave assumption which takes into account both the user angle and distance is more accurate than the conventional planar one that is only related to the user angle. Therefore, the conventional planar wave based far-field channel model as well as its associated estimation algorithms needs to be reconsidered. Here we first propose a distance-parameterized angular-domain sparse model to represent the near-field channel. In this model, the user distance is included in the dictionary as an unknown parameter, so that the number of dictionary columns depends only on the angular space division. This is different from the existing polar-domain near-field channel model where the dictionary is constructed on an angle-distance two-dimensional (2D) space. Next, based on this model, joint dictionary learning and sparse recovery based channel estimation methods are proposed for both line of sight (LoS) and multi-path settings. To further demonstrate the effectiveness of the suggested algorithms, recovery conditions and computational complexity are studied. Our analysis shows that with the decrease of distance estimation error in the dictionary, the angular-domain sparse vector can be exactly recovered after a few iterations. The high storage burden and dictionary coherence issues that arise in the polar-domain 2D representation are well addressed. Finally, simulations in multi-user communication scenarios support the superiority of the proposed near-field channel sparse representation and estimation over the existing polar-domain method in channel estimation error.

eess.SP↗

arXiv2022

6G Wireless Communications: From Far-field Beam Steering to Near-field Beam Focusing

6G networks will be required to support higher data rates, improved energy efficiency, lower latency, and more diverse users compared with 5G systems. To meet these requirements, electrically extremely large-scale antenna arrays are envisioned to be key physical-layer technologies. As a consequence, it is expected that some portion of future 6G wireless communications may take place in the radiating near-field (Fresnel) region, in addition to the far-field operation as in current wireless technologies. In this article, we discuss the opportunities and challenges that arise in radiating near-field communications. We begin by discussing the key physical characteristics of near-field communications, where the standard plane-wave propagation assumption no longer holds, and clarifying its implication on the modelling of wireless channels. Then, we elaborate on the ability to leverage spherical wavefronts via beam focusing, highlighting its advantages for 6G systems. We point out several appealing application scenarios which, with proper design, can benefit from near-field operation, including interference mitigation in multi-user communications, accurate localization and focused sensing, as well as wireless power transfer with minimal energy pollution. We conclude by discussing some of the design challenges and research directions that are yet to be explored to fully harness the potential of near-field operation.

eess.SP↗