arXiv subjects

Shuai Zhang

Publications and source records attributed to Shuai Zhang.

At least 271 records · Page 15Linked to original sources

arXiv2023

Efficient Beam Manipulation with Phase Symmetry Operations on Modulated Metasurfaces

Beam manipulation is of paramount importance in wave engineering, enabling diverse beam shapes like pencil beams, flat-top beams, and isoflux beams to cater to various application missions. Among the beams, shaping flat-top and isoflux beams remains challenging with the traditional synthesis approaches that mainly rely on optimization algorithms. Here, we develop modulated metasurfaces to efficiently generate flat-top and isoflux beams from the first principle of field superposition with negligible optimizations, by performing phase symmetry operations. The theoretical analysis not only facilitates the shaping of 1D and 2D flat-top and isoflux beams but also exhibits controllable beamwidths. Experimental validation confirms the efficacy of the phase symmetry operations in generating flat-top beams with adjustable beamwidths. The concept of phase symmetry operations can be extended to other vectorial components, offering potential applications for the manipulation of various wave types such as acoustic waves, water surface waves, and beyond, thereby advancing related applications.

physics.optics↗

arXiv2023

Reconciling results of 2019 and 2020 stellar occultations on Pluto's atmosphere. New constraints from both the 5 September 2019 event and consistency analysis

A stellar occultation by Pluto on 5 September 2019 yielded positive detections at two separate stations. Using an approach consistent with comparable studies, we derived a surface pressure of $11.478 \pm 0.55~\mathrm{μbar}$ for Pluto's atmosphere from the observations of this event. In addition, to avoid potential method inconsistancies highlighted by Sicardy et al. when comparing with historical pressure measurements, we reanalyzed the data by 15 August 2018 and 17 July 2019 events, respectively. All the new measurements provide a bridge between the two different perspectives on the pressure variation since 2015: a rapid pressure drop from previous studies of the 15 August 2018 and 17 July 2019 events and a plateau phase from that of the 6 June 2020 event. The pressure measurement from the 5 September 2019 event aligns with those from 2016, 2018, and 2020, supporting the latter perspective. While the measurements from the 4 June 2011 and 17 July 2019 events suggest probable V-shaped pressure variations unaccounted for by the volatile transport model (VTM) from Meza et al., the VTM remains applicable on average. And, the validity of the V-shaped variations is debatable due to the stellar faintness of the 4 June 2011 event and the grazing single-chord geometry of the 17 July 2019 event. To reveal and understand all significant pressure variations of Pluto's atmosphere, it is essential to provide constraints on both short-term and long-term evolutions of the interacting atmosphere and surface by continuous pressure monitoring through occultation observations, whenever possible, complemented by frequent spectroscopy and photometry of the surface.

astro-ph.EP↗

arXiv2023

Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

In this paper, we propose a Disentangled Counterfactual Learning~(DCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects' physics commonsense based on both video and audio input, with the main challenge is how to imitate the reasoning ability of humans. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and lacking causal reasoning ability in models impedes the progress of implicit physical knowledge inferring. To address these issues, our proposed DCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space by the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. Our proposed method is a plug-and-play module that can be incorporated into any baseline. In experiments, we show that our proposed method improves baseline methods and achieves state-of-the-art performance. Our source code is available at https://github.com/Andy20178/DCL.

cs.CV↗

arXiv2023

On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$-Greedy Exploration

This paper provides a theoretical understanding of Deep Q-Network (DQN) with the $\varepsilon$-greedy exploration in deep reinforcement learning. Despite the tremendous empirical achievement of the DQN, its theoretical characterization remains underexplored. First, the exploration strategy is either impractical or ignored in the existing analysis. Second, in contrast to conventional Q-learning algorithms, the DQN employs the target network and experience replay to acquire an unbiased estimation of the mean-square Bellman error (MSBE) utilized in training the Q-network. However, the existing theoretical analysis of DQNs lacks convergence analysis or bypasses the technical challenges by deploying a significantly overparameterized neural network, which is not computationally efficient. This paper provides the first theoretical convergence and sample complexity analysis of the practical setting of DQNs with $ε$-greedy policy. We prove an iterative procedure with decaying $ε$ converges to the optimal Q-value function geometrically. Moreover, a higher level of $ε$ values enlarges the region of convergence but slows down the convergence, while the opposite holds for a lower level of $ε$ values. Experiments justify our established theoretical insights on DQNs.

cs.LG↗

arXiv2023

Lightweight In-Context Tuning for Multimodal Unified Models

In-context learning (ICL) involves reasoning from given contextual examples. As more modalities comes, this procedure is becoming more challenging as the interleaved input modalities convolutes the understanding process. This is exemplified by the observation that multimodal models often struggle to effectively extrapolate from contextual examples to perform ICL. To address these challenges, we introduce MultiModal In-conteXt Tuning (M$^2$IXT), a lightweight module to enhance the ICL capabilities of multimodal unified models. The proposed M$^2$IXT module perceives an expandable context window to incorporate various labeled examples of multiple modalities (e.g., text, image, and coordinates). It can be prepended to various multimodal unified models (e.g., OFA, Unival, LLaVA) of different architectures and trained via a mixed-tasks strategy to enable rapid few-shot adaption on multiple tasks and datasets. When tuned on as little as 50K multimodal data, M$^2$IXT can boost the few-shot ICL performance significantly (e.g., 18\% relative increase for OFA), and obtained state-of-the-art results across an array of tasks including visual question answering, image captioning, visual grounding, and visual entailment, while being considerably small in terms of model parameters (e.g., $\sim$$20\times$ smaller than Flamingo or MMICL), highlighting the flexibility and effectiveness of M$^2$IXT as a multimodal in-context learner.

cs.CV↗

arXiv2023

Aerial Base Stations: Practical Considerations for Power Consumption and Service Time

Aerial base stations (ABSs) have emerged as a promising solution to meet the high traffic demands of future wireless networks. Nevertheless, their practical implementation requires efficient utilization of limited payload and onboard energy. Understanding the power consumption streams, such as mechanical and communication power, and their relationship to the payload is crucial for analyzing its feasibility. Specifically, we focus on rotary-wing drones (RWDs), fixed-wing drones (FWDs), and high-altitude platforms (HAPs), analyzing their energy consumption models and key performance metrics such as power consumption, energy harvested-to-consumption ratio, and service time with varying wingspans, battery capacities, and regions. Our findings indicate that FWDs have longer service times and HAPs have energy harvested-to-consumption ratios greater than one, indicating theoretically infinite service time, especially when deployed in near-equator regions or have a large wingspan. Additionally, we investigate the case study of RWD-BS deployment, assessing aerial network dimensioning aspects such as ABS coverage radius based on altitude, environment, and frequency of operation. Our findings provide valuable insights for researchers and telecom operators, facilitating effective cost planning by determining the number of ABSs and backup batteries required for uninterrupted operations.

cs.NI↗

arXiv2023

Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition

This work proposes POMP, a prompt pre-training method for vision-language models. Being memory and computation efficient, POMP enables the learned prompt to condense semantic information for a rich set of visual concepts with over twenty-thousand classes. Once pre-trained, the prompt with a strong transferable ability can be directly plugged into a variety of visual recognition tasks including image classification, semantic segmentation, and object detection, to boost recognition performances in a zero-shot manner. Empirical evaluation shows that POMP achieves state-of-the-art performances on 21 datasets, e.g., 67.0% average accuracy on 10 classification datasets (+3.1% compared to CoOp) and 84.4 hIoU on open-vocabulary Pascal VOC segmentation (+6.9 compared to ZSSeg). Our code is available at https://github.com/amazon-science/prompt-pretraining.

cs.CV↗

arXiv2023

Visualizing moiré ferroelectricity via plasmons and nano-photocurrent in graphene/twisted-WSe2 structures

Ferroelectricity, a spontaneous and reversible electric polarization, is found in certain classes of van der Waals (vdW) material heterostructures. The discovery of ferroelectricity in twisted vdW layers provides new opportunities to engineer spatially dependent electric and optical properties associated with the configuration of moiré superlattice domains and the network of domain walls. Here, we employ near-field infrared nano-imaging and nano-photocurrent measurements to study ferroelectricity in minimally twisted WSe2. The ferroelectric domains are visualized through the imaging of the plasmonic response in a graphene monolayer adjacent to the moiré WSe2 bilayers. Specifically, we find that the ferroelectric polarization in moiré domains is imprinted on the plasmonic response of the graphene. Complementary nano-photocurrent measurements demonstrate that the optoelectronic properties of graphene are also modulated by the proximal ferroelectric domains. Our approach represents an alternative strategy for studying moiré ferroelectricity at native length scales and opens promising prospects for (opto)electronic devices.

cond-mat.mes-hall↗

arXiv2023

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

Considerable research efforts have been devoted to ensuring that large language models (LLMs) align with human values and generate safe text. However, an excessive focus on sensitivity to certain topics can compromise the model's robustness in following instructions, thereby impacting its overall performance in completing tasks. Previous benchmarks for jailbreaking LLMs have primarily focused on evaluating the safety of the models without considering their robustness. In this paper, we propose a benchmark that assesses both the safety and robustness of LLMs, emphasizing the need for a balanced approach. To comprehensively study text safety and output robustness, we introduce a latent jailbreak prompt dataset, each involving malicious instruction embedding. Specifically, we instruct the model to complete a regular task, such as translation, with the text to be translated containing malicious instructions. To further analyze safety and robustness, we design a hierarchical annotation framework. We present a systematic analysis of the safety and robustness of LLMs regarding the position of explicit normal instructions, word replacements (verbs in explicit normal instructions, target groups in malicious instructions, cue words for explicit normal instructions), and instruction replacements (different explicit normal instructions). Our results demonstrate that current LLMs not only prioritize certain instruction verbs but also exhibit varying jailbreak rates for different instruction verbs in explicit normal instructions. Code and data are available at https://github.com/qiuhuachuan/latent-jailbreak.

cs.CL↗

arXiv2023

InfeRE: Step-by-Step Regex Generation via Chain of Inference

Automatically generating regular expressions (abbrev. regexes) from natural language description (NL2RE) has been an emerging research area. Prior studies treat regex as a linear sequence of tokens and generate the final expressions autoregressively in a single pass. They did not take into account the step-by-step internal text-matching processes behind the final results. This significantly hinders the efficacy and interpretability of regex generation by neural language models. In this paper, we propose a new paradigm called InfeRE, which decomposes the generation of regexes into chains of step-by-step inference. To enhance the robustness, we introduce a self-consistency decoding mechanism that ensembles multiple outputs sampled from different models. We evaluate InfeRE on two publicly available datasets, NL-RX-Turk and KB13, and compare the results with state-of-the-art approaches and the popular tree-based generation approach TRANX. Experimental results show that InfeRE substantially outperforms previous baselines, yielding 16.3% and 14.7% improvement in DFA@5 accuracy on two datasets, respectively. Particularly, InfeRE outperforms the popular tree-based generation approach by 18.1% and 11.3% on both datasets, respectively, in terms of DFA@5 accuracy.

cs.AI↗

arXiv2023

A Benchmark for Understanding Dialogue Safety in Mental Health Support

Dialogue safety remains a pervasive challenge in open-domain human-machine interaction. Existing approaches propose distinctive dialogue safety taxonomies and datasets for detecting explicitly harmful responses. However, these taxonomies may not be suitable for analyzing response safety in mental health support. In real-world interactions, a model response deemed acceptable in casual conversations might have a negligible positive impact on users seeking mental health support. To address these limitations, this paper aims to develop a theoretically and factually grounded taxonomy that prioritizes the positive impact on help-seekers. Additionally, we create a benchmark corpus with fine-grained labels for each dialogue session to facilitate further research. We analyze the dataset using popular language models, including BERT-base, RoBERTa-large, and ChatGPT, to detect and understand unsafe responses within the context of mental health support. Our study reveals that ChatGPT struggles to detect safety categories with detailed safety definitions in a zero- and few-shot paradigm, whereas the fine-tuned model proves to be more suitable. The developed dataset and findings serve as valuable benchmarks for advancing research on dialogue safety in mental health support, with significant implications for improving the design and deployment of conversation agents in real-world applications. We release our code and data here: https://github.com/qiuhuachuan/DialogueSafety.

cs.CL↗

arXiv2023

MAS: Towards Resource-Efficient Federated Multiple-Task Learning

Federated learning (FL) is an emerging distributed machine learning method that empowers in-situ model training on decentralized edge devices. However, multiple simultaneous FL tasks could overload resource-constrained devices. In this work, we propose the first FL system to effectively coordinate and train multiple simultaneous FL tasks. We first formalize the problem of training simultaneous FL tasks. Then, we present our new approach, MAS (Merge and Split), to optimize the performance of training multiple simultaneous FL tasks. MAS starts by merging FL tasks into an all-in-one FL task with a multi-task architecture. After training for a few rounds, MAS splits the all-in-one FL task into two or more FL tasks by using the affinities among tasks measured during the all-in-one training. It then continues training each split of FL tasks based on model parameters from the all-in-one training. Extensive experiments demonstrate that MAS outperforms other methods while reducing training time by 2x and reducing energy consumption by 40%. We hope this work will inspire the community to further study and optimize training simultaneous FL tasks.

cs.LG↗

arXiv2023

A refined numerical investigation of a large equivalent shallow-depth underwater explosion

The large equivalent shallow-depth explosion problem is very significant in the field of naval architecture and ocean engineering, as such explosions can be used to attack and demolish ships and anti-ship missiles. In the current work, a refined numerical study of the flow-field characteristics of a large equivalent shallow-depth explosion is carried out using a self-developed Eulerian finite element solver. Firstly, the numerical model is validated against theoretical results and a small equivalent explosion test in a tank. The numerical results are found to agree well with the theoretical and experimental results. In the next step, the cavitation cut-off effect is added to the underwater explosion model, and the cavitation phenomenon is quantitatively analyzed through the flow-field pressure. In addition, the dynamic characteristics of the bubble and water hump under various initial conditions for different stand-off parameters are analyzed. The effect of gravity on these physical processes is also discussed. The bubble pulsation period, taking into account the free surface effect, is then quantitatively studied and compared with Cole's experimental formula for an underwater explosion. Overall, when the stand-off parameter > 2, the influence of the free surface on the empirical period of the bubble is not significant. Our investigation provides broad insights into shallow-depth underwater explosions from theoretical, experimental, and numerical perspectives.

physics.flu-dyn↗

arXiv2023

Co-design Hardware and Algorithm for Vector Search

Vector search has emerged as the foundation for large-scale information retrieval and machine learning systems, with search engines like Google and Bing processing tens of thousands of queries per second on petabyte-scale document datasets by evaluating vector similarities between encoded query texts and web documents. As performance demands for vector search systems surge, accelerated hardware offers a promising solution in the post-Moore's Law era. We introduce \textit{FANNS}, an end-to-end and scalable vector search framework on FPGAs. Given a user-provided recall requirement on a dataset and a hardware resource budget, \textit{FANNS} automatically co-designs hardware and algorithm, subsequently generating the corresponding accelerator. The framework also supports scale-out by incorporating a hardware TCP/IP stack in the accelerator. \textit{FANNS} attains up to 23.0$\times$ and 37.2$\times$ speedup compared to FPGA and CPU baselines, respectively, and demonstrates superior scalability to GPUs, achieving 5.5$\times$ and 7.6$\times$ speedup in median and 95\textsuperscript{th} percentile (P95) latency within an eight-accelerator configuration. The remarkable performance of \textit{FANNS} lays a robust groundwork for future FPGA integration in data centers and AI supercomputers.

cs.LG↗

arXiv2023

Toward an accurate equation of state and B1-B2 phase boundary for magnesium oxide to TPa pressures and eV temperatures

By applying auxiliary-field quantum Monte Carlo, we calculate the equation of state (EOS) and B1-B2 phase transition of magnesium oxide (MgO) up to 1 TPa. The results agree with available experimental data at low pressures and are used to benchmark the performance of various exchange-correlation functionals in density functional theory calculations. We determine PBEsol is an optimal choice for the exchange-correlation functional and perform extensive phonon and quantum molecular-dynamics calculations to obtain the thermal EOS. Our results provide a preliminary reference for the EOS and B1-B2 phase boundary of MgO from zero up to 10,500 K.

cond-mat.mtrl-sci↗

arXiv2023

Non-Convex Optimizations for Machine Learning with Theoretical Guarantee: Robust Matrix Completion and Neural Network Learning

Despite the recent development in machine learning, most learning systems are still under the concept of "black box", where the performance cannot be understood and derived. With the rise of safety and privacy concerns in public, designing an explainable learning system has become a new trend in machine learning. In general, many machine learning problems are formulated as minimizing (or maximizing) some loss function. Since real data are most likely generated from non-linear models, the loss function is non-convex in general. Unlike the convex optimization problem, gradient descent algorithms will be trapped in spurious local minima in solving non-convex optimization. Therefore, it is challenging to provide explainable algorithms when studying non-convex optimization problems. In this thesis, two popular non-convex problems are studied: (1) low-rank matrix completion and (2) neural network learning.

cs.LG↗

arXiv2023

Understanding Client Reactions in Online Mental Health Counseling

Communication success relies heavily on reading participants' reactions. Such feedback is especially important for mental health counselors, who must carefully consider the client's progress and adjust their approach accordingly. However, previous NLP research on counseling has mainly focused on studying counselors' intervention strategies rather than their clients' reactions to the intervention. This work aims to fill this gap by developing a theoretically grounded annotation framework that encompasses counselors' strategies and client reaction behaviors. The framework has been tested against a large-scale, high-quality text-based counseling dataset we collected over the past two years from an online welfare counseling platform. Our study shows how clients react to counselors' strategies, how such reactions affect the final counseling outcomes, and how counselors can adjust their strategies in response to these reactions. We also demonstrate that this study can help counselors automatically predict their clients' states.

cs.CL↗

arXiv2023

A Semi-Paired Approach For Label-to-Image Translation

Data efficiency, or the ability to generalize from a few labeled data, remains a major challenge in deep learning. Semi-supervised learning has thrived in traditional recognition tasks alleviating the need for large amounts of labeled data, yet it remains understudied in image-to-image translation (I2I) tasks. In this work, we introduce the first semi-supervised (semi-paired) framework for label-to-image translation, a challenging subtask of I2I which generates photorealistic images from semantic label maps. In the semi-paired setting, the model has access to a small set of paired data and a larger set of unpaired images and labels. Instead of using geometrical transformations as a pretext task like previous works, we leverage an input reconstruction task by exploiting the conditional discriminator on the paired data as a reverse generator. We propose a training algorithm for this shared network, and we present a rare classes sampling algorithm to focus on under-represented classes. Experiments on 3 standard benchmarks show that the proposed model outperforms state-of-the-art unsupervised and semi-supervised approaches, as well as some fully supervised approaches while using a much smaller number of paired samples.

cs.CV↗