arXiv subjects

Shuai Zhang

Publications and source records attributed to Shuai Zhang.

At least 199 records · Page 11Linked to original sources

arXiv2025

Imitating from auxiliary imperfect demonstrations via Adversarial Density Weighted Regression

We propose a novel one-step supervised imitation learning (IL) framework called Adversarial Density Regression (ADR). This IL framework aims to correct the policy learned on unknown-quality to match the expert distribution by utilizing demonstrations, without relying on the Bellman operator. Specifically, ADR addresses several limitations in previous IL algorithms: First, most IL algorithms are based on the Bellman operator, which inevitably suffer from cumulative offsets from sub-optimal rewards during multi-step update processes. Additionally, off-policy training frameworks suffer from Out-of-Distribution (OOD) state-actions. Second, while conservative terms help solve the OOD issue, balancing the conservative term is difficult. To address these limitations, we fully integrate a one-step density-weighted Behavioral Cloning (BC) objective for IL with auxiliary imperfect demonstration. Theoretically, we demonstrate that this adaptation can effectively correct the distribution of policies trained on unknown-quality datasets to align with the expert policy's distribution. Moreover, the difference between the empirical and the optimal value function is proportional to the upper bound of ADR's objective, indicating that minimizing ADR's objective is akin to approaching the optimal value. Experimentally, we validated the performance of ADR by conducting extensive evaluations. Specifically, ADR outperforms all of the selected IL algorithms on tasks from the Gym-Mujoco domain. Meanwhile, it achieves an 89.5% improvement over IQL when utilizing ground truth rewards on tasks from the Adroit and Kitchen domains. Our codebase will be released at: https://github.com/stevezhangzA/Adverserial_Density_Regression.

cs.LG↗

arXiv2025

USV-AUV Collaboration Framework for Underwater Tasks under Extreme Sea Conditions

Autonomous underwater vehicles (AUVs) are valuable for ocean exploration due to their flexibility and ability to carry communication and detection units. Nevertheless, AUVs alone often face challenges in harsh and extreme sea conditions. This study introduces a unmanned surface vehicle (USV)-AUV collaboration framework, which includes high-precision multi-AUV positioning using USV path planning via Fisher information matrix optimization and reinforcement learning for multi-AUV cooperative tasks. Applied to a multi-AUV underwater data collection task scenario, extensive simulations validate the framework's feasibility and superior performance, highlighting exceptional coordination and robustness under extreme sea conditions. To accelerate relevant research in this field, we have made the simulation code (demo version) available as open-source.

cs.RO↗

arXiv2025

PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platforms

Deploying large language models (LLMs) locally on mobile devices is advantageous in scenarios where transmitting data to remote cloud servers is either undesirable due to privacy concerns or impractical due to network connection. Recent advancements (MLC, 2023a; Gerganov, 2023) have facilitated the local deployment of LLMs. However, local deployment also presents challenges, particularly in balancing quality (generative performance), latency, and throughput within the hardware constraints of mobile devices. In this paper, we introduce our lightweight, all-in-one automated benchmarking framework that allows users to evaluate LLMs on mobile devices. We provide a comprehensive benchmark of various popular LLMs with different quantization configurations (both weights and activations) across multiple mobile platforms with varying hardware capabilities. Unlike traditional benchmarks that assess full-scale models on high-end GPU clusters, we focus on evaluating resource efficiency (memory and power consumption) and harmful output for compressed models on mobile devices. Our key observations include i) differences in energy efficiency and throughput across mobile platforms; ii) the impact of quantization on memory usage, GPU execution time, and power consumption; and iii) accuracy and performance degradation of quantized models compared to their non-quantized counterparts; and iv) the frequency of hallucinations and toxic content generated by compressed LLMs on mobile devices.

cs.LG↗

arXiv2024

Thermal Induced Structural Competitiveness and Metastability of Body-centered Cubic Iron under Non-Equilibrium Conditions

The structure and stability of iron near melting at multi-megabar pressures are of significant interest in high pressure physics and earth and planetary sciences. While the body-centered cubic (BCC) phase is generally recognized as unstable at lower temperatures, its stability relative to the hexagonal close-packed (HCP) phase at high temperatures (approximately 0.5 eV) in the Earth's inner core (IC) remains a topic of ongoing theoretical and experimental debate. Our ab initio calculations show a significant drop in energy, the emergence of a plateau and a local minimum in the potential energy surface, and stabilization of all phonon modes at elevated electron temperatures (>1-1.5 eV). These effects increase the competition among the BCC, HCP, and the face-centered cubic (FCC) phases and lead to the metastability of the BCC structure. Furthermore, the thermodynamic stability of BCC iron is enhanced by its substantial lattice vibration entropy. This thermally induced structural competitiveness and metastability under non-equilibrium conditions provide a clear theoretical framework for understanding iron phase relations and solidification processes, both experimentally and in the IC.

cond-mat.mtrl-sci↗

arXiv2024

Field-free current-induced magnetization switching of a room temperature van der Waals magnet for neuromorphic computing

Spin orbit torque (SOT) has become a promising approach to efficiently manipulate the magnetization switching in spintronic devices. As a main factor to impact the device performance, the high quality interface is essentially desired, which can be readily acquired by using the two-dimensional (2D) van der Waals (vdW) materials. Recently, a 2D ferromagnetic material Fe3GaTe2 has been discovered to possess the above-room-temperature Curie temperature and strong perpendicular magnetic anisotropy (PMA), providing an excellent candidate to build spintronic devices. On the other hand, an external magnetic field is necessary for the SOT-driven deterministic switching of perpendicular magnetization, which has become a block for the real applications. Here, we realize the field-free SOT switching of Fe3GaTe2 at room temperature based on the Fe3GaTe2/MnPt heterostructure. In addition, inspired by the superiority of 2D materials in 3D heterogeneous integration, we explore the potential of our device in the computing in memory (CIM). With the application of the current pulses, the gradual switching of our device at zero field imitates the function of artificial synapse in the convolutional neural network (CNN), achieving a high accuracy (~92.8%) pattern recognition. Our work proposes a feasible solution for field-free SOT switching in 2D vdW spintronic devices, which paves the way for applications in magnetic memory and neuromorphic computing.

physics.app-ph↗

arXiv2024

YbNi$_4$Mg: Superheavy fermion with enhanced Wilson ratio and magnetocaloric effect

A comprehensive study of the low-temperature properties of YbNi$_4$Mg has revealed evidence of a superheavy-fermion state, characterized by a large electronic specific-heat coefficient $γ_0$ $\approx$ 5.65 J mol$^{-1}$ K$^{-2}$ and an elevated Wilson ratio $R_W$ = 32.1. No magnetic ordering was observed down to 70 mK; however, a broad maximum appears in the specific heat at $T^*$ = 0.3 K, along with a shoulder in the derivative of susceptibility d$χ$/d$T$ and resistivity d$ρ$/d$T$. These features indicate a cooperative yet short-ranged magnetism entwined with the superheavy Fermi liquid. The large Wilson ratio, which is also detected in other superheavy-fermion compounds lacking long-range order, might be a signature of residual spin fluctuations. Applying a weak magnetic field of $\sim$0.1 T induces a metamagnetic-like crossover, as demonstrated by the quasi-adiabatic demagnetization measurements showing a broad minimum in the temperature-field trace. Here, an enhanced magnetocaloric cooling effect stemming from the field-sensitive superheavy-fermion state is observed, rivaling that of the well-established insulating magnetic coolants like the rare-earth garnet Gd$_3$Ga$_5$O$_{12}$.

cond-mat.str-el↗

arXiv2024

Understanding chiral charge-density wave by frozen chiral phonon

Charge density wave (CDW) is discovered within a wide interval in solids, however, its microscopic nature is still not transparent in most realistic materials, and the recently studied chiral ones with chiral structural distortion remain unclear. In this paper, we try to understand the driving forces of chiral CDW transition by chiral phonons from the electron-phonon coupling scenario. We use the prototypal monolayer 1T-TiSe$_2$ as a case study to unveil the absence of chirality in the CDW transition and propose a general approach, i.e., symmetry-breaking stimuli, to engineer the chirality of CDW in experiments. Inelastic scattering patterns are also studied as a benchmark of chiral CDW (CCDW, which breaks the mirror/inversion symmetry in 2D/3D systems). We notice that the anisotropy changing of Bragg peak profiles, which is contributed by the soft chiral phonons, can show a remarkable signature for CCDW. Our findings pave a path to understanding the CCDW from the chiral phonon perspective, especially in van der Waals materials, and provide a powerful way to manipulate the chirality of CDW.

cond-mat.mtrl-sci↗

arXiv2024

Examining the potential synthesis of new elements with $^{294}$Og

In the relentless pursuit of expanding the periodic table, the discovery of element 119 remains elusive, despite two decades of dedicated research efforts. The traditional fusion-evaporation approach, although fruitful in the past, now appears to be approaching its operational limits. This scenario sets the stage for considering innovative methodologies essential for further advancements in the field of superheavy elements. Here, we introduce a pioneering strategy aimed at synthesizing element 119 by adapting and extending the nuclear reaction processes previously successful in producing element $^{294}$Og. This involved the fusion of $^{48}$Ca and $^{249}$Cf. Building on this, our novel approach incorporates an additional reactive target -- specifically, hydrogen -- positioned strategically behind the $^{249}$Cf. This configuration is designed to facilitate an immediate secondary reaction of the nascent $^{294}$Og with hydrogen, potentially forging new pathways to element 119. Preliminary insights also suggest that employing isotopes like deuterium or helium-3 as targets may not only enhance the production rates of element 119 but might also pave the way for the synthesis of even heavier elements, extending up to elements 120 and 121. We delve into the technicalities and feasibility of employing a dual-target method using a $^{48}$Ca beam, exploring new horizons in the quest for the superheavy unknown.

nucl-th↗

arXiv2024

Multimodal Instruction Tuning with Hybrid State Space Models

Handling lengthy context is crucial for enhancing the recognition and understanding capabilities of multimodal large language models (MLLMs) in applications such as processing high-resolution images or high frame rate videos. The rise in image resolution and frame rate substantially increases computational demands due to the increased number of input tokens. This challenge is further exacerbated by the quadratic complexity with respect to sequence length of the self-attention mechanism. Most prior works either pre-train models with long contexts, overlooking the efficiency problem, or attempt to reduce the context length via downsampling (e.g., identify the key image patches or frames) to decrease the context length, which may result in information loss. To circumvent this issue while keeping the remarkable effectiveness of MLLMs, we propose a novel approach using a hybrid transformer-MAMBA model to efficiently handle long contexts in multimodal applications. Our multimodal model can effectively process long context input exceeding 100k tokens, outperforming existing models across various benchmarks. Remarkably, our model enhances inference efficiency for high-resolution images and high-frame-rate videos by about 4 times compared to current models, with efficiency gains increasing as image resolution or video frames rise. Furthermore, our model is the first to be trained on low-resolution images or low-frame-rate videos while being capable of inference on high-resolution images and high-frame-rate videos, offering flexibility for inference in diverse scenarios.

cs.CV↗

arXiv2024

Unraveling the Gradient Descent Dynamics of Transformers

While the Transformer architecture has achieved remarkable success across various domains, a thorough theoretical foundation explaining its optimization dynamics is yet to be fully developed. In this study, we aim to bridge this understanding gap by answering the following two core questions: (1) Which types of Transformer architectures allow Gradient Descent (GD) to achieve guaranteed convergence? and (2) Under what initial conditions and architectural specifics does the Transformer achieve rapid convergence during training? By analyzing the loss landscape of a single Transformer layer using Softmax and Gaussian attention kernels, our work provides concrete answers to these questions. Our findings demonstrate that, with appropriate weight initialization, GD can train a Transformer model (with either kernel type) to achieve a global optimal solution, especially when the input embedding dimension is large. Nonetheless, certain scenarios highlight potential pitfalls: training a Transformer using the Softmax attention kernel may sometimes lead to suboptimal local solutions. In contrast, the Gaussian attention kernel exhibits a much favorable behavior. Our empirical study further validate the theoretical findings.

cs.LG↗

arXiv2024

A dynamical clipping approach with task feedback for Proximal Policy Optimization

Proximal Policy Optimization (PPO) has been broadly applied to robotics learning, showcasing stable training performance. However, the fixed clipping bound setting may limit the performance of PPO. Specifically, there is no theoretical proof that the optimal clipping bound remains consistent throughout the entire training process. Meanwhile, previous researches suggest that a fixed clipping bound restricts the policy's ability to explore. Therefore, many past studies have aimed to dynamically adjust the PPO clipping bound to enhance PPO's performance. However, the objective of these approaches are not directly aligned with the objective of reinforcement learning (RL) tasks, which is to maximize the cumulative Return. Unlike previous clipping approaches, we propose a bi-level proximal policy optimization objective that can dynamically adjust the clipping bound to better reflect the preference (maximizing Return) of these RL tasks. Based on this bi-level proximal policy optimization paradigm, we introduce a new algorithm named Preference based Proximal Policy Optimization (Pb-PPO). Pb-PPO utilizes a multi-armed bandit approach to refelect RL preference, recommending the clipping bound for PPO that can maximizes the current Return. Therefore, Pb-PPO results in greater stability and improved performance compared to PPO with a fixed clipping bound. We test Pb-PPO on locomotion benchmarks across multiple environments, including Gym-Mujoco and legged-gym. Additionally, we validate Pb-PPO on customized navigation tasks. Meanwhile, we conducted comparisons with PPO using various fixed clipping bounds and various of clipping approaches. The experimental results indicate that Pb-PPO demonstrates superior training performance compared to PPO and its variants. Our codebase has been released at : https://github.com/stevezhangzA/pb_ppo

cs.LG↗

arXiv2024

Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs

Robust therapeutic relationships between counselors and clients are fundamental to counseling effectiveness. The assessment of therapeutic alliance is well-established in traditional face-to-face therapy but may not directly translate to text-based settings. With millions of individuals seeking support through online text-based counseling, understanding the relationship in such contexts is crucial. In this paper, we present an automatic approach using large language models (LLMs) to understand the development of therapeutic alliance in text-based counseling. We adapt a theoretically grounded framework specifically to the context of online text-based counseling and develop comprehensive guidelines for characterizing the alliance. We collect a comprehensive counseling dataset and conduct multiple expert evaluations on a subset based on this framework. Our LLM-based approach, combined with guidelines and simultaneous extraction of supportive evidence underlying its predictions, demonstrates effectiveness in identifying the therapeutic alliance. Through further LLM-based evaluations on additional conversations, our findings underscore the challenges counselors face in cultivating strong online relationships with clients. Furthermore, we demonstrate the potential of LLM-based feedback mechanisms to enhance counselors' ability to build relationships, supported by a small-scale proof-of-concept.

cs.CL↗

arXiv2024

SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Support

Developing specialized dialogue systems for mental health support requires multi-turn conversation data, which has recently garnered increasing attention. However, gathering and releasing large-scale, real-life multi-turn conversations that could facilitate advancements in mental health support presents challenges in data privacy protection and the time and cost involved in crowdsourcing. To address these challenges, we introduce SMILE, a single-turn to multi-turn inclusive language expansion technique that prompts ChatGPT to rewrite public single-turn dialogues into multi-turn ones. Our work begins by analyzing language transformation and validating the feasibility of our proposed method. We conduct a study on dialogue diversity, including lexical features, semantic features, and dialogue topics, demonstrating the effectiveness of our method. Further, we employ our method to generate a large-scale, lifelike, and diverse dialogue dataset named SMILECHAT, consisting of 55k dialogues. Finally, we utilize the collected corpus to develop a mental health chatbot, MeChat. To better assess the quality of SMILECHAT, we collect a small-scale real-life counseling dataset conducted by data anonymization. Both automatic and human evaluations demonstrate significant improvements in our dialogue system and confirm that SMILECHAT is high-quality. Code, data, and model are publicly available at https://github.com/qiuhuachuan/smile.

cs.CL↗

arXiv2024

CausalVE: Face Video Privacy Encryption via Causal Video Prediction

Advanced facial recognition technologies and recommender systems with inadequate privacy technologies and policies for facial interactions increase concerns about bioprivacy violations. With the proliferation of video and live-streaming websites, public-face video distribution and interactions pose greater privacy risks. Existing techniques typically address the risk of sensitive biometric information leakage through various privacy enhancement methods but pose a higher security risk by corrupting the information to be conveyed by the interaction data, or by leaving certain biometric features intact that allow an attacker to infer sensitive biometric information from them. To address these shortcomings, in this paper, we propose a neural network framework, CausalVE. We obtain cover images by adopting a diffusion model to achieve face swapping with face guidance and use the speech sequence features and spatiotemporal sequence features of the secret video for dynamic video inference and prediction to obtain a cover video with the same number of frames as the secret video. In addition, we hide the secret video by using reversible neural networks for video hiding so that the video can also disseminate secret data. Numerous experiments prove that our CausalVE has good security in public video dissemination and outperforms state-of-the-art methods from a qualitative, quantitative, and visual point of view.

cs.CV↗

arXiv2024

AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alternative to Attention in ViTs

We propose and demonstrate an alternating Fourier and image domain filtering approach for feature extraction as an efficient alternative to build a vision backbone without using the computationally intensive attention. The performance among the lightweight models reaches the state-of-the-art level on ImageNet-1K classification, and improves downstream tasks on object detection and segmentation consistently as well. Our approach also serves as a new tool to compress vision transformers (ViTs).

cs.CV↗

arXiv2024

Laboratorial radiative shocks with multiple parameters and first quantifying verifications to core-collapse supernovae

We present experiments to reproduce the characteristics of core-collapse supernovae with different stellar masses and initial explosion energies in the laboratory. In the experiments, shocks are driven in 1.2 atm and 1.9 atm xenon gas by laser with energy from 1600J to 2800J on the SGIII prototype laser facility. The average shock velocities and shocked densities are obtained from experiments. Experimental results reveal that higher laser energy and lower Xe gas density led to higher shock velocity, and lower Xe gas initial density has a higher compression. Modeling of the experiments using the 2D radiation hydrodynamic codes Icefire shows excellent agreement with the experimental results and gives the temperature. These results will contribute to time-domain astrophysical systems, such as gravitational supernovae, where a strong radiative shock propagates outward from the center of the star after the core collapses.

astro-ph.HE↗

arXiv2024

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics. In this setting, the Q-function of each RL problem (task) can be decomposed into a successor feature (SF) and a reward mapping: the former characterizes the transition dynamics, and the latter characterizes the task-specific reward function. This Q-function decomposition, coupled with a policy improvement operator known as generalized policy improvement (GPI), reduces the sample complexity of finding the optimal Q-function, and thus the SF \& GPI framework exhibits promising empirical performance compared to traditional RL methods like Q-learning. However, its theoretical foundations remain largely unestablished, especially when learning the successor features using deep neural networks (SF-DQN). This paper studies the provable knowledge transfer using SFs-DQN in transfer RL problems. We establish the first convergence analysis with provable generalization guarantees for SF-DQN with GPI. The theory reveals that SF-DQN with GPI outperforms conventional RL approaches, such as deep Q-network, in terms of both faster convergence rate and better generalization. Numerical experiments on real and synthetic RL tasks support the superior performance of SF-DQN \& GPI, aligning with our theoretical findings.

cs.LG↗

arXiv2024

TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering

Four-dimensional Digital Subtraction Angiography (4D DSA) is a medical imaging technique that provides a series of 2D images captured at different stages and angles during the process of contrast agent filling blood vessels. It plays a significant role in the diagnosis of cerebrovascular diseases. Improving the rendering quality and speed under sparse sampling is important for observing the status and location of lesions. The current methods exhibit inadequate rendering quality in sparse views and suffer from slow rendering speed. To overcome these limitations, we propose TOGS, a Gaussian splatting method with opacity offset over time, which can effectively improve the rendering quality and speed of 4D DSA. We introduce an opacity offset table for each Gaussian to model the opacity offsets of the Gaussian, using these opacity-varying Gaussians to model the temporal variations in the radiance of the contrast agent. By interpolating the opacity offset table, the opacity variation of the Gaussian at different time points can be determined. This enables us to render the 2D DSA image at that specific moment. Additionally, we introduced a Smooth loss term in the loss function to mitigate overfitting issues that may arise in the model when dealing with sparse view scenarios. During the training phase, we randomly prune Gaussians, thereby reducing the storage overhead of the model. The experimental results demonstrate that compared to previous methods, this model achieves state-of-the-art render quality under the same number of training views. Additionally, it enables real-time rendering while maintaining low storage overhead. The code is available at https://github.com/hustvl/TOGS.

cs.CV↗