arXiv subjects

Zhe Wang

Publications and source records attributed to Zhe Wang.

At least 271 records · Page 15Linked to original sources

arXiv2024

Online Adversarial Knowledge Distillation for Graph Neural Networks

Knowledge distillation, a technique recently gaining popularity for enhancing model generalization in Convolutional Neural Networks (CNNs), operates under the assumption that both teacher and student models are trained on identical data distributions. However, its effect on Graph Neural Networks (GNNs) is less than satisfactory since the graph topology and node attributes are prone to evolve, thereby leading to the issue of distribution shift. In this paper, we tackle this challenge by simultaneously training a group of graph neural networks in an online distillation fashion, where the group knowledge plays a role as a dynamic virtual teacher and the structure changes in graph neural networks are effectively captured. To improve the distillation performance, two types of knowledge are transferred among the students to enhance each other: local knowledge reflecting information in the graph topology and node attributes, and global knowledge reflecting the prediction over classes. We transfer the global knowledge with KL-divergence as the vanilla knowledge distillation does, while exploiting the complicated structure of the local knowledge with an efficient adversarial cyclic learning framework. Extensive experiments verified the effectiveness of our proposed online adversarial distillation approach. The code is published at https://github.com/wangz3066/OnlineDistillGCN.

cs.LG↗

arXiv2024

Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding

Learning effective representations for Continuous-Time Dynamic Graphs (CTDGs) has garnered significant research interest, largely due to its powerful capabilities in modeling complex interactions between nodes. A fundamental and crucial requirement for representation learning in CTDGs is the appropriate estimation and preservation of proximity. However, due to the sparse and evolving characteristics of CTDGs, the spatial-temporal properties inherent in high-order proximity remain largely unexplored. Despite its importance, this property presents significant challenges due to the computationally intensive nature of personalized interaction intensity estimation and the dynamic attributes of CTDGs. To this end, we propose a novel Correlated Spatial-Temporal Positional encoding that incorporates a parameter-free personalized interaction intensity estimation under the weak assumption of the Poisson Point Process. Building on this, we introduce the Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding (CorDGT), which efficiently retains the evolving spatial-temporal high-order proximity for effective node representation learning in CTDGs. Extensive experiments on seven small and two large-scale datasets demonstrate the superior performance and scalability of the proposed CorDGT. The code is available at: https://github.com/wangz3066/CorDGT.

cs.LG↗

arXiv2024

Performance Analysis of XL-MIMO with Rotary and Movable Antennas for High-speed Railway

The rotary and movable antennas (ROMA) technology is efficient in enhancing wireless network capacity by adjusting both the antenna spacing and three-dimensional (3D) rotation of antenna surfaces, based on the spatial distribution of users and channel statistics. Applying ROMA to high-speed rail (HSR) wireless communications can significantly improve system performance in terms of array gain and spatial multiplexing. However, the rapidly changing channel conditions in HSR scenarios present challenges for ROMA configuration. In this correspondence, we propose a analytical framework for configuring ROMA-based extremely large-scale multiple-input-multiple-output (XL-MIMO) system in HSR scenarios based on spatial correlation. First, we develop a localization model based on a mobility-aware near-field beam training algorithm to determine the real-time position of the train relay antennas. Next, we derive the expression for channel orthogonality and antenna spacing based on the spatial correlation matrix, and obtain the optimal antenna spacing when the transceiver panels are aligned in parallel. Moreover, we propose an optimization algorithm for the rotation angle of the transceiver panels, leveraging the differential evolution method, to determine the optimal angle. Finally, numerical results are provided to validate the computational results and optimization algorithm.

eess.SP↗

arXiv2024

EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction

Reconstructing real-world 3D objects has numerous applications in computer vision, such as virtual reality, video games, and animations. Ideally, 3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time. Traditional methods match pixels between images using photo-consistency constraints or learned features, while differentiable rendering methods like Neural Radiance Fields (NeRF) use differentiable volume rendering or surface-based representation to generate high-fidelity scenes. However, these methods require excessive runtime for rendering, making them impractical for daily applications. To address these challenges, we present $\textbf{EvaSurf}$, an $\textbf{E}$fficient $\textbf{V}$iew-$\textbf{A}$ware implicit textured $\textbf{Surf}$ace reconstruction method. In our method, we first employ an efficient surface-based model with a multi-view supervision module to ensure accurate mesh reconstruction. To enable high-fidelity rendering, we learn an implicit texture embedded with view-aware encoding to capture view-dependent information. Furthermore, with the explicit geometry and the implicit texture, we can employ a lightweight neural shader to reduce the expense of computation and further support real-time rendering on common mobile devices. Extensive experiments demonstrate that our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets. Moreover, our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second), with a final package required for rendering taking up only 40-50 MB.

cs.CV↗

arXiv2024

Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach

In ultra-dense unmanned aerial vehicle (UAV) networks, it is challenging to coordinate the resource allocation and interference management among large-scale UAVs, for providing flexible and efficient service coverage to the ground users (GUs). In this paper, we propose a learning-based resource allocation scheme in an ultra-dense UAV communication network, where the GUs' service demands are time-varying with unknown distributions. We formulate the non-cooperative game among multiple co-channel UAVs as a stochastic game, where each UAV jointly optimizes its trajectory, user association, and downlink power control to maximize the expectation of its locally cumulative energy efficiency under the interference and energy constraints. To cope with the scalability issue in a large-scale network, we further formulate the problem as a mean-field game (MFG), which simplifies the interactions among the UAVs into a two-player game between a representative UAV and a mean-field. We prove the existence and uniqueness of the equilibrium for the MFG, and propose a model-free mean-field reinforcement learning algorithm named maximum entropy mean-field deep Q network (ME-MFDQN) to solve the mean-field equilibrium in both fully and partially observable scenarios. The simulation results reveal that the proposed algorithm improves the energy efficiency compared with the benchmark algorithms. Moreover, the performance can be further enhanced if the GUs' service demands exhibit higher temporal correlation or if the UAVs have wider observation capabilities over their nearby GUs.

eess.SY↗

arXiv2024

Magnetoresistance oscillations in vertical junctions of 2D antiferromagnetic semiconductor CrPS$_4$

Magnetoresistance (MR) oscillations serve as a hallmark of intrinsic quantum behavior, traditionally observed only in conducting systems. Here we report the discovery of MR oscillations in an insulating system, the vertical junctions of CrPS$_4$ which is a two dimensional (2D) A-type antiferromagnetic semiconductor. Systematic investigations of MR peaks under varying conditions, including electrode materials, magnetic field direction, temperature, voltage bias and layer number, elucidate a correlation between MR oscillations and spin-canted states in CrPS$_4$. Experimental data and analysis point out the important role of the in-gap electronic states in generating MR oscillations, and we proposed that spin selected interlayer hopping of localized defect states may be responsible for it. Our findings not only illuminate the unusual electronic transport in CrPS$_4$ but also underscore the potential of van der Waals magnets for exploring interesting phenomena.

cond-mat.mes-hall↗

arXiv2024

Magnetization process of a quasi-two-dimensional quantum magnet: Two-step symmetry restoration and dimensional reduction

We report on a comprehensive thermodynamic study of a quasi-two-dimensional (quasi-2D) quantum magnet Cu$_2$(OH)$_3$Br which in the 2D layer can be viewed as strongly coupled alternating antiferromagnetic and ferromagnetic chains. In an applied magnetic field transverse to the ordered spins below $T_N=9.3$ K, a field-induced phase transition from the 3D ordered to a disordered phase occurs at $B_c=16.3$ T for the lowest temperature, which is featured by an onset of a one-half plateau-like magnetization. By performing quantum Monte Carlo simulations of the relevant 2D model, we find that the plateau-like magnetization corresponds to a partial symmetry restoration and the full polarization in the ferromagnetic chains. Our numerical simulations also show that the magnetization saturation occurs with full symmetry restoration at a much higher field of $B_s \simeq 95$ T, corresponding to a 1D quantum phase transition in the antiferromagnetic chains. We argue that the experimentally observed field-induced phase transition at $B_c$ follows from the partial symmetry restoration and the concomitant dimensional reduction.

cond-mat.str-el↗

arXiv2024

Bayesian Calibration of Win Rate Estimation with LLM Evaluators

Recent advances in large language models (LLMs) show the potential of using LLMs as evaluators for assessing the quality of text generations from LLMs. However, applying LLM evaluators naively to compare or judge between different systems can lead to unreliable results due to the intrinsic win rate estimation bias of LLM evaluators. In order to mitigate this problem, we propose two calibration methods, Bayesian Win Rate Sampling (BWRS) and Bayesian Dawid-Skene, both of which leverage Bayesian inference to more accurately infer the true win rate of generative language models. We empirically validate our methods on six datasets covering story generation, summarization, and instruction following tasks. We show that both our methods are effective in improving the accuracy of win rate estimation using LLMs as evaluators, offering a promising direction for reliable automatic text quality evaluation.

cs.CL↗

arXiv2024

Deep Group Interest Modeling of Full Lifelong User Behaviors for CTR Prediction

Extracting users' interests from their lifelong behavior sequence is crucial for predicting Click-Through Rate (CTR). Most current methods employ a two-stage process for efficiency: they first select historical behaviors related to the candidate item and then deduce the user's interest from this narrowed-down behavior sub-sequence. This two-stage paradigm, though effective, leads to information loss. Solely using users' lifelong click behaviors doesn't provide a complete picture of their interests, leading to suboptimal performance. In our research, we introduce the Deep Group Interest Network (DGIN), an end-to-end method to model the user's entire behavior history. This includes all post-registration actions, such as clicks, cart additions, purchases, and more, providing a nuanced user understanding. We start by grouping the full range of behaviors using a relevant key (like item_id) to enhance efficiency. This process reduces the behavior length significantly, from O(10^4) to O(10^2). To mitigate the potential loss of information due to grouping, we incorporate two categories of group attributes. Within each group, we calculate statistical information on various heterogeneous behaviors (like behavior counts) and employ self-attention mechanisms to highlight unique behavior characteristics (like behavior type). Based on this reorganized behavior data, the user's interests are derived using the Transformer technique. Additionally, we identify a subset of behaviors that share the same item_id with the candidate item from the lifelong behavior sequence. The insights from this subset reveal the user's decision-making process related to the candidate item, improving prediction accuracy. Our comprehensive evaluation, both on industrial and public datasets, validates DGIN's efficacy and efficiency.

cs.IR↗

arXiv2024

Decoupled Data Augmentation for Improving Image Classification

Recent advancements in image mixing and generative data augmentation have shown promise in enhancing image classification. However, these techniques face the challenge of balancing semantic fidelity with diversity. Specifically, image mixing involves interpolating two images to create a new one, but this pixel-level interpolation can compromise fidelity. Generative augmentation uses text-to-image generative models to synthesize or modify images, often limiting diversity to avoid generating out-of-distribution data that potentially affects accuracy. We propose that this fidelity-diversity dilemma partially stems from the whole-image paradigm of existing methods. Since an image comprises the class-dependent part (CDP) and the class-independent part (CIP), where each part has fundamentally different impacts on the image's fidelity, treating different parts uniformly can therefore be misleading. To address this fidelity-diversity dilemma, we introduce Decoupled Data Augmentation (De-DA), which resolves the dilemma by separating images into CDPs and CIPs and handling them adaptively. To maintain fidelity, we use generative models to modify real CDPs under controlled conditions, preserving semantic consistency. To enhance diversity, we replace the image's CIP with inter-class variants, creating diverse CDP-CIP combinations. Additionally, we implement an online randomized combination strategy during training to generate numerous distinct CDP-CIP combinations cost-effectively. Comprehensive empirical evaluations validate the effectiveness of our method.

cs.CV↗

arXiv2024

Wasserstein convergence rates in the invariance principle for sequential dynamical systems

In this paper, we consider the convergence rate with respect to the Wasserstein distance in the invariance principle for sequential dynamical systems. We utilize and modify the techniques previously employed for stationary sequences to address our non-stationary case. Under certain assumptions, we can apply our result to a large class of dynamical systems, including sequential $β_n$-transformations, piecewise uniformly expanding maps with additive noise in one-dimensional and multidimensional case, and so on.

math.DS↗

arXiv2024

Decision Transformers for Wireless Communications: A New Paradigm of Resource Management

As the next generation of mobile systems evolves, artificial intelligence (AI) is expected to deeply integrate with wireless communications for resource management in variable environments. In particular, deep reinforcement learning (DRL) is an important tool for addressing stochastic optimization issues of resource allocation. However, DRL has to start each new training process from the beginning once the state and action spaces change, causing low sample efficiency and poor generalization ability. Moreover, each DRL training process may take a large number of epochs to converge, which is unacceptable for time-sensitive scenarios. In this paper, we adopt an alternative AI technology, namely, Decision Transformer (DT), and propose a DT-based adaptive decision architecture for wireless resource management. This architecture innovates through constructing pre-trained models in the cloud and then fine-tuning personalized models at the edges. By leveraging the power of DT models learned over offline datasets, the proposed architecture is expected to achieve rapid convergence with many fewer training epochs and higher performance in new scenarios with different state and action spaces, compared with DRL. We then design DT frameworks for two typical communication scenarios: intelligent reflecting surfaces-aided communications and unmanned aerial vehicle-aided mobile edge computing. Simulations demonstrate that the proposed DT frameworks achieve over $3$-$6$ times speedup in convergence and better performance relative to the classic DRL method, namely, proximal policy optimization.

eess.SP↗

arXiv2024

Sudden change in entanglement Hamiltonian: Phase diagram of an Ising entanglement Hamiltonian

The form of the entanglement Hamiltonian varies with the parameters of the original system. Whether there is a singularity is the key problem for demonstrating/negating the universality of the relation between the entanglement spectrum and edge energy spectrum. We carefully study the phase diagram of a 1D Ising entanglement Hamiltonian as an example to clarify the long-standing controversy of the general relation between the entanglement Hamiltonian and original Hamiltonian. Interestingly, even if the singularities indeed exist, the Li-Haldane-Poilblanc conjecture, i.e., the general relation between the entanglement spectrum and edge energy spectrum, seemingly still holds.

cond-mat.str-el↗

arXiv2024

Parallelize Over Data Particle Advection: Participation, Ping Pong Particles, and Overhead

Particle advection is one of the foundational algorithms for visualization and analysis and is central to understanding vector fields common to scientific simulations. Achieving efficient performance with large data in a distributed memory setting is notoriously difficult. Because of its simplicity and minimized movement of large vector field data, the Parallelize over Data (POD) algorithm has become a de facto standard. Despite its simplicity and ubiquitous usage, the scaling issues with the POD algorithm are known and have been described throughout the literature. In this paper, we describe a set of in-depth analyses of the POD algorithm that shed new light on the underlying causes for the poor performance of this algorithm. We designed a series of representative workloads to study the performance of the POD algorithm and executed them on a supercomputer while collecting timing and statistical data for analysis. We then performed two different types of analysis. In the first analysis, we introduce two novel metrics for measuring algorithmic efficiency over the course of a workload run. The second analysis was from the perspective of the particles being advected. Using particle centric analysis, we identify that the overheads associated with particle movement between processes (not the communication itself) have a dramatic impact on the overall execution time. In the first analysis, we introduce two novel metrics for measuring algorithmic efficiency over the course of a workload run. The second analysis was from the perspective of the particles being advected. Using particle-centric analysis, we identify that the overheads associated with particle movement between processes have a dramatic impact on the overall execution time. These overheads become particularly costly when flow features span multiple blocks, resulting in repeated particle circulation between blocks.

cs.DC↗

arXiv2024

Towards Dynamic Graph Neural Networks with Provably High-Order Expressive Power

Dynamic Graph Neural Networks (DyGNNs) have garnered increasing research attention for learning representations on evolving graphs. Despite their effectiveness, the limited expressive power of existing DyGNNs hinders them from capturing important evolving patterns of dynamic graphs. Although some works attempt to enhance expressive capability with heuristic features, there remains a lack of DyGNN frameworks with provable and quantifiable high-order expressive power. To address this research gap, we firstly propose the k-dimensional Dynamic WL tests (k-DWL) as the referencing algorithms to quantify the expressive power of DyGNNs. We demonstrate that the expressive power of existing DyGNNs is upper bounded by the 1-DWL test. To enhance the expressive power, we propose Dynamic Graph Neural Network with High-order expressive power (HopeDGN), which updates the representation of central node pair by aggregating the interaction history with neighboring node pairs. Our theoretical results demonstrate that HopeDGN can achieve expressive power equivalent to the 2-DWL test. We then present a Transformer-based implementation for the local variant of HopeDGN. Experimental results show that HopeDGN achieved performance improvements of up to 3.12%, demonstrating the effectiveness of HopeDGN.

cs.LG↗

arXiv2024

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

cs.DC↗

arXiv2024

Surface phase transitions in a (1+1)-dimensional $SU(2)_1$ conformal field theory boundary coupled to a (2+1)-dimensional $Z_2$ bulk

We design a (2+1))-dimensional [(2+1)D] quantum spin model in which spin-1/2 ladders are coupled through antiferromagnetic Ising interactions. The model hosts a quantum phase transition in the (2+1)D $Z_2$ universality class from the Haldane phase to the antiferromagnetic Ising ordered phase. We focus on studying the surface properties of three different surface configurations when the Ising couplings are tuned. Different behaviors are found on different surfaces. We find ordinary and two different extraordinary surface critical behaviors (SCBs) at the bulk critical point. The ordinary SCBs belong to the surface universality class of the classical 3D Ising bulk transition. One extraordinary SCBs is induced by the topological properties of the Haldane phase. Another extraordinary SCBs at the bulk critical point is induced by an unconventional surface phase transition where the surface develops an Ising order before the bulk. This surface transition is realized by coupling a (1+1)-dimensional [(1+1)D] $SU(2)_1$ CFT boundary to a (2+1)D bulk with $Z_2$ symmetry. We find that the transition is neither a (1+1)D $Z_2$ transition, expected based on symmetry consideration, nor a Kosterlitz-Thouless-like transition, violating the previous theoretical prediction. This new surface phase transition and related extraordinary SCBs deserve further analytical and numerical exploration.

cond-mat.str-el↗

arXiv2024

Molecular-Scale Insights into the Heterogeneous Interactions Between an m-Terphenyl Isocyanide Ligand and Noble Metal Nanoparticles

The structural and chemical properties of metal nanoparticles are often dictated by their interactions with molecular ligand shells. These interactions are highly material-specific and can vary significantly even among elements within the same group or materials with similar crystal structure. Precise characterization of ligand-metal interactions is crucial for the rational design of ligands and the functionalization of nanoparticles. In this study, we found that the ligation behavior with m-terphenyl isocyanide molecule differs significantly between Au and Ag nanoparticles, with distinct ligand extraction efficiencies and size dependencies. Surface-enhanced Raman spectroscopy measurements revealed unique enhancement factors for two molecular vibrational modes between two metal surfaces, indicating different ligand binding geometries. Molecular-level characterization using scanning tunneling microscopy allowed us to directly visualize these variations between Ag and Au surfaces, which we assign as two distinct binding mechanisms. This molecular-scale visualization provides clear insights into the different ligand-metal interactions, as well as the chemical behavior and spectroscopic characteristics of isocyanide-functionalized nanoparticles.

cond-mat.mes-hall↗