arXiv subjects

Zihao Wang

Publications and source records attributed to Zihao Wang.

At least 271 records · Page 15Linked to original sources

arXiv2024

Diffusion based Zero-shot Medical Image-to-Image Translation for Cross Modality Segmentation

Cross-modality image segmentation aims to segment the target modalities using a method designed in the source modality. Deep generative models can translate the target modality images into the source modality, thus enabling cross-modality segmentation. However, a vast body of existing cross-modality image translation methods relies on supervised learning. In this work, we aim to address the challenge of zero-shot learning-based image translation tasks (extreme scenarios in the target modality is unseen in the training phase). To leverage generative learning for zero-shot cross-modality image segmentation, we propose a novel unsupervised image translation method. The framework learns to translate the unseen source image to the target modality for image segmentation by leveraging the inherent statistical consistency between different modalities for diffusion guidance. Our framework captures identical cross-modality features in the statistical domain, offering diffusion guidance without relying on direct mappings between the source and target domains. This advantage allows our method to adapt to changing source domains without the need for retraining, making it highly practical when sufficient labeled source domain data is not available. The proposed framework is validated in zero-shot cross-modality image segmentation tasks through empirical comparisons with influential generative models, including adversarial-based and diffusion-based models.

eess.IV↗

arXiv2024

Manifolds for which Huber's Theorem holds

Extensions of Huber's Theorem to higher dimensions with $L^\frac{n}{2}$ bounded scalar curvature have been extensively studied over the years. In this paper, we delve into the properties of conformal metrics on a punctured ball with $\|R\|_{L^\frac{n}{2}}<+\infty$, aiming to identify necessary geometric constraints for Huber's theorem to be applicable. Unexpectedly, such metrics are more rigid than we initially anticipated. For instance, we found that the volume density at infinity is precisely one, and the blow-down of the metric is $\mathbb{R}^n$. Specifically, in four dimensions, we derive the $L^2$-integrability of the Ricci curvature, which directly leads to the conclusion that the Pfaffian 4-form is integrable and adheres to a Gauss-Bonnet-Chern formula. Additionally, we demonstrate that a Gauss-Bonnet-Chern formula, previously verified by Lu and Wang under the assumption that the second fundamental form belongs to $L^4$, remains valid for $R \in L^2$. Consequently, on an orientable 4-dimensional manifold conformal to a domain in a closed manifold, Huber's Theorem holds when $R \in L^2$, if and only if the negative part of the Pfaffian 4-form is integrable.

math.DG↗

arXiv2024

Traffic State Estimation and Uncertainty Quantification at Signalized Intersections with Low Penetration Rate Vehicle Trajectory Data

This paper studies the traffic state estimation problem at signalized intersections with low penetration rate vehicle trajectory data. While many existing studies have proposed different methods to estimate unknown traffic states and parameters (e.g., penetration rate, queue length) with this data, most of them only provide a point estimation without knowing the uncertainty of these estimated values. It is important to quantify the estimation uncertainty caused by limited available data since it can explicitly inform us whether the available data is sufficient to satisfy the desired estimation accuracy. To fill this gap, we formulate the partially observable system as a hidden Markov model (HMM) based on the recently developed probabilistic time-space (PTS) model. The PTS model is a stochastic traffic flow model that is designed for modeling traffic flow dynamics near signalized intersections. Based on the HMM formulation, a single recursive program is developed for the Bayesian estimation of both traffic states and parameters. As a Bayesian approach, the proposed method provides the distributional estimation outcomes and directly quantifies the estimation uncertainty. We validate the proposed method with simulation studies and showcase its applicability to real-world vehicle trajectory data.

eess.SY↗

arXiv2024

Meta Operator for Complex Query Answering on Knowledge Graphs

Knowledge graphs contain informative factual knowledge but are considered incomplete. To answer complex queries under incomplete knowledge, learning-based Complex Query Answering (CQA) models are proposed to directly learn from the query-answer samples to avoid the direct traversal of incomplete graph data. Existing works formulate the training of complex query answering models as multi-task learning and require a large number of training samples. In this work, we explore the compositional structure of complex queries and argue that the different logical operator types, rather than the different complex query types, are the key to improving generalizability. Accordingly, we propose a meta-learning algorithm to learn the meta-operators with limited data and adapt them to different instances of operators under various complex queries. Empirical results show that learning meta-operators is more effective than learning original CQA or meta-CQA models.

cs.LG↗

arXiv2024

RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation

We explore how iterative revising a chain of thoughts with the help of information retrieval significantly improves large language models' reasoning and generation ability in long-horizon generation tasks, while hugely mitigating hallucination. In particular, the proposed method -- *retrieval-augmented thoughts* (RAT) -- revises each thought step one by one with retrieved information relevant to the task query, the current and the past thought steps, after the initial zero-shot CoT is generated. Applying RAT to GPT-3.5, GPT-4, and CodeLLaMA-7b substantially improves their performances on various long-horizon generation tasks; on average of relatively increasing rating scores by 13.63% on code generation, 16.96% on mathematical reasoning, 19.2% on creative writing, and 42.78% on embodied task planning. The demo page can be found at https://craftjarvis.github.io/RAT

cs.CL↗

arXiv2024

ZF Beamforming Tensor Compression for Massive MIMO Fronthaul

In the rapidly evolving landscape of 5G and beyond 5G (B5G) mobile cellular communications, efficient data compression and reconstruction strategies become paramount, especially in massive multiple-input multiple-output (MIMO) systems. A critical challenge in these systems is the capacity-limited fronthaul, particularly in the context of the Ethernet-based common public radio interface (eCPRI) connecting baseband units (BBUs) and remote radio units (RRUs). This capacity limitation hinders the effective handling of increased traffic and data flows. We propose a novel two-stage compression approach to address this bottleneck. The first stage employs sparse Tucker decomposition, targeting the weight tensor's low-rank components for compression. The second stage further compresses these components using complex givens decomposition and run-length encoding, substantially improving the compression ratio. Our approach specifically targets the Zero-Forcing (ZF) beamforming weights in BBUs. By reconstructing these weights in RRUs, we significantly alleviate the burden on eCPRI traffic, enabling a higher number of concurrent streams in the radio access network (RAN). Through comprehensive evaluations, we demonstrate the superior effectiveness of our method in Channel State Information (CSI) compression, paving the way for more efficient 5G/B5G fronthaul links.

cs.IT↗

arXiv2024

DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training

Recent developments have underscored the critical role of \textit{differential privacy} (DP) in safeguarding individual data for training machine learning models. However, integrating DP oftentimes incurs significant model performance degradation due to the perturbation introduced into the training process, presenting a formidable challenge in the {differentially private machine learning} (DPML) field. To this end, several mitigative efforts have been proposed, typically revolving around formulating new DPML algorithms or relaxing DP definitions to harmonize with distinct contexts. In spite of these initiatives, the diminishment induced by DP on models, particularly large-scale models, remains substantial and thus, necessitates an innovative solution that adeptly circumnavigates the consequential impairment of model utility. In response, we introduce DPAdapter, a pioneering technique designed to amplify the model performance of DPML algorithms by enhancing parameter robustness. The fundamental intuition behind this strategy is that models with robust parameters are inherently more resistant to the noise introduced by DP, thereby retaining better performance despite the perturbations. DPAdapter modifies and enhances the sharpness-aware minimization (SAM) technique, utilizing a two-batch strategy to provide a more accurate perturbation estimate and an efficient gradient descent, thereby improving parameter robustness against noise. Notably, DPAdapter can act as a plug-and-play component and be combined with existing DPML algorithms to further improve their performance. Our experiments show that DPAdapter vastly enhances state-of-the-art DPML algorithms, increasing average accuracy from 72.92\% to 77.09\% with a privacy budget of $ε=4$.

cs.LG↗

arXiv2024

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?

Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs. In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms. Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt. Further study reveals the common interaction mechanisms of LLMs during the discussion.

cs.CL↗

arXiv2024

Exploring Emerging Trends in 5G Malicious Traffic Analysis and Incremental Learning Intrusion Detection Strategies

The popularity of 5G networks poses a huge challenge for malicious traffic detection technology. The reason for this is that as the use of 5G technology increases, so does the risk of malicious traffic activity on 5G networks. Malicious traffic activity in 5G networks not only has the potential to disrupt communication services, but also to compromise sensitive data. This can have serious consequences for individuals and organizations. In this paper, we first provide an in-depth study of 5G technology and 5G security. Next we analyze and discuss the latest malicious traffic detection under AI and their applicability to 5G networks, and compare the various traffic detection aspects addressed by SOTA. The SOTA in 5G traffic detection is also analyzed. Next, we propose seven criteria for traffic monitoring datasets to confirm their suitability for future traffic detection studies. Finally, we present three major issues that need to be addressed for traffic detection in 5G environment. The concept of incremental learning techniques is proposed and applied in the experiments, and the experimental results prove to be able to solve the three problems to some extent.

cs.CR↗

arXiv2024

Coexistence of Superconductivity and Antiferromagnetism in Topological Magnet MnBi2Te4 Films

The interface of two materials can harbor unexpected emergent phenomena. One example is interface-induced superconductivity. In this work, we employ molecular beam epitaxy to grow a series of heterostructures formed by stacking together two non-superconducting antiferromagnetic materials, an intrinsic antiferromagnetic topological insulator MnBi2Te4 and an antiferromagnetic iron chalcogenide FeTe. Our electrical transport measurements reveal interface-induced superconductivity in these heterostructures. By performing scanning tunneling microscopy and spectroscopy measurements, we observe a proximity-induced superconducting gap on the top surface of the MnBi2Te4 layer, confirming the interaction between superconductivity and antiferromagnetism in the MnBi2Te4 layer. Our findings will advance the fundamental inquiries into the topological superconducting phase in hybrid devices and provide a promising platform for the exploration of chiral Majorana physics in MnBi2Te4-based heterostructures.

cond-mat.supr-con↗

arXiv2024

Rhythmic soliton interactions for integrated dual-microcomb spectroscopy

Rotation symmetry of microresonators supports the generation of phase-locked counter-propagating (CP) solitons that can potentially miniaturize dual-comb systems. Realization of these dual-comb compatible solitons in photonic integrated circuits remains a challenge. Here, we synthesized such CP solitons in an integrated silicon nitride microresonator and observed forced soliton oscillation due to rhythmic, time-varying soliton interactions. The interactions result in seconds mutual-coherence passively. Temporal motion in the soliton streams is discerned by measuring a quadratic-scaling frequency noise peaks and an inverse quadratic-scaling microcomb sidebands. By generating a CP soliton trimer to have two synchronized solitons in one of the orbiting directions, we resolve the incapability of measuring two unsynchronized CP soliton dimer pulses by optical cross-correlation, and show CP solitons undergo complex motion trajectory. We further prove that precise dual-comb spectroscopy with an acquisition time as short as 0.6 $μ$s is feasible using these solitons, although the temporal motion limits the dynamic range. Besides revealing soliton interactions with different group velocities, our work propels the realization of photonic integrated dual-comb spectrometers with high passive coherence.

physics.optics↗

arXiv2024

Concept Algebra for (Score-Based) Text-Controlled Generative Models

This paper concerns the structure of learned representations in text-guided generative models, focusing on score-based models. A key property of such models is that they can compose disparate concepts in a `disentangled' manner. This suggests these models have internal representations that encode concepts in a `disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space. We formalize what this means, show there's a natural choice for the representation, and develop a simple method for identifying the part of the representation corresponding to a given concept. In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with examples using Stable Diffusion. Code in https://github.com/zihao12/concept-algebra-code

cs.CL↗

arXiv2024

Theoretical Analysis of Inductive Biases in Deep Convolutional Networks

In this paper, we provide a theoretical analysis of the inductive biases in convolutional neural networks (CNNs). We start by examining the universality of CNNs, i.e., the ability to approximate any continuous functions. We prove that a depth of $\mathcal{O}(\log d)$ suffices for deep CNNs to achieve this universality, where $d$ in the input dimension. Additionally, we establish that learning sparse functions with CNNs requires only $\widetilde{\mathcal{O}}(\log^2d)$ samples, indicating that deep CNNs can efficiently capture {\em long-range} sparse correlations. These results are made possible through a novel combination of the multichanneling and downsampling when increasing the network depth. We also delve into the distinct roles of weight sharing and locality in CNNs. To this end, we compare the performance of CNNs, locally-connected networks (LCNs), and fully-connected networks (FCNs) on a simple regression task, where LCNs can be viewed as CNNs without weight sharing. On the one hand, we prove that LCNs require $Ω(d)$ samples while CNNs need only $\widetilde{\mathcal{O}}(\log^2d)$ samples, highlighting the critical role of weight sharing. On the other hand, we prove that FCNs require $Ω(d^2)$ samples, whereas LCNs need only $\widetilde{\mathcal{O}}(d)$ samples, underscoring the importance of locality. These provable separations quantify the difference between the two biases, and the major observation behind our proof is that weight sharing and locality break different symmetries in the learning process.

cs.LG↗

arXiv2024

A New Method of Pixel-level In-situ U-value Measurement for Building Envelopes Based on Infrared Thermography

The potential energy loss of aging buildings traps building owners in a cycle of underfunding operations and overpaying maintenance costs. Energy auditors intending to generate an energy model of a target building for performance assessment may struggle to obtain accurate results as the spatial distribution of temperatures is not considered when calculating the U-value of the building envelope. This paper proposes a pixel-level method based on infrared thermography (IRT) that considers two-dimensional (2D) spatial temperature distributions of the outdoor and indoor surfaces of the target wall to generate a 2D U-value map of the wall. The result supports that the proposed method can better reflect the actual thermal insulation performance of the target wall compared to the current IRT-based methods that use a single-point room temperature as input.

cs.CV↗

arXiv2024

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents' capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates' subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance superiority of ProAgent, outperforming five methods based on self-play and population-based training when cooperating with AI agents. Furthermore, in partnered with human proxy models, its performance exhibits an average improvement exceeding 10% compared to the current state-of-the-art method. For more information about our project, please visit~\url{https://pku-proagent.github.io}.

cs.AI↗

arXiv2024

Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects

Intelligent agents stand out as a potential path toward artificial general intelligence (AGI). Thus, researchers have dedicated significant effort to diverse implementations for them. Benefiting from recent progress in large language models (LLMs), LLM-based agents that use universal natural language as an interface exhibit robust generalization capabilities across various applications -- from serving as autonomous general-purpose task assistants to applications in coding, social, and economic domains, LLM-based agents offer extensive exploration opportunities. This paper surveys current research to provide an in-depth overview of LLM-based intelligent agents within single-agent and multi-agent systems. It covers their definitions, research frameworks, and foundational components such as their composition, cognitive and planning methods, tool utilization, and responses to environmental feedback. We also delve into the mechanisms of deploying LLM-based agents in multi-agent systems, including multi-role collaboration, message passing, and strategies to alleviate communication issues between agents. The discussions also shed light on popular datasets and application scenarios. We conclude by envisioning prospects for LLM-based agents, considering the evolving landscape of AI and natural language processing.

cs.AI↗

arXiv2023

NestE: Modeling Nested Relational Structures for Knowledge Graph Reasoning

Reasoning with knowledge graphs (KGs) has primarily focused on triple-shaped facts. Recent advancements have been explored to enhance the semantics of these facts by incorporating more potent representations, such as hyper-relational facts. However, these approaches are limited to \emph{atomic facts}, which describe a single piece of information. This paper extends beyond \emph{atomic facts} and delves into \emph{nested facts}, represented by quoted triples where subjects and objects are triples themselves (e.g., ((\emph{BarackObama}, \emph{holds\_position}, \emph{President}), \emph{succeed\_by}, (\emph{DonaldTrump}, \emph{holds\_position}, \emph{President}))). These nested facts enable the expression of complex semantics like \emph{situations} over time and \emph{logical patterns} over entities and relations. In response, we introduce NestE, a novel KG embedding approach that captures the semantics of both atomic and nested factual knowledge. NestE represents each atomic fact as a $1\times3$ matrix, and each nested relation is modeled as a $3\times3$ matrix that rotates the $1\times3$ atomic fact matrix through matrix multiplication. Each element of the matrix is represented as a complex number in the generalized 4D hypercomplex space, including (spherical) quaternions, hyperbolic quaternions, and split-quaternions. Through thorough analysis, we demonstrate the embedding's efficacy in capturing diverse logical patterns over nested facts, surpassing the confines of first-order logic-like expressions. Our experimental results showcase NestE's significant performance gains over current baselines in triple prediction and conditional link prediction. The code and pre-trained models are open available at https://github.com/xiongbo010/NestE.

cs.AI↗

arXiv2023

Open-orbit induced low field extremely large magnetoresistance in graphene/h-BN superlattices

We report intriguing and hitherto overlooked low-field room temperature extremely large magnetoresistance (XMR) patterns in graphene/hexagonal boron nitride (h-BN) superlattices that emerge due to the existence of open orbits within each miniband. This finding is set against the backdrop of the experimental discovery of the Hofstadter butterfly in moir superlattices, which has sparked considerable interest in the fractal quantum Hall regime. To cope with the challenge of deciphering the low magnetic field dynamics of moir minibands, we utilize a novel semi-classical calculation method, grounded in zero-field Fermi contours, to predict the nontrivial behavior of the Landau-level spectrum. This is compared with fully quantum simulations, enabling an in-depth and contrasted analysis of transport measurements in high-quality graphene-hBN superlattices. Our results not only highlight the primary observation of the open-orbit induced XMR in this system but also shed new light on other intricate phenomena. These include the nuances of single miniband dynamics, evident through Lifshitz transitions, and the complex interplay of semiclassical and quantum effects between these minibands. Specifically, we document transport anomalies linked to trigonal warping, a semiclassical deviation from the expected linear characteristics of Landau levels, and magnetic breakdown phenomena indicative of quantum tunneling, all effects jointly contributing to the intricacies of a rich electronic landscape uncovered at low magnetic fields.

cond-mat.mes-hall↗