arXiv Science⌕ Search

arXiv subjects

Zhenhe Zhang

Publications and source records attributed to Zhenhe Zhang.

10 recordsLinked to original sources

Decentralized Dynamic Stable Matching

We study decentralized computation of the player-optimal stable matching (POSM) with unannounced player arrivals and departures. Without a common clock, event information, or direct communication, players propose to arms and observe only their own accept/reject outcomes. Mismatch cost counts rounds when the realized matching differs from the current POSM. For known preferences, we give a deterministic polynomial-time self-calibrating protocol with cost $O(t_*+K^2\log T)$, improving the previous bound $O(T^{1-1/K}(\log T)^{2/K})$. Here $K$ is the number of arms, $t_*$ the last event time, and $T$ the horizon. Players use only local time and proposal outcomes, knowing neither $t_*$ nor $T$. For any nondecreasing, polynomial-time-computable $g(t)=Ω(t)$, the protocol achieves $O(g(t_*)+K^2N_g(T))$. Here $g$ governs recovery time, while $N_g(T)$ counts iterations of $t\mapsto t+g(t)$ below $T$, exposing a tradeoff between recovery time and asymptotic mismatch cost. Any randomized protocol with recovery scale $g(t)\ge1$ and sublinear asymptotic mismatch cost has worst-case expected cost $Ω(t_*+N_g(T))$. Thus, with $g(t)=t$, our protocol is optimal up to a $K^2$ factor among sublinear-cost protocols with $O(t_*)$ recovery time. For unknown preferences on both sides, each player and arm learns its preferences from its own observations after matching. We give a decentralized learning protocol with expected cost $O(t_*+K^4Δ_{\mathrm{min}}^{-2}\log T)$, where $Δ_{\mathrm{min}}$ measures preference gaps. A complementary static-setting lower bound adapted from the literature shows that the dependence on $T$ and $Δ_{\mathrm{min}}$ is tight.

cs.DC↗

Blockchain Participation Games with Upfront Contribution Commitment

Blockchain participation games combine costly entry, random participation selection, and threshold success into consideration. Their retraction variants charge task-completion costs only to selected contributors. We consider committee tasks whose operators reserve costly computing or communication capacity before assignment, so preparation costs are incurred even when that capacity is not selected. We characterize all pure Nash equilibria under homogeneous selection and analyze how heterogeneous selection shapes who commits. We characterize how the homogeneous minimum-outlay reward scheme supports full contribution, where every user commits, as a Nash equilibrium. Moreover, under heterogeneous selection, requiring zero registration reward can increase the minimum outlay. Finally, when full contribution is a positive-welfare social optimum, the homogeneous zero-registration-reward design attains price of stability one; and under independent Bernoulli selection, every full-contribution Nash equilibrium is also a strong equilibrium.

cs.GT↗

Data Unlearning via Inverse Distillation

Multi-step matching models, including flow and diffusion models, produce high-quality outputs but incur substantial inference costs and may reproduce unwanted components of their training datasets. We introduce Inverse Distillation Unlearning (IDU), a unified framework that simultaneously distills a teacher multi-step matching model into an efficient one-step student generator and suppresses outputs corresponding to a designated training subset. We first formulate distillation as a min-max objective over a data distribution and then represent this distribution as a mixture of the forget-set and the generated distributions. This allows us to compare this mixture with the teacher's training distribution and recover only the retained data at the optimum. Our method requires only a pretrained full-data teacher and data from the forget set, without access to retained training examples, extra feature extractors or classifiers. Extensive experiments on MNIST and CIFAR-10 datasets under flow-matching and score-based diffusion settings demonstrate that IDU substantially reduces the generation frequency of forgotten classes while preserving generation quality on the retained classes. To the best of our knowledge, IDU is the first unified framework for simultaneous unlearning and distillation in unconditional flow-matching and score-based models.

cs.LG↗

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs

We present Pangu Ultra, a Large Language Model (LLM) with 135 billion parameters and dense Transformer modules trained on Ascend Neural Processing Units (NPUs). Although the field of LLM has been witnessing unprecedented advances in pushing the scale and capability of LLM in recent years, training such a large-scale model still involves significant optimization and system challenges. To stabilize the training process, we propose depth-scaled sandwich normalization, which effectively eliminates loss spikes during the training process of deep models. We pre-train our model on 13.2 trillion diverse and high-quality tokens and further enhance its reasoning capabilities during post-training. To perform such large-scale training efficiently, we utilize 8,192 Ascend NPUs with a series of system optimizations. Evaluations on multiple diverse benchmarks indicate that Pangu Ultra significantly advances the state-of-the-art capabilities of dense LLMs such as Llama 405B and Mistral Large 2, and even achieves competitive results with DeepSeek-R1, whose sparse model structure contains much more parameters. Our exploration demonstrates that Ascend NPUs are capable of efficiently and effectively training dense models with more than 100 billion parameters. Our model and system will be available for our commercial customers.

cs.CL↗

AIGS: Generating Science from AI-Powered Automated Falsification

Rapid development of artificial intelligence has drastically accelerated the development of scientific discovery. Trained with large-scale observation data, deep neural networks extract the underlying patterns in an end-to-end manner and assist human researchers with highly-precised predictions in unseen scenarios. The recent rise of Large Language Models (LLMs) and the empowered autonomous agents enable scientists to gain help through interaction in different stages of their research, including but not limited to literature review, research ideation, idea implementation, and academic writing. However, AI researchers instantiated by foundation model empowered agents with full-process autonomy are still in their infancy. In this paper, we study $\textbf{AI-Generated Science}$ (AIGS), where agents independently and autonomously complete the entire research process and discover scientific laws. By revisiting the definition of scientific research, we argue that $\textit{falsification}$ is the essence of both human research process and the design of an AIGS system. Through the lens of falsification, prior systems attempting towards AI-Generated Science either lack the part in their design, or rely heavily on existing verification engines that narrow the use in specialized domains. In this work, we propose Baby-AIGS as a baby-step demonstration of a full-process AIGS system, which is a multi-agent system with agents in roles representing key research process. By introducing FalsificationAgent, which identify and then verify possible scientific discoveries, we empower the system with explicit falsification. Experiments on three tasks preliminarily show that Baby-AIGS could produce meaningful scientific discoveries, though not on par with experienced human researchers. Finally, we discuss on the limitations of current Baby-AIGS, actionable insights, and related ethical issues in detail.

cs.LG↗

PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs

While Large language models (LLMs) have demonstrated considerable capabilities across various natural language tasks, they often fall short of the performance achieved by domain-specific state-of-the-art models. One potential approach to enhance domain-specific capabilities of LLMs involves fine-tuning them using corresponding datasets. However, this method can be both resource and time-intensive, and not applicable to closed-source commercial LLMs. In this paper, we propose Preference Adaptation for Enhancing Domain-specific Abilities of LLMs (PANDA), a method designed to augment the domain-specific capabilities of LLMs by leveraging insights from the response preference of expert models without requiring fine-tuning. Our experimental results reveal that PANDA significantly enhances the domain-specific ability of LLMs on text classification and interactive decision tasks. Moreover, LLM with PANDA even outperforms the expert model that being learned on 4 tasks of ScienceWorld. This finding highlights the potential of exploring tuning-free approaches to achieve weak-to-strong generalization.

cs.CL↗

Towards Unified Alignment Between Agents, Humans, and Environment

The rapid progress of foundation models has led to the prosperity of autonomous agents, which leverage the universal capabilities of foundation models to conduct reasoning, decision-making, and environmental interaction. However, the efficacy of agents remains limited when operating in intricate, realistic environments. In this work, we introduce the principles of $\mathbf{U}$nified $\mathbf{A}$lignment for $\mathbf{A}$gents ($\mathbf{UA}^2$), which advocate for the simultaneous alignment of agents with human intentions, environmental dynamics, and self-constraints such as the limitation of monetary budgets. From the perspective of $\mathbf{UA}^2$, we review the current agent research and highlight the neglected factors in existing agent benchmarks and method candidates. We also conduct proof-of-concept studies by introducing realistic features to WebShop, including user profiles to demonstrate intentions, personalized reranking for complex environmental dynamics, and runtime cost statistics to reflect self-constraints. We then follow the principles of $\mathbf{UA}^2$ to propose an initial design of our agent, and benchmark its performance with several candidate baselines in the retrofitted WebShop. The extensive experimental results further prove the importance of the principles of $\mathbf{UA}^2$. Our research sheds light on the next steps of autonomous agent research with improved general problem-solving abilities.

cs.AI↗

Power Tracking Control of Heterogeneous Populations of TCLs with Partially Measured States

This paper presents a new aggregate power tracking control scheme for populations of thermostatically controlled loads (TCLs). The control design is performed in the framework of partial differential equations (PDEs) based on a late-lumping procedure without truncating the infinite-dimensional model describing the dynamics of the TCL population. An input-output linearization control scheme, which is independent of system parameters and uses only partial state measurement, is derived, and a sliding model-like control is applied to achieve finite-time input-to-state stability for tracking error dynamics. Such a control strategy can ensure robust performance in the presence of modeling uncertainties, while considerably reducing the communication burden in large scale distributed systems similar to that considered in the present work. A rigorous analysis of the closed-loop stability of the underlying PDE system was conducted, which guaranteed the validity of the developed control scheme. Simulation studies were performed while considering two TCL populations with a significant difference in their size, and the results show that the developed control scheme performs well in both cases, thereby confirming the effectiveness of the proposed solution.

eess.SY↗

Power Tracking Control of Heterogeneous TCL Populations with Modeling Uncertainties and Communication Restrictions

This paper presents a new aggregate power tracking control scheme for populations of thermostatically controlled loads (TCLs). The control design is carried out in the framework of partial differential equations (PDEs) based on a late-lumping procedure without truncating the infinite-dimensional model describing the dynamics of the TCL population. An input-output linearization control scheme, which is independent of the system parameters and uses only partial state measurement, is derived, and a sliding model control is applied, which allows achieving a finite-time input-to-state stability for the tracking error dynamics. Such a control strategy can ensure a robust performance in the presence of modeling uncertainties while considerably reducing the communication burden in large scale distributed systems as the one considered in the present work. To guarantee the validity of the developed control scheme, a rigourous analysis on the solutions to the underlying PDE is conducted. Two implementations of the proposed control strategy, based on discrete-time approximation and fuzzy logic control, respectively, are validated through simulation studies.

math.OC↗

SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Training

We introduce a new approach for speech pre-training named SPIRAL which works by learning denoising representation of perturbed data in a teacher-student framework. Specifically, given a speech utterance, we first feed the utterance to a teacher network to obtain corresponding representation. Then the same utterance is perturbed and fed to a student network. The student network is trained to output representation resembling that of the teacher. At the same time, the teacher network is updated as moving average of student's weights over training steps. In order to prevent representation collapse, we apply an in-utterance contrastive loss as pre-training objective and impose position randomization on the input to the teacher. SPIRAL achieves competitive or better results compared to state-of-the-art speech pre-training method wav2vec 2.0, with significant reduction of training cost (80% for BASE model, 65% for LARGE model). Furthermore, we address the problem of noise-robustness that is critical to real-world speech applications. We propose multi-condition pre-training by perturbing the student's input with various types of additive noise. We demonstrate that multi-condition pre-trained SPIRAL models are more robust to noisy speech (9.0% - 13.3% relative word error rate reduction on real noisy test data), compared to applying multi-condition training solely in the fine-tuning stage. Source code is available at https://github.com/huawei-noah/Speech-Backbones/tree/main/SPIRAL.

eess.AS↗