arXiv Science⌕ Search

arXiv subjects

Meng Zhang

Publications and source records attributed to Meng Zhang.

At least 181 records · Page 10Linked to original sources

Wind Power Forecasting Considering Data Privacy Protection: A Federated Deep Reinforcement Learning Approach

In a modern power system with an increasing proportion of renewable energy, wind power prediction is crucial to the arrangement of power grid dispatching plans due to the volatility of wind power. However, traditional centralized forecasting methods raise concerns regarding data privacy-preserving and data islands problem. To handle the data privacy and openness, we propose a forecasting scheme that combines federated learning and deep reinforcement learning (DRL) for ultra-short-term wind power forecasting, called federated deep reinforcement learning (FedDRL). Firstly, this paper uses the deep deterministic policy gradient (DDPG) algorithm as the basic forecasting model to improve prediction accuracy. Secondly, we integrate the DDPG forecasting model into the framework of federated learning. The designed FedDRL can obtain an accurate prediction model in a decentralized way by sharing model parameters instead of sharing private data which can avoid sensitive privacy issues. The simulation results show that the proposed FedDRL outperforms the traditional prediction methods in terms of forecasting accuracy. More importantly, while ensuring the forecasting performance, FedDRL can effectively protect the data privacy and relieve the communication pressure compared with the traditional centralized forecasting method. In addition, a simulation with different federated learning parameters is conducted to confirm the robustness of the proposed scheme.

cs.LG↗

Generalization to the Natural Gradient Descent

Optimization problem, which is aimed at finding the global minimal value of a given cost function, is one of the central problem in science and engineering. Various numerical methods have been proposed to solve this problem, among which the Gradient Descent (GD) method is the most popular one due to its simplicity and efficiency. However, the GD method suffers from two main issues: the local minima and the slow convergence especially near the minima point. The Natural Gradient Descent(NGD), which has been proved as one of the most powerful method for various optimization problems in machine learning, tensor network, variational quantum algorithms and so on, supplies an efficient way to accelerate the convergence. Here, we give a unified method to extend the NGD method to a more general situation which keeps the fast convergence by looking for a more suitable metric through introducing a 'proper' reference Riemannian manifold. Our method generalizes the NDG, and may give more insight of the optimization methods.

math.OC↗

A quantum spin liquid phase in the Kitaev-Hubbard model

The quantum spin liquid (QSL) state has been searched intensively in Kitaev-like materials, such as the Iridium oxides $A_2$IrO$_3$ and $α$-RuCl$_3$. The half-filled Kitaev-Hubbard model with bond dependent hopping terms is used to describe the Kitaev-like materials, which is calculated using the state-of-the-art fermionic projected entangled pair states (fPEPS) method. We find a QSL phase near the Mott insulator transition, which has a strong first-order transition to the semi-metal phase with the decrease of Hubbard $U$. We suggest that a promising routine to find the QSL is to find the Iridium oxides that are near the Mott insulator transitions.

cond-mat.str-el↗

Superatom Orbitals, Orbital Splitting and Structure Prediction of Pure Alkali Metal Clusters

Jellium model achieved great success in predicting stable clusters with closed electronic shells and zero spin. In order to explain the stability of open shell clusters, it is necessary to consider the case of non-degenerate energy levels. In this paper the energy levels in nine low-lying Li$_{19}$ clusters are analysed systematically through superatomic orbital splitting effect. It is found that for originally degenerate orbitals like five 1D orbitals, the more the orbital extends in the direction of the cluster extension, the lower the energy of the orbital becomes. So oblate Li$_{19}$ clusters have the orbital sequence of $1\mathrm{S}^2-1\mathrm{P}_{x/y}^{4}-1\mathrm{P}_{z}^{2}-1\mathrm{D}_{xy/x^2-y^2}^{4}-2\mathrm{S}^2-1\mathrm{D}_{xz/yz}^{4}-1\mathrm{D}_{z^2}^{1}$, while prolate Li$_{19}$ clusters have the sequence of $1\mathrm{S}^2-1\mathrm{P}_{z}^{2}-1\mathrm{P}_{x/y}^{4}-1\mathrm{D}_{z^2}^{2}-1\mathrm{D}_{xz/yz}^{4}-1\mathrm{D}_{xy/x^2-y^2}^{4}-2\mathrm{S}^1$. This electron configuration is applied to predict the shape and magnetic moment of the alkali metal Li$_{n}$ clusters. The stability of the Li$_{14}$ cluster can be successfully interpreted in the framework of orbital splitting effect without resorting to the super valence bond (SVB) model, indicating a non-spherical cluster can achieve good stability without meeting the magic number. It is also proposed that the orbital splitting can be used to predict the shape (prolate, oblate or sphere) and magnetic moment of clusters. 11 out of 16 predicted shapes of Li$_n(n=3-18)$ are consistent with the results obtained by the principle of minimum energy.

cond-mat.mtrl-sci↗

Motion Guided Deep Dynamic 3D Garments

Realistic dynamic garments on animated characters have many AR/VR applications. While authoring such dynamic garment geometry is still a challenging task, data-driven simulation provides an attractive alternative, especially if it can be controlled simply using the motion of the underlying character. In this work, we focus on motion guided dynamic 3D garments, especially for loose garments. In a data-driven setup, we first learn a generative space of plausible garment geometries. Then, we learn a mapping to this space to capture the motion dependent dynamic deformations, conditioned on the previous state of the garment as well as its relative position with respect to the underlying body. Technically, we model garment dynamics, driven using the input character motion, by predicting per-frame local displacements in a canonical state of the garment that is enriched with frame-dependent skinning weights to bring the garment to the global space. We resolve any remaining per-frame collisions by predicting residual local displacements. The resultant garment geometry is used as history to enable iterative rollout prediction. We demonstrate plausible generalization to unseen body shapes and motion inputs, and show improvements over multiple state-of-the-art alternatives.

cs.CV↗

Age-Dependent Differential Privacy

The proliferation of real-time applications has motivated extensive research on analyzing and optimizing data freshness in the context of \textit{age of information}. However, classical frameworks of privacy (e.g., differential privacy (DP)) have overlooked the impact of data freshness on privacy guarantees, which may lead to unnecessary accuracy loss when trying to achieve meaningful privacy guarantees in time-varying databases. In this work, we introduce \textit{age-dependent DP}, taking into account the underlying stochastic nature of a time-varying database. In this new framework, we establish a connection between classical DP and age-dependent DP, based on which we characterize the impact of data staleness and temporal correlation on privacy guarantees. Our characterization demonstrates that \textit{aging}, i.e., using stale data inputs and/or postponing the release of outputs, can be a new strategy to protect data privacy in addition to noise injection in the traditional DP framework. Furthermore, to generalize our results to a multi-query scenario, we present a sequential composition result for age-dependent DP under any publishing and aging policies. We then characterize the optimal tradeoffs between privacy risk and utility and show how this can be achieved. Finally, case studies show that to achieve a target of an arbitrarily small privacy risk in a single-query case, combing aging and noise injection only leads to a bounded accuracy loss, whereas using noise injection only (as in the benchmark case of DP) will lead to an unbounded accuracy loss.

cs.CR↗

Experiments on Anomaly Detection in Autonomous Driving by Forward-Backward Style Transfers

Great progress has been achieved in the community of autonomous driving in the past few years. As a safety-critical problem, however, anomaly detection is a huge hurdle towards a large-scale deployment of autonomous vehicles in the real world. While many approaches, such as uncertainty estimation or segmentation-based image resynthesis, are extremely promising, there is more to be explored. Especially inspired by works on anomaly detection based on image resynthesis, we propose a novel approach for anomaly detection through style transfer. We leverage generative models to map an image from its original style domain of road traffic to an arbitrary one and back to generate pixelwise anomaly scores. However, our experiments have proven our hypothesis wrong, and we were unable to produce significant results. Nevertheless, we want to share our findings, so that others can learn from our experiments.

cs.CV↗

Universal Conditional Masked Language Pre-training for Neural Machine Translation

Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT). Different from prior works where pre-trained models usually adopt an unidirectional decoder, this paper demonstrates that pre-training a sequence-to-sequence model but with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT. Specifically, we propose CeMAT, a conditional masked language model pre-trained on large-scale bilingual and monolingual corpora in many languages. We also introduce two simple but effective methods to enhance the CeMAT, aligned code-switching & masking and dynamic dual-masking. We conduct extensive experiments and show that our CeMAT can achieve significant performance improvement for all scenarios from low- to extremely high-resource languages, i.e., up to +14.4 BLEU on low resource and +7.9 BLEU improvements on average for Autoregressive NMT. For Non-autoregressive NMT, we demonstrate it can also produce consistent performance gains, i.e., up to +5.3 BLEU. To the best of our knowledge, this is the first work to pre-train a unified model for fine-tuning on both NMT tasks. Code, data, and pre-trained models are available at https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/CeMAT.

cs.CL↗

Triangular Transfer: Freezing the Pivot for Triangular Machine Translation

Triangular machine translation is a special case of low-resource machine translation where the language pair of interest has limited parallel data, but both languages have abundant parallel data with a pivot language. Naturally, the key to triangular machine translation is the successful exploitation of such auxiliary data. In this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data. As we train auxiliary source-pivot and pivot-target translation models, we initialize some parameters of the pivot side with a pre-trained language model and freeze them to encourage both translation models to work in the same pivot language space, so that they can be smoothly transferred to the source-target translation model. Experiments show that our approach can outperform previous ones.

cs.CL↗

Dynamic Multi-Branch Layers for On-Device Neural Machine Translation

With the rapid development of artificial intelligence (AI), there is a trend in moving AI applications, such as neural machine translation (NMT), from cloud to mobile devices. Constrained by limited hardware resources and battery, the performance of on-device NMT systems is far from satisfactory. Inspired by conditional computation, we propose to improve the performance of on-device NMT systems with dynamic multi-branch layers. Specifically, we design a layer-wise dynamic multi-branch network with only one branch activated during training and inference. As not all branches are activated during training, we propose shared-private reparameterization to ensure sufficient training for each branch. At almost the same computational cost, our method achieves improvements of up to 1.7 BLEU points on the WMT14 English-German translation task and 1.8 BLEU points on the WMT20 Chinese-English translation task over the Transformer model, respectively. Compared with a strong baseline that also uses multiple branches, the proposed method is up to 1.5 times faster with the same number of parameters.

cs.CL↗

A review of data-driven short-term voltage stability assessment of power systems: Concept, principle, and challenges

With the rapid growth of power market reform and power demand, the power transmission capacity of a power grid is approaching its limit, and the secure and stable operation of power systems becomes increasingly important. In particular, in modern power grids, the proportion of dynamic loads with fast recovery characteristics such as air conditioners, refrigerators, and industrial motors is increasing. As well as the increasing proportion of different forms of renewable energy in power systems. Therefore, short-term voltage stability (STVS) of power systems cannot be ignored. This article comprehensively sorts out the STVS problems of power systems from the perspective of data-driven methods and discusses existing challenges.

eess.SY↗

SimulSLT: End-to-End Simultaneous Sign Language Translation

Sign language translation as a kind of technology with profound social significance has attracted growing researchers' interest in recent years. However, the existing sign language translation methods need to read all the videos before starting the translation, which leads to a high inference latency and also limits their application in real-life scenarios. To solve this problem, we propose SimulSLT, the first end-to-end simultaneous sign language translation model, which can translate sign language videos into target text concurrently. SimulSLT is composed of a text decoder, a boundary predictor, and a masked encoder. We 1) use the wait-k strategy for simultaneous translation. 2) design a novel boundary predictor based on the integrate-and-fire module to output the gloss boundary, which is used to model the correspondence between the sign language video and the gloss. 3) propose an innovative re-encode method to help the model obtain more abundant contextual information, which allows the existing video features to interact fully. The experimental results conducted on the RWTH-PHOENIX-Weather 2014T dataset show that SimulSLT achieves BLEU scores that exceed the latest end-to-end non-simultaneous sign language translation model while maintaining low latency, which proves the effectiveness of our method.

cs.CV↗

Inhibition of Current-spike Formation Based on Longitudinal Phase Space Manipulation for High-Repetition-Rate X-ray FEL

The formation of a double-horn current profile is a challenging issue in the achievement of electron bunch with high peak current, especially for high-repetition-rate X-ray free-electron lasers (XFELs) where more nonlinear and intricate beam dynamics propagation is involved. Here, we propose to correct the nonlinear beam longitudinal phase space distortion in the photoinjector section with a dual-mode buncher. In combination with the evolutionary many-objective beam dynamics optimization, this method is shown to be effective in manipulating the longitudinal phase space at the injector exit. Furthermore, the formation of the current spike is avoided after the multi-stage charge density modulation and electron bunch with a peak current of 1.6 kA is achieved for 100-pC bunch charge.Start-to-end simulations based on the Shanghai high-repetition-rate XFEL and extreme light facility demonstrate that the proposed scheme can increase the FEL pulse energy by more than 3 times in a cascading operation of echo-enabled harmonic generation and high-gain harmonic generation. Moreover, this method can also be used for longitudinal phase space shaping of electron beams operating at a high repetition rate to meet the specific demands of different researches.

physics.acc-ph↗

A Deep-Learning Intelligent System Incorporating Data Augmentation for Short-Term Voltage Stability Assessment of Power Systems

Facing the difficulty of expensive and trivial data collection and annotation, how to make a deep learning-based short-term voltage stability assessment (STVSA) model work well on a small training dataset is a challenging and urgent problem. Although a big enough dataset can be directly generated by contingency simulation, this data generation process is usually cumbersome and inefficient; while data augmentation provides a low-cost and efficient way to artificially inflate the representative and diversified training datasets with label preserving transformations. In this respect, this paper proposes a novel deep-learning intelligent system incorporating data augmentation for STVSA of power systems. First, due to the unavailability of reliable quantitative criteria to judge the stability status for a specific power system, semi-supervised cluster learning is leveraged to obtain labeled samples in an original small dataset. Second, to make deep learning applicable to the small dataset, conditional least squares generative adversarial networks (LSGAN)-based data augmentation is introduced to expand the original dataset via artificially creating additional valid samples. Third, to extract temporal dependencies from the post-disturbance dynamic trajectories of a system, a bi-directional gated recurrent unit with attention mechanism based assessment model is established, which bi-directionally learns the significant time dependencies and automatically allocates attention weights. The test results demonstrate the presented approach manages to achieve better accuracy and a faster response time with original small datasets. Besides classification accuracy, this work employs statistical measures to comprehensively examine the performance of the proposal.

cs.LG↗

Faithful Edge Federated Learning: Scalability and Privacy

Federated learning enables machine learning algorithms to be trained over a network of multiple decentralized edge devices without requiring the exchange of local datasets. Successfully deploying federated learning requires ensuring that agents (e.g., mobile devices) faithfully execute the intended algorithm, which has been largely overlooked in the literature. In this study, we first use risk bounds to analyze how the key feature of federated learning, unbalanced and non-i.i.d. data, affects agents' incentives to voluntarily participate and obediently follow traditional federated learning algorithms. To be more specific, our analysis reveals that agents with less typical data distributions and relatively more samples are more likely to opt out of or tamper with federated learning algorithms. To this end, we formulate the first faithful implementation problem of federated learning and design two faithful federated learning mechanisms which satisfy economic properties, scalability, and privacy. Further, the time complexity of computing all agents' payments in the number of agents is $\mathcal{O}(1)$. First, we design a Faithful Federated Learning (FFL) mechanism which approximates the Vickrey-Clarke-Groves (VCG) payments via an incremental computation. We show that it achieves (probably approximate) optimality, faithful implementation, voluntary participation, and some other economic properties (such as budget balance). Second, by partitioning agents into several subsets, we present a scalable VCG mechanism approximation. We further design a scalable and Differentially Private FFL (DP-FFL) mechanism, the first differentially private faithful mechanism, that maintains the economic properties. Our mechanism enables one to make three-way performance tradeoffs among privacy, the iterations needed, and payment accuracy loss.

cs.LG↗

A Review of Deep Reinforcement Learning for Smart Building Energy Management

Global buildings account for about 30% of the total energy consumption and carbon emission, raising severe energy and environmental concerns. Therefore, it is significant and urgent to develop novel smart building energy management (SBEM) technologies for the advance of energy-efficient and green buildings. However, it is a nontrivial task due to the following challenges. Firstly, it is generally difficult to develop an explicit building thermal dynamics model that is both accurate and efficient enough for building control. Secondly, there are many uncertain system parameters (e.g., renewable generation output, outdoor temperature, and the number of occupants). Thirdly, there are many spatially and temporally coupled operational constraints. Fourthly, building energy optimization problems can not be solved in real-time by traditional methods when they have extremely large solution spaces. Fifthly, traditional building energy management methods have respective applicable premises, which means that they have low versatility when confronted with varying building environments. With the rapid development of Internet of Things technology and computation capability, artificial intelligence technology find its significant competence in control and optimization. As a general artificial intelligence technology, deep reinforcement learning (DRL) is promising to address the above challenges. Notably, the recent years have seen the surge of DRL for SBEM. However, there lacks a systematic overview of different DRL methods for SBEM. To fill the gap, this paper provides a comprehensive review of DRL for SBEM from the perspective of system scale. In particular, we identify the existing unresolved issues and point out possible future research directions.

eess.SY↗

Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training

Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model's uncertainty on a small set of trusted clean data for multi-corpus machine translation. We experiments with two classes of uncertainty measures on multilingual (16 languages with 4 settings) and multi-domain settings (4 for in-domain and 2 for out-of-domain on English-German translation) and demonstrate our approach MultiUAT substantially outperforms its baselines, including both static and dynamic strategies. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.

cs.CL↗

Most "young" $α$-rich stars have high masses but are actually old

Recent observations have revealed a population of $α$-element abundances enhanced giant stars with unexpected high masses ($\gtrsim$1 $M_\odot$) from asteroseismic analysis and spectroscopy. Assuming single-star evolution, their masses imply young ages ($τ<6$Gyr) incompatible with the canonical Galactic chemical evolution scenario. Here we study the chemistry and kinematics of a large sample of such $α$-rich, high-mass red giant branch (RGB) stars drawn from the LAMOST spectroscopic surveys. Using LAMOST and Gaia, we found these stars share the same kinematics as the canonical high-$α$ old stellar population in the Galactic thick disk. The stellar abundances show that these high-$α$ massive stars have $α$- and iron-peak element abundances similar to those of the high-$α$ old thick disk stars. However, a portion of them exhibit higher [(N+C)/Fe] and [Ba/Fe] ratios, which implies they have gained C- and Ba-rich materials from extra sources, presumably asymptotic giant branch (AGB) companions. The results support the previous suggestion that these RGB stars are products of binary evolution. Their high masses thus mimic "young" single stars, yet in fact they belong to an intrinsic old stellar population. To fully explain the stellar abundance patterns of our sample stars, a variety of binary evolution channels, such as, main-sequence (MS) + RGB, MS + AGB, RGB + RGB and RGB + AGB, are required, pointing to diverse formation mechanisms of these seemly rejuvenated cannibals. With this larger sample, our results confirm earlier findings that most, if not all, $α$-rich stars in the Galactic disk seem to be old.

astro-ph.GA↗