arXiv subjects

Rui Yang

Publications and source records attributed to Rui Yang.

At least 271 records · Page 15Linked to original sources

arXiv2023

Generalization in Visual Reinforcement Learning with the Reward Sequence Distribution

Generalization in partially observed markov decision processes (POMDPs) is critical for successful applications of visual reinforcement learning (VRL) in real scenarios. A widely used idea is to learn task-relevant representations that encode task-relevant information of common features in POMDPs, i.e., rewards and transition dynamics. As transition dynamics in the latent state space -- which are task-relevant and invariant to visual distractions -- are unknown to the agents, existing methods alternatively use transition dynamics in the observation space to extract task-relevant information in transition dynamics. However, such transition dynamics in the observation space involve task-irrelevant visual distractions, degrading the generalization performance of VRL methods. To tackle this problem, we propose the reward sequence distribution conditioned on the starting observation and the predefined subsequent action sequence (RSD-OA). The appealing features of RSD-OA include that: (1) RSD-OA is invariant to visual distractions, as it is conditioned on the predefined subsequent action sequence without task-irrelevant information from transition dynamics, and (2) the reward sequence captures long-term task-relevant information in both rewards and transition dynamics. Experiments demonstrate that our representation learning approach based on RSD-OA significantly improves the generalization performance on unseen environments, outperforming several state-of-the-arts on DeepMind Control tasks with visual distractions.

cs.LG↗

arXiv2023

Ice melting in salty water: layering and non-monotonic dependence on the mean salinity

The presence of salt in ocean water strongly affects the melt rate and the shape evolution of ice, both of utmost relevance in geophysical and ocean flow and thus for the climate. To get a better quantitative understanding of the physical mechanics at play in ice melting in salty water, we numerically investigate the lateral melting of an ice block in stably stratified saline water, using a realistic, nonlinear equation of state (EOS). The developing ice shape from our numerical results shows good agreement with the experiments and theory from Huppert & Turner (J. Fluid Mech. 100, 367 (1980)). Furthermore, we find that the melt rate of ice depends non-monotonically on the mean ambient salinity: It first decreases for increasing salt concentration until a local minimum is attained, and then increases again. This non-monotonic behavior of the ice melt rate is due to the competition among salinity-driven buoyancy, temperature-driven buoyancy, and salinity-induced stratification. We develop a theoretical model based on the energy balance which gives a prediction of the salt concentration for which the melt rate is minimal, and is consistent with our data. Our findings give insight into the interplay between phase transitions and double-diffusive convective flows.

physics.flu-dyn↗

arXiv2022

Learning-based Predictive Path Following Control for Nonlinear Systems Under Uncertain Disturbances

Accurate path following is challenging for autonomous robots operating in uncertain environments. Adaptive and predictive control strategies are crucial for a nonlinear robotic system to achieve high-performance path following control. In this paper, we propose a novel learning-based predictive control scheme that couples a high-level model predictive path following controller (MPFC) with a low-level learning-based feedback linearization controller (LB-FBLC) for nonlinear systems under uncertain disturbances. The low-level LB-FBLC utilizes Gaussian Processes to learn the uncertain environmental disturbances online and tracks the reference state accurately with a probabilistic stability guarantee. Meanwhile, the high-level MPFC exploits the linearized system model augmented with a virtual linear path dynamics model to optimize the evolution of path reference targets, and provides the reference states and controls for the low-level LB-FBLC. Simulation results illustrate the effectiveness of the proposed control strategy on a quadrotor path following task under unknown wind disturbances.

cs.RO↗

arXiv2022

RDP-Net: Region Detail Preserving Network for Change Detection

Change detection (CD) is an essential earth observation technique. It captures the dynamic information of land objects. With the rise of deep learning, convolutional neural networks (CNN) have shown great potential in CD. However, current CNN models introduce backbone architectures that lose detailed information during learning. Moreover, current CNN models are heavy in parameters, which prevents their deployment on edge devices such as UAVs. In this work, we tackle this issue by proposing RDP-Net: a region detail preserving network for CD. We propose an efficient training strategy that constructs the training tasks during the warmup period of CNN training and lets the CNN learn from easy to hard. The training strategy enables CNN to learn more powerful features with fewer FLOPs and achieve better performance. Next, we propose an effective edge loss that increases the penalty for errors on details and improves the network's attention to details such as boundary regions and small areas. Furthermore, we provide a CNN model with a brand new backbone that achieves the state-of-the-art empirical performance in CD with only 1.70M parameters. We hope our RDP-Net would benefit the practical CD applications on compact devices and could inspire more people to bring change detection to a new level with the efficient training strategy. The code and models are publicly available at https://github.com/Chnja/RDPNet.

eess.IV↗

arXiv2022

On Distribution Grid Optimal Power Flow Development and Integration

Due to changes in electric distribution grid operation, new operation regimes have been recommended. Distribution grid optimal power flow (DOPF) has received tremendous attention in the research community, yet it has not been fully adopted across the utility industry. Our paper recognizes this problem and suggests a development and integration procedure for DOPF. We propose development of DOPF as a three step procedure of 1) processing the grid, 2) obtaining a tractable solution, and 3) implementing multiple solution algorithms and benchmarking them to improve application reliability. For the integration of DOPF, we demonstrate how a DOPF federate may be developed that can be integrated in a co-simulation environment to mimic the real-world conditions and hence improve its practicality to be deployed in the field. To demonstrate the efficacy of the proposed methods, tests on IEEE 123-bus system are performed where the usage of tractable formulation in DOPF algorithm development and its comparison to the benchmark solution are demonstrated.

eess.SY↗

arXiv2022

Dual adaptive training of photonic neural networks

Photonic neural network (PNN) is a remarkable analog artificial intelligence (AI) accelerator that computes with photons instead of electrons to feature low latency, high energy efficiency, and high parallelism. However, the existing training approaches cannot address the extensive accumulation of systematic errors in large-scale PNNs, resulting in a significant decrease in model performance in physical systems. Here, we propose dual adaptive training (DAT) that allows the PNN model to adapt to substantial systematic errors and preserves its performance during the deployment. By introducing the systematic error prediction networks with task-similarity joint optimization, DAT achieves the high similarity mapping between the PNN numerical models and physical systems and high-accurate gradient calculations during the dual backpropagation training. We validated the effectiveness of DAT by using diffractive PNNs and interference-based PNNs on image classification tasks. DAT successfully trained large-scale PNNs under major systematic errors and preserved the model classification accuracies comparable to error-free systems. The results further demonstrated its superior performance over the state-of-the-art in situ training approaches. DAT provides critical support for constructing large-scale PNNs to achieve advanced architectures and can be generalized to other types of AI systems with analog computing errors.

cs.LG↗

arXiv2022

Nanomechanical Resonators: Toward Atomic Scale

The quest for realizing and manipulating ever smaller man-made movable structures and dynamical machines has spurred tremendous endeavors, led to important discoveries, and inspired researchers to venture to new grounds. Scientific feats and technological milestones of miniaturization of mechanical structures have been widely accomplished by advances in machining and sculpturing ever shrinking features out of bulk materials such as silicon. With the flourishing multidisciplinary field of low-dimensional nanomaterials, including one-dimensional (1D) nanowires/nanotubes, and two-dimensional (2D) atomic layers such as graphene/phosphorene, growing interests and sustained efforts have been devoted to creating mechanical devices toward the ultimate limit of miniaturization--genuinely down to the molecular or even atomic scale. These ultrasmall movable structures, particularly nanomechanical resonators that exploit the vibratory motion in these 1D and 2D nano-to-atomic-scale structures, offer exceptional device-level attributes, such as ultralow mass, ultrawide frequency tuning range, broad dynamic range, and ultralow power consumption, thus holding strong promises for both fundamental studies and engineering applications. In this Review, we offer a comprehensive overview and summary of this vibrant field, present the state-of-the-art devices and evaluate their specifications and performance, outline important achievements, and postulate future directions for studying these miniscule yet intriguing molecular-scale machines.

cond-mat.mes-hall↗

arXiv2022

ReLoc: A Restoration-Assisted Framework for Robust Image Tampering Localization

With the spread of tampered images, locating the tampered regions in digital images has drawn increasing attention. The existing image tampering localization methods, however, suffer from severe performance degradation when the tampered images are subjected to some post-processing, as the tampering traces would be distorted by the post-processing operations. The poor robustness against post-processing has become a bottleneck for the practical applications of image tampering localization techniques. In order to address this issue, this paper proposes a novel restoration-assisted framework for image tampering localization (ReLoc). The ReLoc framework mainly consists of an image restoration module and a tampering localization module. The key idea of ReLoc is to use the restoration module to recover a high-quality counterpart of the distorted tampered image, such that the distorted tampering traces can be re-enhanced, facilitating the tampering localization module to identify the tampered regions. To achieve this, the restoration module is optimized not only with the conventional constraints on image visual quality but also with a forensics-oriented objective function. Furthermore, the restoration module and the localization module are trained alternately, which can stabilize the training process and is beneficial for improving the performance. The proposed framework is evaluated by fighting against JPEG compression, the most commonly used post-processing. Extensive experimental results show that ReLoc can significantly improve the robustness against JPEG compression. The restoration module in a well-trained ReLoc model is transferable. Namely, it is still effective when being directly deployed with another tampering localization module.

cs.CV↗

arXiv2022

Latent Neural ODE for Integrating Multi-timescale measurements in Smart Distribution Grids

Under a smart grid paradigm, there has been an increase in sensor installations to enhance situational awareness. The measurements from these sensors can be leveraged for real-time monitoring, control, and protection. However, these measurements are typically irregularly sampled. These measurements may also be intermittent due to communication bandwidth limitations. To tackle this problem, this paper proposes a novel latent neural ordinary differential equations (LODE) approach to aggregate the unevenly sampled multivariate time-series measurements. The proposed approach is flexible in performing both imputations and predictions while being computationally efficient. Simulation results on IEEE 37 bus test systems illustrate the efficiency of the proposed approach.

eess.SY↗

arXiv2022

RORL: Robust Offline Reinforcement Learning via Conservative Smoothing

Offline reinforcement learning (RL) provides a promising direction to exploit massive amount of offline data for complex decision-making tasks. Due to the distribution shift issue, current offline RL algorithms are generally designed to be conservative in value estimation and action selection. However, such conservatism can impair the robustness of learned policies when encountering observation deviation under realistic conditions, such as sensor errors and adversarial attacks. To trade off robustness and conservatism, we propose Robust Offline Reinforcement Learning (RORL) with a novel conservative smoothing technique. In RORL, we explicitly introduce regularization on the policy and the value function for states near the dataset, as well as additional conservative value estimation on these states. Theoretically, we show RORL enjoys a tighter suboptimality bound than recent theoretical results in linear MDPs. We demonstrate that RORL can achieve state-of-the-art performance on the general offline RL benchmark and is considerably robust to adversarial observation perturbations.

cs.LG↗

arXiv2022

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

In this work, we study the simple yet universally applicable case of reward shaping in value-based Deep Reinforcement Learning (DRL). We show that reward shifting in the form of the linear transformation is equivalent to changing the initialization of the $Q$-function in function approximation. Based on such an equivalence, we bring the key insight that a positive reward shifting leads to conservative exploitation, while a negative reward shifting leads to curiosity-driven exploration. Accordingly, conservative exploitation improves offline RL value estimation, and optimistic value estimation improves exploration for online RL. We validate our insight on a range of RL tasks and show its improvement over baselines: (1) In offline RL, the conservative exploitation leads to improved performance based on off-the-shelf algorithms; (2) In online continuous control, multiple value functions with different shifting constants can be used to tackle the exploration-exploitation dilemma for better sample efficiency; (3) In discrete control tasks, a negative reward shifting yields an improvement over the curiosity-based exploration method.

cs.LG↗

arXiv2022

Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learning

Multi-goal reinforcement learning is widely applied in planning and robot manipulation. Two main challenges in multi-goal reinforcement learning are sparse rewards and sample inefficiency. Hindsight Experience Replay (HER) aims to tackle the two challenges via goal relabeling. However, HER-related works still need millions of samples and a huge computation. In this paper, we propose Multi-step Hindsight Experience Replay (MHER), incorporating multi-step relabeled returns based on $n$-step relabeling to improve sample efficiency. Despite the advantages of $n$-step relabeling, we theoretically and experimentally prove the off-policy $n$-step bias introduced by $n$-step relabeling may lead to poor performance in many environments. To address the above issue, two bias-reduced MHER algorithms, MHER($λ$) and Model-based MHER (MMHER) are presented. MHER($λ$) exploits the $λ$ return while MMHER benefits from model-based value expansions. Experimental results on numerous multi-goal robotic tasks show that our solutions can successfully alleviate off-policy $n$-step bias and achieve significantly higher sample efficiency than HER and Curriculum-guided HER with little additional computation beyond HER.

cs.LG↗

arXiv2022

Over-the-Air Computation over Balanced Numerals

In this study, a digital over-the-air computation (OAC) scheme for achieving continuous-valued gradient aggregation is proposed. It is shown that the average of a set of real-valued parameters can be calculated approximately by using the average of the corresponding numerals, where the numerals are obtained based on a balanced number system. By using this property, the proposed scheme encodes the local gradients into a set of numerals. It then determines the positions of the activated orthogonal frequency division multiplexing (OFDM) subcarriers by using the values of the numerals. To eliminate the need for a precise sample-level time synchronization, channel estimation overhead, and power instabilities due to the channel inversion, the proposed scheme also uses a non-coherent receiver at the edge server (ES) and does not utilize a pre-equalization at the edge devices (EDs). Finally, the theoretical mean squared error (MSE) performance of the proposed scheme is derived and its performance for federated edge learning (FEEL) is demonstrated.

eess.SP↗

arXiv2022

UniInst: Unique Representation for End-to-End Instance Segmentation

Existing instance segmentation methods have achieved impressive performance but still suffer from a common dilemma: redundant representations (e.g., multiple boxes, grids, and anchor points) are inferred for one instance, which leads to multiple duplicated predictions. Thus, mainstream methods usually rely on a hand-designed non-maximum suppression (NMS) post-processing step to select the optimal prediction result, which hinders end-to-end training. To address this issue, we propose a box-free and NMS-free end-to-end instance segmentation framework, termed UniInst, that yields only one unique representation for each instance. Specifically, we design an instance-aware one-to-one assignment scheme, namely Only Yield One Representation (OYOR), which dynamically assigns one unique representation to each instance according to the matching quality between predictions and ground truths. Then, a novel prediction re-ranking strategy is elegantly integrated into the framework to address the misalignment between the classification score and the mask quality, enabling the learned representation to be more discriminative. With these techniques, our UniInst, the first FCN-based box-free and NMS-free instance segmentation framework, achieves competitive performance, e.g., 39.0 mask AP using ResNet-50-FPN and 40.2 mask AP using ResNet-101-FPN, against mainstream methods on COCO test-dev 2017. Moreover, the proposed instance-aware method is robust to occlusion scenes, outperforming common baselines by remarkable mask AP on the heavily-occluded OCHuman benchmark. Code is available at https://github.com/b03505036/UniInst.

cs.CV↗

arXiv2022

Resilience assessment and improvement for electric power transmission systems against typhoon disasters: A data-model hybrid driven approach

In response to the damage to electric power transmission systems caused by typhoon disasters in coastal areas, a planning-targeted resilience assessment framework that considers the impact of multiple factors is established to accurately find the weak links of the transmission system and improve the system resilience. Firstly, this paper constructs the attenuation model of the wind field and the comprehensive failure model of the system, in which the model drive to establish the cumulative failure model of the transmission system is adopted, and multiple data-driven schemes to correct the cumulative failure rate by relying on the feature factor information of the transmission system is adopted. At the same time, an analytic hierarchy process-weighted arithmetic averaging (AHP-WAA) method is introduced to select the optimal data-driven evaluation scheme. Secondly, this paper adopts the impact-increment-based state enumeration (IISE) method to establish resilience indicators for systems and corridors separately. On this basis, the optimal promotion strategy is selected according to the construction difficulty, resilience improvement ability, and cost analysis. Finally, simulations on the IEEE RTS-79 system have been carried out considering the influence of the real typhoon scenarios, micro-topographic and transmission corridor information factors. The results demonstrate that the hybrid-driven system resilience assessment and improvement method can assist planners in accurately judging the resilience level of the system against typhoon disasters and selecting the best resilience improvement strategy based on the cost-effectiveness ratio.

eess.SY↗

arXiv2022

ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer

The vanilla self-attention mechanism inherently relies on pre-defined and steadfast computational dimensions. Such inflexibility restricts it from possessing context-oriented generalization that can bring more contextual cues and global representations. To mitigate this issue, we propose a Scalable Self-Attention (SSA) mechanism that leverages two scaling factors to release dimensions of query, key, and value matrices while unbinding them with the input. This scalability fetches context-oriented generalization and enhances object sensitivity, which pushes the whole network into a more effective trade-off state between accuracy and cost. Furthermore, we propose an Interactive Window-based Self-Attention (IWSA), which establishes interaction between non-overlapping regions by re-merging independent value tokens and aggregating spatial information from adjacent windows. By stacking the SSA and IWSA alternately, the Scalable Vision Transformer (ScalableViT) achieves state-of-the-art performance in general-purpose vision tasks. For example, ScalableViT-S outperforms Twins-SVT-S by 1.4% and Swin-T by 1.8% on ImageNet-1K classification.

cs.CV↗

arXiv2022

Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions

Generalization across different environments with the same tasks is critical for successful applications of visual reinforcement learning (RL) in real scenarios. However, visual distractions -- which are common in real scenes -- from high-dimensional observations can be hurtful to the learned representations in visual RL, thus degrading the performance of generalization. To tackle this problem, we propose a novel approach, namely Characteristic Reward Sequence Prediction (CRESP), to extract the task-relevant information by learning reward sequence distributions (RSDs), as the reward signals are task-relevant in RL and invariant to visual distractions. Specifically, to effectively capture the task-relevant information via RSDs, CRESP introduces an auxiliary task -- that is, predicting the characteristic functions of RSDs -- to learn task-relevant representations, because we can well approximate the high-dimensional distributions by leveraging the corresponding characteristic functions. Experiments demonstrate that CRESP significantly improves the performance of generalization on unseen environments, outperforming several state-of-the-arts on DeepMind Control tasks with different visual distractions.

cs.LG↗

arXiv2022

Bowen's equations for upper metric mean dimension with potential

Firstly, we introduce a new notion called induced upper metric mean dimension with potential, which naturally generalizes the definition of upper metric mean dimension with potential given by Tsukamoto to more general cases, then we establish variational principles for it in terms of upper and lower rate distortion dimensions and show there exists a Bowen's equation between induced upper metric mean dimension with potential and upper metric mean dimension with potential. Secondly, we continue to introduce two new notions, called BS metric mean dimension and Packing BS metric mean dimension on arbitrary subsets, to establish Bowen's equations for Bowen upper metric mean dimension and Packing upper metric mean dimension with potential on subsets. Besides, we also obtain two variational principles for BS metric mean dimension and Packing BS metric mean dimension on subsets. Finally, the special interest about the Bowen upper metric mean dimension of the set of generic points of ergodic measures are also involved.

math.DS↗