arXiv subjects

Jie Fu

Publications and source records attributed to Jie Fu.

At least 217 records · Page 12Linked to original sources

arXiv2022

Synthesis of Proactive Sensor Placement In Probabilistic Attack Graphs

This paper studies the deployment of joint moving target defense (MTD) and deception against multi-stage cyberattacks. Given the system equipped with MTD that randomizes between different configurations, we investigate how to allocate a bounded number of sensors in each configuration to optimize the attack detection rate before the attacker achieves its objective. Specifically, two types of sensors are considered: intrusion detectors that are observable by the attacker and stealthy sensors that are not observable to the attacker. We propose a two-step optimization-based approach for allocating intrusion detectors and stealthy sensors: Firstly, the defender allocates intrusion detectors assuming the attacker will best respond to evade detection by intrusion detectors. Secondly, the defender will allocate stealthy sensors, given the best response attack strategy computed in the first step, to further reduce the attacker's chance of success. We illustrate the effectiveness of the proposed methods using a cyber defense example.

cs.CR↗

arXiv2022

Opportunistic Qualitative Planning in Stochastic Systems with Incomplete Preferences over Reachability Objectives

Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this paper, we study how to synthesize preference satisfying plans in stochastic systems, modeled as an MDP, given a (possibly incomplete) combinative preference model over temporally extended goals. We start by introducing new semantics to interpret preferences over infinite plays of the stochastic system. Then, we introduce a new notion of improvement to enable comparison between two prefixes of an infinite play. Based on this, we define two solution concepts called safe and positively improving (SPI) and safe and almost-surely improving (SASI) that enforce improvements with a positive probability and with probability one, respectively. We construct a model called an improvement MDP, in which the synthesis of SPI and SASI strategies that guarantee at least one improvement reduces to computing positive and almost-sure winning strategies in an MDP. We present an algorithm to synthesize the SPI and SASI strategies that induce multiple sequential improvements. We demonstrate the proposed approach using a robot motion planning problem.

cs.AI↗

arXiv2022

Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning

Artificial neural networks (ANNs) are typically confined to accomplishing pre-defined tasks by learning a set of static parameters. In contrast, biological neural networks (BNNs) can adapt to various new tasks by continually updating the neural connections based on the inputs, which is aligned with the paradigm of learning effective learning rules in addition to static parameters, e.g., meta-learning. Among various biologically inspired learning rules, Hebbian plasticity updates the neural network weights using local signals without the guide of an explicit target function, thus enabling an agent to learn automatically without human efforts. However, typical plastic ANNs using a large amount of meta-parameters violate the nature of the genomics bottleneck and potentially deteriorate the generalization capacity. This work proposes a new learning paradigm decomposing those connection-dependent plasticity rules into neuron-dependent rules thus accommodating $Θ(n^2)$ learnable parameters with only $Θ(n)$ meta-parameters. We also thoroughly study the effect of different neural modulation on plasticity. Our algorithms are tested in challenging random 2D maze environments, where the agents have to use their past experiences to shape the neural connections and improve their performances for the future. The results of our experiment validate the following: 1. Plasticity can be adopted to continually update a randomly initialized RNN to surpass pre-trained, more sophisticated recurrent models, especially when coming to long-term memorization. 2. Following the genomics bottleneck, the proposed decomposed plasticity can be comparable to or even more effective than canonical plasticity rules in some instances.

cs.AI↗

arXiv2022

Patient-specific mean teacher UNet for enhancing PET image and low-dose PET reconstruction on RefleXion X1 biology-guided radiotherapy system

The RefleXion X1 is the first biology-guided radiotherapy (BgRT) system. Its dual 90-degree PET detector collects fewer pair production events compared to a full-ring diagnostic PET system. In the proposed BgRT workflow, a short scan is acquired before treatment delivery to ensure image quality and consistency. The shorter scan time, a quarter of the simulation scan time, also leads to fewer coincidence events and hence reduced image quality. In this study, we proposed a patient-specific mean teacher UNet (MT-UNet) to enhance PET image quality and low-dose PET reconstruction on RefleXion X1. PET/CT scans of nine cancer patients were acquired using RefleXion X1. Every patient had one simulation scan. Five patients had additional scans acquired during the first and the final treatment fractions. Treatment scans were acquired using the same imaging protocol as the simulation scan. For each scan, we reconstructed a full-dose image and evenly split coincidence events into four sessions to reconstruct four quarter-dose PET images. For each patient, our proposed MT-UNet was trained using quarter-dose and full-dose images of the simulation scan. For the image quality enhancement task, we applied nine trained MT-UNets to full-dose simulation PET images of the nine patients to generate enhanced images, respectively. The enhanced images were compared with the original full-dose images using CNR and SNR. For the low-dose image reconstruction task, we applied five trained MT-UNets to ten quarter-dose treatment images of five patients to predict full-dose images, respectively. The predicted and ground truth full-dose images were compared using SSIM and PSNR. We also trained and evaluated patient-specific UNets for model comparison. Our proposed patient-specific MT-UNet achieved better performance in improving the quality of RefleXion low-dose and full-dose images compared to the patient-specific UNet.

physics.med-ph↗

arXiv2022

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

The recent advancements in wireless technology enable connected autonomous vehicles (CAVs) to gather information about their environment by vehicle-to-vehicle (V2V) communication. In this work, we design an information-sharing-based multi-agent reinforcement learning (MARL) framework for CAVs, to take advantage of the extra information when making decisions to improve traffic efficiency and safety. The safe actor-critic algorithm we propose has two new techniques: the truncated Q-function and safe action mapping. The truncated Q-function utilizes the shared information from neighboring CAVs such that the joint state and action spaces of the Q-function do not grow in our algorithm for a large-scale CAV system. We prove the bound of the approximation error between the truncated-Q and global Q-functions. The safe action mapping provides a provable safety guarantee for both the training and execution based on control barrier functions. Using the CARLA simulator for experiments, we show that our approach can improve the CAV system's efficiency in terms of average velocity and comfort under different CAV ratios and different traffic densities. We also show that our approach avoids the execution of unsafe actions and always maintains a safe distance from other vehicles. We construct an obstacle-at-corner scenario to show that the shared vision can help CAVs to observe obstacles earlier and take action to avoid traffic jams.

cs.AI↗

arXiv2022

On Almost-Sure Intention Deception Planning that Exploits Imperfect Observers

Intention deception involves computing a strategy which deceives the opponent into a wrong belief about the agent's intention or objective. This paper studies a class of probabilistic planning problems with intention deception and investigates how a defender's limited sensing modality can be exploited by an attacker to achieve its attack objective almost surely (with probability one) while hiding its intention. In particular, we model the attack planning in a stochastic system modeled as a Markov decision process (MDP). The attacker is to reach some target states while avoiding unsafe states in the system and knows that his behavior is monitored by a defender with partial observations. Given partial state observations for the defender, we develop qualitative intention deception planning algorithms that construct attack strategies to play against an action-visible defender and an action-invisible defender, respectively. The synthesized attack strategy not only ensures the attack objective is satisfied almost surely but also deceives the defender into believing that the observed behavior is generated by a normal/legitimate user and thus failing to detect the presence of an attack. We show the proposed algorithms are correct and complete and illustrate the deceptive planning methods with examples.

cs.GT↗

arXiv2022

CoCon: A Self-Supervised Approach for Controlled Text Generation

Pretrained Transformer-based language models (LMs) display remarkable natural language generation capabilities. With their immense potential, controlling text generation of such LMs is getting attention. While there are studies that seek to control high-level attributes (such as sentiment and topic) of generated text, there is still a lack of more precise control over its content at the word- and phrase-level. Here, we propose Content-Conditioner (CoCon) to control an LM's output text with a content input, at a fine-grained level. In our self-supervised approach, the CoCon block learns to help the LM complete a partially-observed text sequence by conditioning with content inputs that are withheld from the LM. Through experiments, we show that CoCon can naturally incorporate target content into generated texts and control high-level text attributes in a zero-shot manner.

cs.CL↗

arXiv2022

Opportunistic Qualitative Planning in Stochastic Systems with Preferences over Temporal Logic Objectives

Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this work, we study preference-based planning in a stochastic system modeled as a Markov decision process, subject to a possible incomplete preference over temporally extended goals. Our contributions are three folds: First, we introduce a preference language to specify preferences over temporally extended goals. Second, we define a novel automata-theoretic model to represent the preorder induced by given preference relation. The automata representation of preferences enables us to develop a preference-based planning algorithm for stochastic systems. Finally, we show how to synthesize opportunistic strategies that achieves an outcome that improves upon the current satisfiable outcome, with positive probability or with probability one, in a stochastic system. We illustrate our solution approaches using a robot motion planning example.

cs.FL↗

arXiv2022

Unifying Likelihood-free Inference with Black-box Optimization and Beyond

Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on the pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, we provide a recipe for constructing various sequence design methods based on this framework. We show how previous optimization approaches can be "reinvented" in our framework, and further propose new probabilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.

cs.LG↗

arXiv2021

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

Mental health is a critical issue in modern society, and mental disorders could sometimes turn to suicidal ideation without adequate treatment. Early detection of mental disorders and suicidal ideation from social content provides a potential way for effective social intervention. Recent advances in pretrained contextualized language representations have promoted the development of several domain-specific pretrained models and facilitated several downstream applications. However, there are no existing pretrained language models for mental healthcare. This paper trains and release two pretrained masked language models, i.e., MentalBERT and MentalRoBERTa, to benefit machine learning for the mental healthcare research community. Besides, we evaluate our trained domain-specific models and several variants of pretrained language models on several mental disorder detection benchmarks and demonstrate that language representations pretrained in the target domain improve the performance of mental health detection tasks.

cs.CL↗

arXiv2021

Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning

In recent years, the fast rise in number of studies on graph neural network (GNN) has put it from the theories research to reality application stage. Despite the encouraging performance achieved by GNN, less attention has been paid to the privacy-preserving training and inference over distributed graph data in the related literature. Due to the particularity of graph structure, it is challenging to extend the existing private learning framework to GNN. Motivated by the idea of split learning, we propose a \textbf{S}erver \textbf{A}ided \textbf{P}rivacy-preserving \textbf{GNN} (SAPGNN) for the node level task on horizontally partitioned cross-silo scenario. It offers a natural extension of centralized GNN to isolated graph with max/min pooling aggregation, while guaranteeing that all the private data involved in computation still stays at local data holders. To further enhancing the data privacy, a secure pooling aggregation mechanism is proposed. Theoretical and experimental results show that the proposed model achieves the same accuracy as the one learned over the combined data.

cs.LG↗

arXiv2021

Few-Shot Domain Adaptation with Polymorphic Transformers

Deep neural networks (DNNs) trained on one set of medical images often experience severe performance drop on unseen test images, due to various domain discrepancy between the training images (source domain) and the test images (target domain), which raises a domain adaptation issue. In clinical settings, it is difficult to collect enough annotated target domain data in a short period. Few-shot domain adaptation, i.e., adapting a trained model with a handful of annotations, is highly practical and useful in this case. In this paper, we propose a Polymorphic Transformer (Polyformer), which can be incorporated into any DNN backbones for few-shot domain adaptation. Specifically, after the polyformer layer is inserted into a model trained on the source domain, it extracts a set of prototype embeddings, which can be viewed as a "basis" of the source-domain features. On the target domain, the polyformer layer adapts by only updating a projection layer which controls the interactions between image features and the prototype embeddings. All other model weights (except BatchNorm parameters) are frozen during adaptation. Thus, the chance of overfitting the annotations is greatly reduced, and the model can perform robustly on the target domain after being trained on a few annotated images. We demonstrate the effectiveness of Polyformer on two medical segmentation tasks (i.e., optic disc/cup segmentation, and polyp segmentation). The source code of Polyformer is released at https://github.com/askerlee/segtran.

cs.CV↗

arXiv2021

GLGE: A New General Language Generation Evaluation Benchmark

Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus on a range of Natural Language Understanding (NLU) tasks, without considering the Natural Language Generation (NLG) models. In this paper, we present the General Language Generation Evaluation (GLGE), a new multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks. For each task, we continue to design three subtasks in terms of task difficulty (GLGE-Easy, GLGE-Medium, and GLGE-Hard). This introduces 24 subtasks to comprehensively compare model performance. To encourage research on pretraining and transfer learning on NLG models, we make GLGE publicly available and build a leaderboard with strong baselines including MASS, BART, and ProphetNet (The source code and dataset are publicly available at https://github.com/microsoft/glge).

cs.CL↗

arXiv2021

Policy Synthesis for Metric Interval Temporal Logic with Probabilistic Distributions

Metric Temporal Logic can express temporally evolving properties with time-critical constraints or time-triggered constraints for real-time systems. This paper extends the Metric Interval Temporal Logic with a distribution eventuality operator to express time-sensitive missions for a system interacting with a dynamic, probabilistic environment. This formalism enables us to describe the probabilistic occurrences of random external events as part of the task specification and event-triggered temporal constraints for the intended system's behavior. The main contributions of this paper are two folds: First, we propose a procedure to translate a specification into a stochastic timed automaton. Second, we develop an approximate-optimal probabilistic planning problem for synthesizing the control policy that maximizes the probability for the planning agent to achieve the task, provided that the external events satisfy the specification. The planning algorithm employs a truncation in the clocks for the timed automaton to reduce the planning in a countably infinite state space to a finite state space with a bounded error guarantee. We illustrate the method with a robot motion planning example.

cs.FL↗

arXiv2021

Learning Contact-aware CPG-based Locomotion in a Soft Snake Robot

In this paper, we present a model-free learning-based control scheme for the soft snake robot to improve its contact-aware locomotion performance in a cluttered environment. The control scheme includes two cooperative controllers: A bio-inspired controller (C1) that controls both the steering and velocity of the soft snake robot, and an event-triggered regulator (R2) that controls the steering of the snake in anticipation of obstacle contacts and during contact. The inputs from the two controllers are composed as the input to a Matsuoka CPG network to generate smooth and rhythmic actuation inputs to the soft snake. To enable stable and efficient learning with two controllers, we develop a game-theoretic process, fictitious play, to train C1 and R2 with a shared potential-field-based reward function for goal tracking tasks. The proposed approach is tested and evaluated in the simulator and shows significant improvement of locomotion performance in the obstacle-based environment comparing to two baseline controllers.

cs.RO↗

arXiv2021

Qualitative Planning in Imperfect Information Games with Active Sensing and Reactive Sensor Attacks: Cost of Unawareness

We consider the probabilistic planning problem where the agent (called Player 1, or P1) can jointly plan the control actions and sensor queries in a sensor network and an attacker (called player 2, or P2) can carry out attacks on the sensors. We model such an adversarial interaction using a formal model -- a reachability game with partially controllable observation functions. The main contribution of this paper is to assess the cost of P1's unawareness: Suppose P1 misinterprets the sensor failures as probabilistic node failures due to unreliable network communication, and P2 is aware of P1's misinterpretation in addition to her partial observability. Then, from which states can P2 carry out sensor attacks to ensure, with probability one, that P1 will not be able to complete her reachability task even though, due to misinterpretation, P1 believes that she can almost-surely achieve her task. We develop an algorithm to solve the almost-sure winning sensor-attack strategy given P1's observation-based strategy. Our attack analysis could be used for attack detection in wireless communication networks and the design of provably secured attack-aware sensor allocation in decision-theoretic models for cyber-physical systems.

cs.CR↗

arXiv2021

Synthesis of Deceptive Strategies in Reachability Games with Action Misperception (Technical Report)

Strategic deception is an act of manipulating the opponent's perception to gain strategic advantages. In this paper, we study synthesis of deceptive winning strategies in two-player turn-based zero-sum reachability games on graphs with one-sided incomplete information of action sets. In particular, we consider the class of games in which Player 1 (P1) starts with a non-empty set of private actions, which she may 'reveal' to Player 2 (P2) during the course of the game. P2 is equipped with an inference mechanism using which he updates his perception of P1's action set whenever a new action is revealed. Under this information structure, the objective of P1 is to reach a set of goal states in the game graph while that of P2 is to prevent it. We address the question: how can P1 leverage her information advantages to deceive P2 into choosing actions that in turn benefit P1? To this end, we introduce a dynamic hypergame model to capture the reachability game with evolving misperception of P2. Analyzing the game qualitatively, we design algorithms to synthesize deceptive sure and almost-sure winning regions, and establish two key results: (1) under sure-winning condition, deceptive winning strategy is equivalent to the non-deceptive winning strategy - i.e. use of deception has no advantages, (2) under almost-sure winning condition, the deceptive winning strategy could be more powerful than the non-deceptive strategy. We illustrate our algorithms using a capture-the-flag game, and demonstrate the use of proposed approach to a larger class of games with temporal logic objectives.

cs.GT↗

arXiv2021

Probabilistic Planning with Preferences over Temporal Goals

We present a formal language for specifying qualitative preferences over temporal goals and a preference-based planning method in stochastic systems. Using automata-theoretic modeling, the proposed specification allows us to express preferences over different sets of outcomes, where each outcome describes a set of temporal sequences of subgoals. We define the value of preference satisfaction given a stochastic process over possible outcomes and develop an algorithm for time-constrained probabilistic planning in labeled Markov decision processes where an agent aims to maximally satisfy its preference formula within a pre-defined finite time duration. We present experimental results using a stochastic gridworld example and discuss possible extensions of the proposed preference model.

cs.AI↗