arXiv subjects

Liang Zhao

Publications and source records attributed to Liang Zhao.

At least 433 records · Page 24Linked to original sources

arXiv2019

CBOWRA: A Representation Learning Approach for Medication Anomaly Detection

Electronic health record is an important source for clinical researches and applications, and errors inevitably occur in the data, which could lead to severe damages to both patients and hospital services. One of such error is the mismatches between diagnoses and prescriptions, which we address as 'medication anomaly' in the paper, and clinicians used to manually identify and correct them. With the development of machine learning techniques, researchers are able to train specific model for the task, but the process still requires expert knowledge to construct proper features, and few semantic relations are considered. In this paper, we propose a simple, yet effective detection method that tackles the problem by detecting the semantic inconsistency between diagnoses and prescriptions. Unlike traditional outlier or anomaly detection, the scheme uses continuous bag of words to construct the semantic connection between specific central words and their surrounding context. The detection of medication anomaly is transformed into identifying the least possible central word based on given context. To help distinguish the anomaly from normal context, we also incorporate a ranking accumulation strategy. The experiments were conducted on two real hospital electronic medical records, and the topN accuracy of the proposed method increased by 3.91 to 10.91% and 0.68 to 2.13% on the datasets, respectively, which is highly competitive to other traditional machine learning-based approaches.

cs.CL↗

arXiv2019

The Application of Multi-block ADMM on Isotonic Regression Problems

The multi-block ADMM has received much attention from optimization researchers due to its excellent scalability. In this paper, the multi-block ADMM is applied to solve two large-scale problems related to isotonic regression. Numerical experiments show that the multi-block ADMM is convergent when the chosen parameter is small enough and the multi-block ADMM scales well compared with baselines.

math.OC↗

arXiv2019

Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net

Accurate segmentation of organ at risk (OAR) play a critical role in the treatment planning of image guided radiation treatment of head and neck cancer. This segmentation task is challenging for both human and automatic algorithms because of the relatively large number of OARs to be segmented, the large variability of the size and morphology across different OARs, and the low contrast of between some OARs and the background. In this paper, we proposed a two-stage segmentation framework based on 3D U-Net. In this framework, the segmentation of each OAR is decomposed into two sub-tasks: locating a bounding box of the OAR and segment it from a small volume within the bounding box, and each sub-tasks is fulfilled by a dedicated 3D U-Net. The decomposition makes each of the two sub-tasks much easier, so that they can be better completed. We evaluated the proposed method and compared it to state-of-the-art methods by using the MICCAI 2015 Challenge dataset. In terms of the boundary-based metric 95HD, the proposed method ranked first in eight of all nine OARs and ranked second in the other OAR. In terms of the area-based metric DSC, the proposed method ranked first in six of the nine OARs and ranked second in the other three OARs with small difference with the first one.

cs.CV↗

arXiv2019

Compositional Generalization for Primitive Substitutions

Compositional generalization is a basic mechanism in human language learning, but current neural networks lack such ability. In this paper, we conduct fundamental research for encoding compositionality in neural networks. Conventional methods use a single representation for the input sentence, making it hard to apply prior knowledge of compositionality. In contrast, our approach leverages such knowledge with two representations, one generating attention maps, and the other mapping attended input words to output symbols. We reduce the entropy in each representation to improve generalization. Our experiments demonstrate significant improvements over the conventional methods in five NLP tasks including instruction learning and machine translation. In the SCAN domain, it boosts accuracies from 14.0% to 98.8% in Jump task, and from 92.0% to 99.7% in TurnLeft task. It also beats human performance on a few-shot learning task. We hope the proposed approach can help ease future research towards human-level compositional language learning.

cs.CL↗

arXiv2019

Multi-stage Deep Classifier Cascades for Open World Recognition

At present, object recognition studies are mostly conducted in a closed lab setting with classes in test phase typically in training phase. However, real-world problem is far more challenging because: i) new classes unseen in the training phase can appear when predicting; ii) discriminative features need to evolve when new classes emerge in real time; and iii) instances in new classes may not follow the "independent and identically distributed" (iid) assumption. Most existing work only aims to detect the unknown classes and is incapable of continuing to learn newer classes. Although a few methods consider both detecting and including new classes, all are based on the predefined handcrafted features that cannot evolve and are out-of-date for characterizing emerging classes. Thus, to address the above challenges, we propose a novel generic end-to-end framework consisting of a dynamic cascade of classifiers that incrementally learn their dynamic and inherent features. The proposed method injects dynamic elements into the system by detecting instances from unknown classes, while at the same time incrementally updating the model to include the new classes. The resulting cascade tree grows by adding a new leaf node classifier once a new class is detected, and the discriminative features are updated via an end-to-end learning strategy. Experiments on two real-world datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.

cs.LG↗

arXiv2019

DynGraph2Seq: Dynamic-Graph-to-Sequence Interpretable Learning for Health Stage Prediction in Online Health Forums

Online health communities such as the online breast cancer forum enable patients (i.e., users) to interact and help each other within various subforums, which are subsections of the main forum devoted to specific health topics. The changing nature of the users' activities in different subforums can be strong indicators of their health status changes. This additional information could allow health-care organizations to respond promptly and provide additional help for the patient. However, modeling complex transitions of an individual user's activities among different subforums over time and learning how these correspond to his/her health stage are extremely challenging. In this paper, we first formulate the transition of user activities as a dynamic graph with multi-attributed nodes, then formalize the health stage inference task as a dynamic graph-to-sequence learning problem, and hence propose a novel dynamic graph-to-sequence neural networks architecture (DynGraph2Seq) to address all the challenges. Our proposed DynGraph2Seq model consists of a novel dynamic graph encoder and an interpretable sequence decoder that learn the mapping between a sequence of time-evolving user activity graphs and a sequence of target health stages. We go on to propose dynamic graph hierarchical attention mechanisms to facilitate the necessary multi-level interpretability. A comprehensive experimental analysis of its use for a health stage prediction task demonstrates both the effectiveness and the interpretability of the proposed models.

cs.LG↗

arXiv2019

Counting Roots of a Polynomial in a Convex Compact Region by Means of Winding Number Calculation via Sampling

In this paper we propose a novel efficient algorithm for calculating winding numbers, aiming at counting the number of roots of a given polynomial in a convex region on the complex plane. This algorithm can be used for counting and exclusion tests in a subdivision algorithms for polynomial root-finding, and would be especially usefull in application scenarios where high-precision polynomial coefficients are hard to obtain but we succeed with counting already by using polynomial evaluation with lower precision. We provide the pseudo code of the algorithm, proof of its correctness as well as estimation of its complexity.

math.NA↗

arXiv2019

Existence and convergence of solutions for nonlinear biharmonic equations on graphs

In this paper, we first prove some propositions of Sobolev spaces defined on a locally finite graph $G=(V,E)$, which are fundamental when dealing with equations on graphs under the variational framework. Then we consider a nonlinear biharmonic equation $$ Δ^{2} u -Δu+(λa+1)u= |u|^{p-2}u $$ on $G=(V,E)$. Under some suitable assumptions, we prove that for any $λ>1$ and $p>2$, the equation admits a ground state solution $u_λ$. Moreover, we prove that as $λ\rightarrow +\infty$, the solutions $u_λ$ converge to a solution of the equation \begin{align*} \begin{cases} Δ^{2}u -Δu+u = |u|^{p-2}u, &\text{in}\ \ Ω, u=0, &\text{on}\ \ \partialΩ, \end{cases} \end{align*} where $Ω=\{x\in V: a(x)=0\}$ is the potential well and $\partialΩ$ denotes the the boundary of $Ω$.

math.AP↗

arXiv2019

Pyramid: Machine Learning Framework to Estimate the Optimal Timing and Resource Usage of a High-Level Synthesis Design

The emergence of High-Level Synthesis (HLS) tools shifted the paradigm of hardware design by making the process of mapping high-level programming languages to hardware design such as C to VHDL/Verilog feasible. HLS tools offer a plethora of techniques to optimize designs for both area and performance, but resource usage and timing reports of HLS tools mostly deviate from the post-implementation results. In addition, to evaluate a hardware design performance, it is critical to determine the maximum achievable clock frequency. Obtaining such information using static timing analysis provided by CAD tools is difficult, due to the multitude of tool options. Moreover, a binary search to find the maximum frequency is tedious, time-consuming, and often does not obtain the optimal result. To address these challenges, we propose a framework, called Pyramid, that uses machine learning to accurately estimate the optimal performance and resource utilization of an HLS design. For this purpose, we first create a database of C-to-FPGA results from a diverse set of benchmarks. To find the achievable maximum clock frequency, we use Minerva, which is an automated hardware optimization tool. Minerva determines the close-to-optimal settings of tools, using static timing analysis and a heuristic algorithm, and targets either optimal throughput or throughput-to-area. Pyramid uses the database to train an ensemble machine learning model to map the HLS-reported features to the results of Minerva. To this end, Pyramid re-calibrates the results of HLS to bridge the accuracy gap and enable developers to estimate the throughput or throughput-to-area of hardware design with more than 95% accuracy and alleviates the need to perform actual implementation for estimation.

cs.AR↗

arXiv2019

An observable time series based SLAM algorithm for deforming environment

In this paper, we study the back-end of simultaneous localization and mapping (SLAM) problem in deforming environment, where robot localizes itself and tracks multiple non-rigid soft surface using its onboard sensor measurements. An elaborate analysis is conducted on conventional deformation modelling method, Embedded Deformation (ED) graph. We demonstrate and prove that the ED graph widely used in such scenarios is unobservable and leads to multiple solutions unless suitable priors are provided. Example as well as theoretical prove are provided to show the ambiguity of ED graph and camera pose. In modelling non-rigid scenario with ED graph, motion priors of the deforming environment is essential to separate robot pose and deforming environment. The conclusion can be extrapolated to any free form deformation formulation. In solving the observability, this research proposes a preliminary deformable SLAM approach to estimate robot pose in complex environments that exhibits regular motion. A strategy that approximates deformed shape using a linear combination of several previous shapes is proposed to avoid the ambiguity in robot movement and rigid and non-rigid motions of the environment. Fisher information matrix rank analysis with a base case is discussed to prove the effectiveness. Moreover, the proposed algorithm is validated extensively on Monte Carlo simulations and real experiments. It is demonstrated that the new algorithm significantly outperforms conventional rigid SLAM and ED based SLAM especially in scenarios where there is large deformation.

cs.RO↗

arXiv2019

Non-Bayesian Social Learning with Imperfect Private Signal Structure

As one of the classic models that describe the belief dynamics over social networks, a non-Bayesian social learning model assumes that members in the network possess accurate signal knowledge through the process of Bayesian inference. In order to make the non-Bayesian social learning model more applicable to human and animal societies, this paper extended this model by assuming the existence of private signal structure bias. Each social member in each time step uses an imperfect signal knowledge to form its Bayesian part belief and then incorporates its neighbors' beliefs into this Bayesian part belief to form a new belief report. First, we investigated the intrinsic learning ability of an isolated agent and deduced the conditions that the signal structure needs to satisfy for this isolated agent to make an eventually correct decision. According to these conditions, agents' signal structures were further divided into three different types, "conservative," "radical," and "negative." Then, we switched the context from isolated agents to a connected network; our propositions and simulations show that the conservative agents are the dominant force for the social network to learn the real state, while the other two types might prevent the network from successful learning. Although fragilities do exist in non-Bayesian social learning mechanism, "be more conservative" and "avoid overconfidence" could be effective strategies for each agent in the real social networks to collectively improve social learning processes and results.

cs.SI↗

arXiv2019

Interpreting and Evaluating Neural Network Robustness

Recently, adversarial deception becomes one of the most considerable threats to deep neural networks. However, compared to extensive research in new designs of various adversarial attacks and defenses, the neural networks' intrinsic robustness property is still lack of thorough investigation. This work aims to qualitatively interpret the adversarial attack and defense mechanism through loss visualization, and establish a quantitative metric to evaluate the neural network model's intrinsic robustness. The proposed robustness metric identifies the upper bound of a model's prediction divergence in the given domain and thus indicates whether the model can maintain a stable prediction. With extensive experiments, our metric demonstrates several advantages over conventional adversarial testing accuracy based robustness estimation: (1) it provides a uniformed evaluation to models with different structures and parameter scales; (2) it over-performs conventional accuracy based robustness estimation and provides a more reliable evaluation that is invariant to different test settings; (3) it can be fast generated without considerable testing cost.

cs.LG↗

arXiv2019

Dynamic jamming of dense suspensions under tilted impact

Dense particulate suspensions can not only increase their viscosity and shear thicken under external forcing, but also jam into a solid-like state that is fully reversible when the force is removed. An impact on the surface of a dense suspension can trigger this jamming process by generating a shear front that propagates into the bulk of the system. Tracking and visualizing such a front is difficult because suspensions are optically opaque and the front can propagate as fast as several meters per second. Recently, high-speed ultrasound imaging has been used to overcome this problem and extract two-dimensional sections of the flow field associated with jamming front propagation. Here we extend this method to reconstruct the three-dimensional flow field. This enables us to investigate the evolution of jamming fronts for which axisymmetry cannot be assumed, such as impact at angles tilted away from the normal to the free surface of the suspension. We find that sufficiently far from solid boundaries the resulting flow field is approximately identical to that generated by normal impact, but rotated and aligned with the angle of impact. However, once the front approaches the solid boundary at the bottom of the container, it generates a squeeze flow that deforms the front profile and causes jamming to proceed in a non-axisymmetric manner.

cond-mat.soft↗

arXiv2019

Learning Good Representation via Continuous Attention

In this paper we present our scientific discovery that good representation can be learned via continuous attention during the interaction between Unsupervised Learning(UL) and Reinforcement Learning(RL) modules driven by intrinsic motivation. Specifically, we designed intrinsic rewards generated from UL modules for driving the RL agent to focus on objects for a period of time and to learn good representations of objects for later object recognition task. We evaluate our proposed algorithm in both with and without extrinsic reward settings. Experiments with end-to-end training in simulated environments with applications to few-shot object recognition demonstrated the effectiveness of the proposed algorithm.

cs.LG↗

arXiv2019

Algorithms to compute the Burrows-Wheeler Similarity Distribution

The Burrows-Wheeler transform (BWT) is a well studied text transformation widely used in data compression and text indexing. The BWT of two strings can also provide similarity measures between them, based on the observation that the more their symbols are intermixed in the transformation, the more the strings are similar. In this article we present two new algorithms to compute similarity measures based on the BWT for string collections. In particular, we present practical and theoretical improvements to the computation of the Burrows-Wheeler similarity distribution for all pairs of strings in a collection. Our algorithms take advantage of the BWT computed for the concatenation of all strings, and use compressed data structures that allow reducing the running time with a small memory footprint, as shown by a set of experiments with real and artificial datasets.

cs.DS↗

arXiv2019

CircConv: A Structured Convolution with Low Complexity

Deep neural networks (DNNs), especially deep convolutional neural networks (CNNs), have emerged as the powerful technique in various machine learning applications. However, the large model sizes of DNNs yield high demands on computation resource and weight storage, thereby limiting the practical deployment of DNNs. To overcome these limitations, this paper proposes to impose the circulant structure to the construction of convolutional layers, and hence leads to circulant convolutional layers (CircConvs) and circulant CNNs. The circulant structure and models can be either trained from scratch or re-trained from a pre-trained non-circulant model, thereby making it very flexible for different training environments. Through extensive experiments, such strong structure-imposing approach is proved to be able to substantially reduce the number of parameters of convolutional layers and enable significant saving of computational cost by using fast multiplication of the circulant tensor.

cs.CV↗

arXiv2019

Robust Regression via Online Feature Selection under Adversarial Data Corruption

The presence of data corruption in user-generated streaming data, such as social media, motivates a new fundamental problem that learns reliable regression coefficient when features are not accessible entirely at one time. Until now, several important challenges still cannot be handled concurrently: 1) corrupted data estimation when only partial features are accessible; 2) online feature selection when data contains adversarial corruption; and 3) scaling to a massive dataset. This paper proposes a novel RObust regression algorithm via Online Feature Selection (\textit{RoOFS}) that concurrently addresses all the above challenges. Specifically, the algorithm iteratively updates the regression coefficients and the uncorrupted set via a robust online feature substitution method. We also prove that our algorithm has a restricted error bound compared to the optimal solution. Extensive empirical experiments in both synthetic and real-world datasets demonstrated that the effectiveness of our new method is superior to that of existing methods in the recovery of both feature selection and regression coefficients, with very competitive efficiency.

cs.LG↗

arXiv2019

WALL-E: An Efficient Reinforcement Learning Research Framework

There are two halves to RL systems: experience collection time and policy learning time. For a large number of samples in rollouts, experience collection time is the major bottleneck. Thus, it is necessary to speed up the rollout generation time with multi-process architecture support. Our work, dubbed WALL-E, utilizes multiple rollout samplers running in parallel to rapidly generate experience. Due to our parallel samplers, we experience not only faster convergence times, but also higher average reward thresholds. For example, on the MuJoCo HalfCheetah-v2 task, with $N = 10$ parallel sampler processes, we are able to achieve much higher average return than those from using only a single process architecture.

cs.LG↗