arXiv subjects

Yike Guo

Publications and source records attributed to Yike Guo.

At least 235 records · Page 13Linked to original sources

arXiv2019

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

Deep neural networks are a promising approach towards multi-task learning because of their capability to leverage knowledge across domains and learn general purpose representations. Nevertheless, they can fail to live up to these promises as tasks often compete for a model's limited resources, potentially leading to lower overall performance. In this work we tackle the issue of interfering tasks through a comprehensive analysis of their training, derived from looking at the interaction between gradients within their shared parameters. Our empirical results show that well-performing models have low variance in the angles between task gradients and that popular regularization methods implicitly reduce this measure. Based on this observation, we propose a novel gradient regularization term that minimizes task interference by enforcing near orthogonal gradients. Updating the shared parameters using this property encourages task specific decoders to optimize different parts of the feature extractor, thus reducing competition. We evaluate our method with classification and regression tasks on the multiDigitMNIST, NYUv2 and SUN RGB-D datasets where we obtain competitive results.

cs.LG↗

arXiv2019

Biologically inspired architectures for sample-efficient deep reinforcement learning

Deep reinforcement learning requires a heavy price in terms of sample efficiency and overparameterization in the neural networks used for function approximation. In this work, we use tensor factorization in order to learn more compact representation for reinforcement learning policies. We show empirically that in the low-data regime, it is possible to learn online policies with 2 to 10 times less total coefficients, with little to no loss of performance. We also leverage progress in second order optimization, and use the theory of wavelet scattering to further reduce the number of learned coefficients, by foregoing learning the topmost convolutional layer filters altogether. We evaluate our results on the Atari suite against recent baseline algorithms that represent the state-of-the-art in data efficiency, and get comparable results with an order of magnitude gain in weight parsimony.

cs.LG↗

arXiv2019

Automatic Brain Tumour Segmentation and Biophysics-Guided Survival Prediction

Gliomas are the most common malignant brain tumourswith intrinsic heterogeneity. Accurate segmentation of gliomas and theirsub-regions on multi-parametric magnetic resonance images (mpMRI)is of great clinical importance, which defines tumour size, shape andappearance and provides abundant information for preoperative diag-nosis, treatment planning and survival prediction. Recent developmentson deep learning have significantly improved the performance of auto-mated medical image segmentation. In this paper, we compare severalstate-of-the-art convolutional neural network models for brain tumourimage segmentation. Based on the ensembled segmentation, we presenta biophysics-guided prognostic model for patient overall survival predic-tion which outperforms a data-driven radiomics approach. Our methodwon the second place of the MICCAI 2019 BraTS Challenge for theoverall survival prediction.

eess.IV↗

arXiv2019

Unsupervised Annotation of Phenotypic Abnormalities via Semantic Latent Representations on Electronic Health Records

The extraction of phenotype information which is naturally contained in electronic health records (EHRs) has been found to be useful in various clinical informatics applications such as disease diagnosis. However, due to imprecise descriptions, lack of gold standards and the demand for efficiency, annotating phenotypic abnormalities on millions of EHR narratives is still challenging. In this work, we propose a novel unsupervised deep learning framework to annotate the phenotypic abnormalities from EHRs via semantic latent representations. The proposed framework takes the advantage of Human Phenotype Ontology (HPO), which is a knowledge base of phenotypic abnormalities, to standardize the annotation results. Experiments have been conducted on 52,722 EHRs from MIMIC-III dataset. Quantitative and qualitative analysis have shown the proposed framework achieves state-of-the-art annotation performance and computational efficiency compared with other methods.

cs.CL↗

arXiv2019

Blockchain-based Personal Data Management: From Fiction to Solution

The emerging blockchain technology has enabled various decentralised applications in a trustless environment without relying on a trusted intermediary. It is expected as a promising solution to tackle sophisticated challenges on personal data management, thanks to its advanced features such as immutability, decentralisation and transparency. Although certain approaches have been proposed to address technical difficulties in personal data management; most of them only provided preliminary methodological exploration. Alarmingly, when utilising Blockchain for developing a personal data management system, fictions have occurred in existing approaches and been promulgated in the literature. Such fictions are theoretically doable; however, by thoroughly breaking down consensus protocols and transaction validation processes, we clarify that such existing approaches are either impractical or highly inefficient due to the natural limitations of the blockchain and Smart Contracts technologies. This encourages us to propose a feasible solution in which such fictions are reduced by designing a novel system architecture with a blockchain-based "proof of permission" protocol. We demonstrate the feasibility and efficiency of the proposed models by implementing a clinical data sharing service built on top of a public blockchain platform. We believe that our research resolves existing ambiguity and take a step further on providing a practically feasible solution for decentralised personal data management.

cs.DC↗

arXiv2019

Transfer Learning from Partial Annotations for Whole Brain Segmentation

Brain MR image segmentation is a key task in neuroimaging studies. It is commonly conducted using standard computational tools, such as FSL, SPM, multi-atlas segmentation etc, which are often registration-based and suffer from expensive computation cost. Recently, there is an increased interest using deep neural networks for brain image segmentation, which have demonstrated advantages in both speed and performance. However, neural networks-based approaches normally require a large amount of manual annotations for optimising the massive amount of network parameters. For 3D networks used in volumetric image segmentation, this has become a particular challenge, as a 3D network consists of many more parameters compared to its 2D counterpart. Manual annotation of 3D brain images is extremely time-consuming and requires extensive involvement of trained experts. To address the challenge with limited manual annotations, here we propose a novel multi-task learning framework for brain image segmentation, which utilises a large amount of automatically generated partial annotations together with a small set of manually created full annotations for network training. Our method yields a high performance comparable to state-of-the-art methods for whole brain segmentation.

eess.IV↗

arXiv2019

Integrated Multi-omics Analysis Using Variational Autoencoders: Application to Pan-cancer Classification

Different aspects of a clinical sample can be revealed by multiple types of omics data. Integrated analysis of multi-omics data provides a comprehensive view of patients, which has the potential to facilitate more accurate clinical decision making. However, omics data are normally high dimensional with large number of molecular features and relatively small number of available samples with clinical labels. The "dimensionality curse" makes it challenging to train a machine learning model using high dimensional omics data like DNA methylation and gene expression profiles. Here we propose an end-to-end deep learning model called OmiVAE to extract low dimensional features and classify samples from multi-omics data. OmiVAE combines the basic structure of variational autoencoders with a classification network to achieve task-oriented feature extraction and multi-class classification. The training procedure of OmiVAE is comprised of an unsupervised phase without the classifier and a supervised phase with the classifier. During the unsupervised phase, a hierarchical cluster structure of samples can be automatically formed without the need for labels. And in the supervised phase, OmiVAE achieved an average classification accuracy of 97.49% after 10-fold cross-validation among 33 tumour types and normal samples, which shows better performance than other existing methods. The OmiVAE model learned from multi-omics data outperformed that using only one type of omics data, which indicates that the complementary information from different omics datatypes provides useful insights for biomedical tasks like cancer classification.

cs.LG↗

arXiv2019

Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction

In the recent years, convolutional neural networks have transformed the field of medical image analysis due to their capacity to learn discriminative image features for a variety of classification and regression tasks. However, successfully learning these features requires a large amount of manually annotated data, which is expensive to acquire and limited by the available resources of expert image analysts. Therefore, unsupervised, weakly-supervised and self-supervised feature learning techniques receive a lot of attention, which aim to utilise the vast amount of available data, while at the same time avoid or substantially reduce the effort of manual annotation. In this paper, we propose a novel way for training a cardiac MR image segmentation network, in which features are learnt in a self-supervised manner by predicting anatomical positions. The anatomical positions serve as a supervisory signal and do not require extra manual annotation. We demonstrate that this seemingly simple task provides a strong signal for feature learning and with self-supervised learning, we achieve a high segmentation accuracy that is better than or comparable to a U-net trained from scratch, especially at a small data setting. When only five annotated subjects are available, the proposed method improves the mean Dice metric from 0.811 to 0.852 for short-axis image segmentation, compared to the baseline U-net.

cs.CV↗

arXiv2019

Static Activation Function Normalization

Recent seminal work at the intersection of deep neural networks practice and random matrix theory has linked the convergence speed and robustness of these networks with the combination of random weight initialization and nonlinear activation function in use. Building on those principles, we introduce a process to transform an existing activation function into another one with better properties. We term such transform \emph{static activation normalization}. More specifically we focus on this normalization applied to the ReLU unit, and show empirically that it significantly promotes convergence robustness, maximum training depth, and anytime performance. We verify these claims by examining empirical eigenvalue distributions of networks trained with those activations. Our static activation normalization provides a first step towards giving benefits similar in spirit to schemes like batch normalization, but without computational cost.

cs.LG↗

arXiv2019

Integrating Semantic Knowledge to Tackle Zero-shot Text Classification

Insufficient or even unavailable training data of emerging classes is a big challenge of many classification tasks, including text classification. Recognising text documents of classes that have never been seen in the learning stage, so-called zero-shot text classification, is therefore difficult and only limited previous works tackled this problem. In this paper, we propose a two-phase framework together with data augmentation and feature augmentation to solve this problem. Four kinds of semantic knowledge (word embeddings, class descriptions, class hierarchy, and a general knowledge graph) are incorporated into the proposed framework to deal with instances of unseen classes effectively. Experimental results show that each and the combination of the two phases achieve the best overall accuracy compared with baselines and recent approaches in classifying real-world texts under the zero-shot scenario.

cs.CL↗

arXiv2019

Trust Evaluation Mechanism for User Recruitment in Mobile Crowd-Sensing in the Internet of Things

Mobile Crowd-Sensing (MCS) has appeared as a prospective solution for large-scale data collection, leveraging built-in sensors and social applications in mobile devices that enables a variety of Internet of Things (IoT) services. However, the human involvement in MCS results in a high possibility for unintentionally contributing corrupted and falsified data or intentionally spreading disinformation for malevolent purposes, consequently undermining IoT services. Therefore, recruiting trustworthy contributors plays a crucial role in collecting high-quality data and providing better quality of services while minimizing the vulnerabilities and risks to MCS systems. In this article, a novel trust model called Experience-Reputation (E-R) is proposed for evaluating trust relationships between any two mobile device users in a MCS platform. To enable the E-R model, virtual interactions among the users are manipulated by considering an assessment of the quality of contributed data from such users. Based on these interactions, two indicators of trust called Experience and Reputation are calculated accordingly. By incorporating the Experience and Reputation trust indicators (TIs), trust relationships between the users are established, evaluated and maintained. Based on these trust relationships, a novel trust-based recruitment scheme is carried out for selecting the most trustworthy MCS users to contribute to data sensing tasks. In order to evaluate the performance and effectiveness of the proposed trust-based mechanism as well as the E-R trust model, we deploy several recruitment schemes in a MCS testbed which consists of both normal and malicious users. The results highlight the strength of the trust-based scheme as it delivers better quality for MCS services while being able to detect malicious users.

cs.NI↗

arXiv2019

Combining learning rate decay and weight decay with complexity gradient descent - Part I

The role of $L^2$ regularization, in the specific case of deep neural networks rather than more traditional machine learning models, is still not fully elucidated. We hypothesize that this complex interplay is due to the combination of overparameterization and high dimensional phenomena that take place during training and make it unamenable to standard convex optimization methods. Using insights from statistical physics and random fields theory, we introduce a parameter factoring in both the level of the loss function and its remaining nonconvexity: the \emph{complexity}. We proceed to show that it is desirable to proceed with \emph{complexity gradient descent}. We then show how to use this intuition to derive novel and efficient annealing schemes for the strength of $L^2$ regularization when performing standard stochastic gradient descent in deep neural networks.

cs.LG↗

arXiv2018

The Deep Poincaré Map: A Novel Approach for Left Ventricle Segmentation

Precise segmentation of the left ventricle (LV) within cardiac MRI images is a prerequisite for the quantitative measurement of heart function. However, this task is challenging due to the limited availability of labeled data and motion artifacts from cardiac imaging. In this work, we present an iterative segmentation algorithm for LV delineation. By coupling deep learning with a novel dynamic-based labeling scheme, we present a new methodology where a policy model is learned to guide an agent to travel over the the image, tracing out a boundary of the ROI -- using the magnitude difference of the Poincaré map as a stopping criterion. Our method is evaluated on two datasets, namely the Sunnybrook Cardiac Dataset (SCD) and data from the STACOM 2011 LV segmentation challenge. Our method outperforms the previous research over many metrics. In order to demonstrate the transferability of our method we present encouraging results over the STACOM 2011 data, when using a model trained on the SCD dataset.

cs.CV↗

arXiv2018

Deep Sequence Learning with Auxiliary Information for Traffic Prediction

Predicting traffic conditions from online route queries is a challenging task as there are many complicated interactions over the roads and crowds involved. In this paper, we intend to improve traffic prediction by appropriate integration of three kinds of implicit but essential factors encoded in auxiliary information. We do this within an encoder-decoder sequence learning framework that integrates the following data: 1) offline geographical and social attributes. For example, the geographical structure of roads or public social events such as national celebrations; 2) road intersection information. In general, traffic congestion occurs at major junctions; 3) online crowd queries. For example, when many online queries issued for the same destination due to a public performance, the traffic around the destination will potentially become heavier at this location after a while. Qualitative and quantitative experiments on a real-world dataset from Baidu have demonstrated the effectiveness of our framework.

cs.CV↗

arXiv2018

Generative Creativity: Adversarial Learning for Bionic Design

Bionic design refers to an approach of generative creativity in which a target object (e.g. a floor lamp) is designed to contain features of biological source objects (e.g. flowers), resulting in creative biologically-inspired design. In this work, we attempt to model the process of shape-oriented bionic design as follows: given an input image of a design target object, the model generates images that 1) maintain shape features of the input design target image, 2) contain shape features of images from the specified biological source domain, 3) are plausible and diverse. We propose DesignGAN, a novel unsupervised deep generative approach to realising bionic design. Specifically, we employ a conditional Generative Adversarial Networks architecture with several designated losses (an adversarial loss, a regression loss, a cycle loss and a latent loss) that respectively constrict our model to meet the corresponding aforementioned requirements of bionic design modelling. We perform qualitative and quantitative experiments to evaluate our method, and demonstrate that our proposed approach successfully generates creative images of bionic design.

cs.CV↗

arXiv2018

The Future of Spreadsheets in the Big Data Era

The humble spreadsheet is the most widely used data storage, manipulation and modelling tool. Its ubiquity over the past 30 years has seen its successful application in every area of life. Surprisingly the spreadsheet has remained fundamentally unchanged over the past three decades. As spreadsheet technology enters its 4th decade a number of drivers of change are beginning to impact upon the spreadsheet. The rise of Big Data, increased end-user computing and mobile computing will undoubtedly increasingly shape the evolution and use of spreadsheet technology. To explore the future of spreadsheet technology a workshop was convened with the aim of "bringing together academia and industry to examine the future direction of spreadsheet technology and the consequences for users". This paper records the views of the participants on the reasons for the success of the spreadsheet, the trends driving change and the likely directions of change for the spreadsheet. We then set out key directions for further research in the evolution and use of spreadsheets. Finally we look at the implications of these trends for the end users who after all are the reason for the remarkable success of the spreadsheet.

cs.CY↗

arXiv2017

Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Security

Deep learning methods can play a crucial role in anomaly detection, prediction, and supporting decision making for applications like personal health-care, pervasive body sensing, etc. However, current architecture of deep networks suffers the privacy issue that users need to give out their data to the model (typically hosted in a server or a cluster on Cloud) for training or prediction. This problem is getting more severe for those sensitive health-care or medical data (e.g fMRI or body sensors measures like EEG signals). In addition to this, there is also a security risk of leaking these data during the data transmission from user to the model (especially when it's through Internet). Targeting at these issues, in this paper we proposed a new architecture for deep network in which users don't reveal their original data to the model. In our method, feed-forward propagation and data encryption are combined into one process: we migrate the first layer of deep network to users' local devices, and apply the activation functions locally, and then use "dropping activation output" method to make the output non-invertible. The resulting approach is able to make model prediction without accessing users' sensitive raw data. Experiment conducted in this paper showed that our approach achieves the desirable privacy protection requirement, and demonstrated several advantages over the traditional approach with encryption / decryption

cs.CV↗

arXiv2017

DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG

The present study proposes a deep learning model, named DeepSleepNet, for automatic sleep stage scoring based on raw single-channel EEG. Most of the existing methods rely on hand-engineered features which require prior knowledge of sleep analysis. Only a few of them encode the temporal information such as transition rules, which is important for identifying the next sleep stages, into the extracted features. In the proposed model, we utilize Convolutional Neural Networks to extract time-invariant features, and bidirectional-Long Short-Term Memory to learn transition rules among sleep stages automatically from EEG epochs. We implement a two-step training algorithm to train our model efficiently. We evaluated our model using different single-channel EEGs (F4-EOG(Left), Fpz-Cz and Pz-Oz) from two public sleep datasets, that have different properties (e.g., sampling rate) and scoring standards (AASM and R&K). The results showed that our model achieved similar overall accuracy and macro F1-score (MASS: 86.2%-81.7, Sleep-EDF: 82.0%-76.9) compared to the state-of-the-art methods (MASS: 85.9%-80.5, Sleep-EDF: 78.9%-73.7) on both datasets. This demonstrated that, without changing the model architecture and the training algorithm, our model could automatically learn features for sleep stage scoring from different raw single-channel EEGs from different datasets without utilizing any hand-engineered features.

stat.ML↗