arXiv subjects

Jie Li

Publications and source records attributed to Jie Li.

At least 613 records · Page 34Linked to original sources

arXiv2020

QoE-driven Coupled Uplink and Downlink Rate Adaptation for 360-degree Video Live Streaming

360-degree video provides an immersive 360-degree viewing experience and has been widely used in many areas. The 360-degree video live streaming systems involve capturing, compression, uplink (camera to video server) and downlink (video server to user) transmissions. However, few studies have jointly investigated such complex systems, especially the rate adaptation for the coupled uplink and downlink in the 360-degree video streaming under limited bandwidth constraints. In this letter, we propose a quality of experience (QoE)-driven 360-degree video live streaming system, in which a video server performs rate adaptation based on the uplink and downlink bandwidths and information concerning each user's real-time field-of-view (FOV). We formulate it as a nonlinear integer programming problem and propose an algorithm, which combines the Karush-Kuhn-Tucker (KKT) condition and branch and bound method, to solve it. The numerical results show that the proposed optimization model can improve users' QoE significantly in comparison with other baseline schemes.

cs.MM↗

arXiv2020

Speaker-aware speech-transformer

Recently, end-to-end (E2E) models become a competitive alternative to the conventional hybrid automatic speech recognition (ASR) systems. However, they still suffer from speaker mismatch in training and testing condition. In this paper, we use Speech-Transformer (ST) as the study platform to investigate speaker aware training of E2E models. We propose a model called Speaker-Aware Speech-Transformer (SAST), which is a standard ST equipped with a speaker attention module (SAM). The SAM has a static speaker knowledge block (SKB) that is made of i-vectors. At each time step, the encoder output attends to the i-vectors in the block, and generates a weighted combined speaker embedding vector, which helps the model to normalize the speaker variations. The SAST model trained in this way becomes independent of specific training speakers and thus generalizes better to unseen testing speakers. We investigate different factors of SAM. Experimental results on the AISHELL-1 task show that SAST achieves a relative 6.5% CER reduction (CERR) over the speaker-independent (SI) baseline. Moreover, we demonstrate that SAST still works quite well even if the i-vectors in SKB all come from a different data source other than the acoustic training set.

cs.CL↗

arXiv2019

Mission Oriented Miniature Fixed-wing UAV Swarms: A Multi-layered and Distributed Architecture

UAV swarms have triggered wide concern due to their potential application values in recent years. While there are studies proposed in terms of the architecture design for UAV swarms, two main challenges still exist: (1) Scalability, supporting a large scale of vehicles; (2) Versatility, integrating diversified missions. To this end, a multi-layered and distributed architecture for mission oriented miniature fixed-wing UAV swarms is presented in this paper. The proposed architecture is built on the concept of modularity. It divides the overall system to five layers: low-level control, high-level control, coordination, communication and human interaction layers, and many modules that can be viewed as black boxes with interfaces of inputs and outputs. In this way, not only the complexity of developing a large system can be reduced, but also the versatility of supporting diversified missions can be ensured. Furthermore, the proposed architecture is fully distributed that each UAV performs the decision-making procedure autonomously so as to achieve better scalability. Moreover, different kinds of aerial platforms can be feasibly extended by using the control allocation matrices and the integrated hardware box. A prototype swarm system based on the proposed architecture is built and the proposed architecture is evaluated through field experiments with a scale of 21 fixed-wing UAVs. Particularly, to the best of our knowledge, this paper is the first work which successfully demonstrates formation flight, target recognition and tracking missions within an integrated architecture for fixed-wing UAV swarms through field experiments.

cs.MA↗

arXiv2019

Noise robustness of synchronization of two nanomechanical resonators coupled to the same cavity field

We study synchronization of a room temperature optomechanical system formed by two resonators coupled via radiation pressure to the same driven optical cavity mode. By using stochastic Langevin equations and effective slowly-varying amplitude equations, we explore the long-time dynamics of the system. We see that thermal noise can induce significant non-Gaussian dynamical properties, including the coexistence of multi-stable synchronized limit cycles and phase diffusion. Synchronization in this optomechanical system is very robust with respect to thermal noise: in fact, even though each oscillator phase progressively diffuses over the whole limit cycle, their phase difference is locked, and such a phase correlation remains strong in the presence of thermal noise.

physics.optics↗

arXiv2019

Accurate Prediction of Chemical Shifts for Aqueous Protein Structure for "Real World" Cases using Machine Learning

Accurate prediction of NMR chemical shifts can in principle help refine aqueous solution structure of proteins to the quality of X-ray structures. We report a new machine learning algorithm for protein chemical shift prediction that outperforms existing chemical shift calculators on realistic NMR solution data. Our UCBShift predictor implements two modules: a transfer prediction module that employs both sequence and structural alignment to select reference candidates for experimental chemical shift replication, and a redesigned machine learning module based on random forest regression which utilizes more, and more carefully curated, feature extracted data. When combined together, this new predictor achieves state of the art accuracy for predicting chemical shifts on a "real-world" dataset, with root-mean-square errors of 0.31 ppm for amide hydrogens, 0.19 ppm for Halpha, 0.87 ppm for C, 0.81 ppm for Calpha, 1.01 ppm for Cbeta, and 1.83 ppm for N, exceeding current prediction accuracy of popular chemical shift predictors such as SPARTA+ and SHIFTX2.

physics.chem-ph↗

arXiv2019

Boosting Real-Time Driving Scene Parsing with Shared Semantics

Real-time scene parsing is a fundamental feature for autonomous driving vehicles with multiple cameras. In this letter we demonstrate that sharing semantics between cameras with different perspectives and overlapped views can boost the parsing performance when compared with traditional methods, which individually process the frames from each camera. Our framework is based on a deep neural network for semantic segmentation but with two kinds of additional modules for sharing and fusing semantics. On the one hand, a semantics sharing module is designed to establish the pixel-wise mapping between the input images. Features as well as semantics are shared by the map to reduce duplicated workload which leads to more efficient computation. On the other hand, feature fusion modules are designed to combine different modal of semantic features, which leverage the information from both inputs for better accuracy. To evaluate the effectiveness of the proposed framework, we have applied our network to a dual-camera vision system for driving scene parsing. Experimental results show that our network outperforms the baseline method on the parsing accuracy with comparable computations.

cs.CV↗

arXiv2019

Two Stream Networks for Self-Supervised Ego-Motion Estimation

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.

cs.CV↗

arXiv2019

Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances

Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks. Initially viewed as a direct regression problem, requiring annotated labels as supervision at training time, in the past few years a substantial amount of work has been done in self-supervised depth training based on strong geometric cues, both from stereo cameras and more recently from monocular video sequences. In this paper we investigate how these two approaches (supervised & self-supervised) can be effectively combined, so that a depth model can learn to encode true scale from sparse supervision while achieving high fidelity local accuracy by leveraging geometric cues. To this end, we propose a novel supervised loss term that complements the widely used photometric loss, and show how it can be used to train robust semi-supervised monocular depth estimation models. Furthermore, we evaluate how much supervision is actually necessary to train accurate scale-aware monocular depth models, showing that with our proposed framework, very sparse LiDAR information, with as few as 4 beams (less than 100 valid depth values per image), is enough to achieve results competitive with the current state-of-the-art.

cs.CV↗

arXiv2019

AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Results

This paper reviews the AIM 2019 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided in the challenge. In Track 1: Source Domain the aim is to super-resolve such images while preserving the low level image characteristics of the source input domain. In Track 2: Target Domain a set of high-quality images is also provided for training, that defines the output domain and desired quality of the super-resolved images. To allow for quantitative evaluation, the source input images in both tracks are constructed using artificial, but realistic, image degradations. The challenge is the first of its kind, aiming to advance the state-of-the-art and provide a standard benchmark for this newly emerging task. In total 7 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.

cs.CV↗

arXiv2019

HighEr-Resolution Network for Image Demosaicing and Enhancing

Neural-networks based image restoration methods tend to use low-resolution image patches for training. Although higher-resolution image patches can provide more global information, state-of-the-art methods cannot utilize them due to their huge GPU memory usage, as well as the instable training process. However, plenty of studies have shown that global information is crucial for image restoration tasks like image demosaicing and enhancing. In this work, we propose a HighEr-Resolution Network (HERN) to fully learning global information in high-resolution image patches. To achieve this, the HERN employs two parallel paths to learn image features in two different resolutions, respectively. By combining global-aware features and multi-scale features, our HERN is able to learn global information with feasible GPU memory usage. Besides, we introduce a progressive training method to solve the instability issue and accelerate model convergence. On the task of image demosaicing and enhancing, our HERN achieves state-of-the-art performance on the AIM2019 RAW to RGB mapping challenge. The source code of our implementation is available at https://github.com/MKFMIKU/RAW2RGBNet.

eess.IV↗

arXiv2019

DeLottery: A Novel Decentralized Lottery System Based on Blockchain Technology

In this paper, we design DeLottery, a decentralized lottery system based on block chain technology and smart contracts. Lottery is a classical form of entertainment and charity for centuries. Facing the bottleneck of the combination between lottery and information technology, we use smart contracts and blockchain in decentralized, intelligent, and secure systems for lottery industries. Moreover, we are inspired by the algorithm of RANDAO, an outstanding way of random number generation in blockchain scenario. The components and the functions of the novel system are described in details. We implement DeLottery in a blockchain network and show functioning procedure and security of the proposed lottery system.

cs.DC↗

arXiv2019

MoB2:a new multifunctional transition metal diboride monolayer

Several layered transition metal borides can now be realized by a simple and general fabrication method [Fokwa et al.,Adv. Mater. 2018, 30, 1704181],inspiring our interest to transition metal borides monolayer. Here, we predict a new two-dimensional (2D) transition metal diboride MoB2 monolayer (ML) and study its intrinsic mechanical, thermal, electronic, and transport properties. The MoB2 ML has isotropic mechanic properties along the zigzag and armchair directions with a large Young's stiffness, and has an ultralow room-temperature thermal conductivity. The Mo atoms dominate the metallic nature of MoB2 ML. It shows an obvious electrical anisotropy and a current-limiting behavior. Our findings suggest that MoB2 ML is a promising multifunctional material used in ultrathin high-strength mechanical materials, heat insulating materials, electrical-anisotropy-based materials, and current limiters. It is helpful for the experimentalists to further prepare and utilize the transition metal diboride 2D materials.

cond-mat.mtrl-sci↗

arXiv2019

Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning

Reconstructing visual stimulus (image) only from human brain activity measured with functional Magnetic Resonance Imaging (fMRI) is a significant and meaningful task in Human-AI collaboration. However, the inconsistent distribution and representation between fMRI signals and visual images cause the heterogeneity gap. Moreover, the fMRI data is often extremely high-dimensional and contains a lot of visually-irrelevant information. Existing methods generally suffer from these issues so that a satisfactory reconstruction is still challenging. In this paper, we show that it is possible to overcome these challenges by learning visually-guided cognitive latent representations from the fMRI signals, and inversely decoding them to the image stimuli. The resulting framework is called Dual-Variational Autoencoder/ Generative Adversarial Network (D-VAE/GAN), which combines the advantages of adversarial representation learning with knowledge distillation. In addition, we introduce a novel three-stage learning approach which enables the (cognitive) encoder to gradually distill useful knowledge from the paired (visual) encoder during the learning process. Extensive experimental results on both artificial and natural images have demonstrated that our method could achieve surprisingly good results and outperform all other alternatives.

cs.CV↗

arXiv2019

Trident Segmentation CNN: A Spatiotemporal Transformation CNN for Punctate White Matter Lesions Segmentation in Preterm Neonates

Accurate segmentation of punctate white matter lesions (PWML) in preterm neonates by an automatic algorithm can better assist doctors in diagnosis. However, the existing algorithms have many limitations, such as low detection accuracy and large resource consumption. In this paper, a novel spatiotemporal transformation deep learning method called Trident Segmentation CNN (TS-CNN) is proposed to segment PWML in MR images. It can convert spatial information into temporal information, which reduces the consumption of computing resources. Furthermore, a new improved training loss called Self-balancing Focal Loss (SBFL) is proposed to balance the loss during the training process. The whole model is evaluated on a dataset of 704 MR images. Overall the method achieves median DSC, sensitivity, specificity, and Hausdorff distance of 0.6355, 0.7126, 0.9998, and 24.5836 mm which outperforms the state-of-the-art algorithm. (The code is now available on https://github.com/YalongLiu/Trident-Segmentation-CNN)

eess.IV↗

arXiv2019

AF-DCGAN: Amplitude Feature Deep Convolutional GAN for Fingerprint Construction in Indoor Localization Systems

With widely deployed WiFi network and the uniqueness feature (fingerprint) of wireless channel information, fingerprinting based WiFi positioning is currently the mainstream indoor positioning method, in which fingerprint database construction is crucial. However, for accuracy, this approach requires enough data to be sampled at many reference points, which consumes excessive efforts and time. In this paper, we collect Channel State Information (CSI) data at reference points by the method of device-free localization, then we convert collected CSI data into amplitude feature maps and extend the fingerprint database using the proposed Amplitude-Feature Deep Convolutional Generative Adversarial Network (AF-DCGAN) model. The use of AF-DCGAN accelerates convergence during the training phase, and substantially increases the diversity of the CSI amplitude feature map. The extended fingerprint database both reduces the human effort involved in fingerprint database construction and the accuracy of an indoor localization system, as demonstrated in the experiments.

cs.NI↗

arXiv2019

Node similarity distribution of complex networks and its application in link prediction

Over the years, quantifying the similarity of nodes has been a hot topic in complex networks, yet little has been known about the distributions of node-similarity. In this paper, we consider a typical measure of node-similarity called the common neighbor based similarity (CNS). By means of the generating function, we propose a general framework for calculating the CNS distributions of node sets in various complex networks. In particular, we show that for the Erdös-Rényi (ER) random network, the CNS distribution of node sets of any particular size obeys the Poisson law. We also connect the node-similarity distribution to the link prediction problem. We found that the performance of link prediction depends solely on the CNS distributions of the connected and unconnected node pairs in the network. Furthermore, we derive theoretical solutions of two key evaluation metrics in link prediction: i) precision and ii) area under the receiver operating characteristic curve (AUC). We show that for any link prediction method, if the similarity distributions of the connected and unconnected node pairs are identical, the AUC will be $0.5$. The theoretical solutions are elegant alternatives of the traditional experimental evaluation methods with nevertheless much lower computational cost.

cs.SI↗

arXiv2019

ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surround-view image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature points located in the free space area are utilized to ensure the 2D-2D matching for pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize the feature points reprojection error and scan matching error. We apply our system to an autonomous vehicle equipped with four fisheye cameras. The testing scenarios include an outdoor parking lot as well as an indoor garage. Experimental results demonstrate that our system achieves a more robust and accurate performance comparing with a fisheye camera based monocular visual odometry system.

cs.CV↗

arXiv2019

Identifying significant edges via neighborhood information

Heterogeneous nature of real networks implies that different edges play different roles in network structure and functions, and thus to identify significant edges is of high value in both theoretical studies and practical applications. We propose the so-called second-order neighborhood (SN) index to quantify an edge's significance in a network. We compare SN index with many other benchmark methods based on 15 real networks via edge percolation. Results show that the proposed SN index outperforms other well-known methods.

physics.soc-ph↗