arXiv subjects

Lin Wang

Publications and source records attributed to Lin Wang.

At least 397 records · Page 22Linked to original sources

arXiv2021

Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks

Deep neural models in recent years have been successful in almost every field, including extremely complex problem statements. However, these models are huge in size, with millions (and even billions) of parameters, thus demanding more heavy computation power and failing to be deployed on edge devices. Besides, the performance boost is highly dependent on redundant labeled data. To achieve faster speeds and to handle the problems caused by the lack of data, knowledge distillation (KD) has been proposed to transfer information learned from one model to another. KD is often characterized by the so-called `Student-Teacher' (S-T) learning framework and has been broadly applied in model compression and knowledge transfer. This paper is about KD and S-T learning, which are being actively studied in recent years. First, we aim to provide explanations of what KD is and how/why it works. Then, we provide a comprehensive survey on the recent progress of KD methods together with S-T frameworks typically for vision tasks. In general, we consider some fundamental questions that have been driving this research area and thoroughly generalize the research progress and technical details. Additionally, we systematically analyze the research status of KD in vision applications. Finally, we discuss the potentials and open challenges of existing methods and prospect the future directions of KD and S-T learning.

cs.CV↗

arXiv2021

FAST Globular Cluster Pulsar Survey: Twenty-Four Pulsars Discovered in Fifteen Globular Clusters

We present the discovery of 24 pulsars in 15 Globular Clusters (GCs) using the Five-hundred-meter Aperture Spherical radio Telescope (FAST). These include the first pulsar discoveries in M2, M10, and M14. Most of the new systems are either confirmed or likely members of binary systems. M53C, NGC6517H and I are the only three pulsars confirmed to be isolated. M14A is a black widow pulsar with an orbital period of 5.5 hours and a minimum companion mass of 0.016 \Ms. M14E is an eclipsing binary pulsar with an orbital period of 20.3 hours. With the other 8 discoveries that have been reported elsewhere, in total 32 GC pulsars have been discovered by FAST so far. In addition, We detected M3A twice. This was enough to determine that it is a black widow pulsar with an orbital period of 3.3 hours and a minimum companion mass of 0.0125 \Ms.

astro-ph.HE↗

arXiv2021

Orthogonal Subsampling for Big Data Linear Regression

The dramatic growth of big datasets presents a new challenge to data storage and analysis. Data reduction, or subsampling, that extracts useful information from datasets is a crucial step in big data analysis. We propose an orthogonal subsampling (OSS) approach for big data with a focus on linear regression models. The approach is inspired by the fact that an orthogonal array of two levels provides the best experimental design for linear regression models in the sense that it minimizes the average variance of the estimated parameters and provides the best predictions. The merits of OSS are three-fold: (i) it is easy to implement and fast; (ii) it is suitable for distributed parallel computing and ensures the subsamples selected in different batches have no common data points; and (iii) it outperforms existing methods in minimizing the mean squared errors of the estimated parameters and maximizing the efficiencies of the selected subsamples. Theoretical results and extensive numerical results show that the OSS approach is superior to existing subsampling approaches. It is also more robust to the presence of interactions among covariates and, when they do exist, OSS provides more precise estimates of the interaction effects than existing methods. The advantages of OSS are also illustrated through analysis of real data.

stat.ME↗

arXiv2021

DEFT: Distilling Entangled Factors by Preventing Information Diffusion

Disentanglement is a highly desirable property of representation owing to its similarity to human understanding and reasoning. Many works achieve disentanglement upon information bottlenecks (IB). Despite their elegant mathematical foundations, the IB branch usually exhibits lower performance. In order to provide an insight into the problem, we develop an annealing test to calculate the information freezing point (IFP), which is a transition state to freeze information into the latent variables. We also explore these clues or inductive biases for separating the entangled factors according to the differences in the IFP distributions. We found the existing approaches suffer from the information diffusion problem, according to which the increased information diffuses in all latent variables. Based on this insight, we propose a novel disentanglement framework, termed the distilling entangled factor (DEFT), to address the information diffusion problem by scaling backward information. DEFT applies a multistage training strategy, including multigroup encoders with different learning rates and piecewise disentanglement pressure, to disentangle the factors stage by stage. We evaluate DEFT on three variants of dSprite and SmallNORB, which show low-variance and high-level disentanglement scores. Furthermore, the experiment under the correlative factors shows incapable of TC-based approaches. DEFT also exhibits a competitive performance in the unsupervised setting.

cs.LG↗

arXiv2021

Large depth of range Maxwellian-viewing SMV near-eye display based on a Pancharatnam-Berry optical element

In order to overcome the accommodation and convergence (A-C) conflict that commonly causes visual fatigue in AR display, we propose a Maxwellian-viewing-super-multi-view (MV-SMV) near-eye display system based on a Pancharatnam-Berry optical element (PBOE). The PBOE, which is constituted with an array of high-efficiency polarization gratings, is implemented to direct different views to different directions simultaneously, constructing the 3D light field. Meanwhile, each view is like a Maxwellian view display that possesses a small viewpoint and a large depth of field (DOF). Hence, the MV-SMV display can display virtual images with correct accommodation depth cue within a large DOF. We implement a proof-of-concept MV-SMV display prototype with 3 x 1 and 3 x 2 viewpoints using a 1D PBOE and a 2D PBOE, respectively, and achieve a DOF of 4.37 diopters experimentally.

physics.optics↗

arXiv2021

Knowledge-Based Prediction of Network Controllability Robustness

Network controllability robustness reflects how well a networked system can maintain its controllability against destructive attacks. Its measure is quantified by a sequence of values that record the remaining controllability of the network after a sequence of node-removal or edge-removal attacks. Traditionally, the controllability robustness is determined by attack simulations, which is computationally time consuming or even infeasible. In the present paper, an improved method for predicting the network controllability robustness is developed based on machine learning using a group of convolutional neural networks (CNNs). In this scheme, a number of training data generated by simulations are used to train the group of CNNs for classification and prediction, respectively. Extensive experimental studies are carried out, which demonstrate that 1) the proposed method predicts more precisely than the classical single-CNN predictor; 2) the proposed CNN-based predictor provides a better predictive measure than the traditional spectral measures and network heterogeneity.

physics.soc-ph↗

arXiv2021

Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition

In the real world, medical datasets often exhibit a long-tailed data distribution (i.e., a few classes occupy most of the data, while most classes have rarely few samples), which results in a challenging imbalance learning scenario. For example, there are estimated more than 40 different kinds of retinal diseases with variable morbidity, however with more than 30+ conditions are very rare from the global patient cohorts, which results in a typical long-tailed learning problem for deep learning-based screening models. In this study, we propose class subset learning by dividing the long-tailed data into multiple class subsets according to prior knowledge, such as regions and phenotype information. It enforces the model to focus on learning the subset-specific knowledge. More specifically, there are some relational classes that reside in the fixed retinal regions, or some common pathological features are observed in both the majority and minority conditions. With those subsets learnt teacher models, then we are able to distill the multiple teacher models into a unified model with weighted knowledge distillation loss. The proposed framework proved to be effective for the long-tailed retinal diseases recognition task. The experimental results on two different datasets demonstrate that our method is flexible and can be easily plugged into many other state-of-the-art techniques with significant improvements.

cs.CV↗

arXiv2021

A Novel Neural Network-Based Symbolic Regression Method: Neuro-Encoded Expression Programming

Neuro-encoded expression programming(NEEP) that aims to offer a novel continuous representation of combinatorial encoding for genetic programming methods is proposed in this paper. Genetic programming with linear representation uses nature-inspired operators (e.g., crossover, mutation) to tune expressions and finally search out the best explicit function to simulate data. The encoding mechanism is essential for genetic programmings to find a desirable solution efficiently. However, the linear representation methods manipulate the expression tree in discrete solution space, where a small change of the input can cause a large change of the output. The unsmooth landscapes destroy the local information and make difficulty in searching. The neuro-encoded expression programming constructs the gene string with recurrent neural network (RNN) and the weights of the network are optimized by powerful continuous evolutionary algorithms. The neural network mappings smoothen the sharp fitness landscape and provide rich neighborhood information to find the best expression. The experiments indicate that the novel approach improves training efficiency and reduces test errors on several well-known symbolic regression problems.

cs.NE↗

arXiv2021

Conspiracy vs science: A large-scale analysis of online discussion cascades

With the emergence and rapid proliferation of social media platforms and social networking sites, recent years have witnessed a surge of misinformation spreading in our daily life. Drawing on a large-scale dataset which covers more than 1.4M posts and 18M comments, we investigate the propagation of two distinct narratives--(i) conspiracy information, whose claims are generally unsubstantiated and thus referred as misinformation to some extent, and (ii) scientific information, whose origins are generally readily identifiable and verifiable--in an online social media platform. We find that conspiracy cascades tend to propagate in a multigenerational branching process while science cascades are more likely to grow in a breadth-first manner. Specifically, conspiracy information triggers larger cascades, involves more users and generations, persists longer, is more viral and bursty than science information. Content analysis reveals that conspiracy cascades contain more negative words and emotional words which convey anger, fear, disgust, surprise and trust. We also find that conspiracy cascades are more concerned with political and controversial topics. After applying machine learning models, we achieve an AUC score of nearly 90% in discriminating conspiracy from science narratives. We find that conspiracy cascades are more likely to be controlled by a broader set of users than science cascades, imposing new challenges on the management of misinformation. Although political affinity is thought to affect the consumption of misinformation, there is very little evidence that political orientation of the information source plays a role during the propagation of conspiracy information. Our study provides complementing evidence to current misinformation research and has practical policy implications to stem the propagation and mitigate the influence of misinformation online.

cs.SI↗

arXiv2021

Three Pulsars Discovered by FAST in the Globular Cluster NGC 6517 with a Pulsar Candidate Sifting Code Based on Dispersion Measure to Signal-to-Noise Ratio Plots

We report the discovery of three new pulsars in the Globular Cluster (GC) NGC6517, namely NGC 6517 E, F, and G, made with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The spin periods of NGC 6517 E, F, and G are 7.60~ms, 24.89~ms, and 51.59~ms, respectively. Their dispersion measures are 183.29, 183.713, and 185.3~pc~cm$^{-3}$, respectively, all slightly larger than those of the previously known pulsars in this cluster. The spin period derivatives are at the level of 1$\times$10$^{-18}$~s~s$^{-1}$, which suggests these are recycled pulsars. In addition to the discovery of these three new pulsars, we updated the timing solutions of the known isolated pulsars, NGC 6517 A, C, and D. The solutions are consistent with those from Lynch et al. (2011) and with smaller timing residuals. From the timing solution, NGC 6517 A, B (position from Lynch et al. 2011), C, E, and F are very close to each other on the sky and only a few arcseconds from the optical core of NGC 6517. With currently published and unpublished discoveries, NGC6517 now has 9 pulsars, ranking 5$^{th}$ of the GCs with the most pulsars. The discoveries take advantage of the high sensitivity of FAST and a new algorithm used to check and filter possible candidate signals.

astro-ph.HE↗

arXiv2021

Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation

Deep neural networks are known to be data-driven and label noise can have a marked impact on model performance. Recent studies have shown great robustness to classic image recognition even under a high noisy rate. In medical applications, learning from datasets with label noise is more challenging since medical imaging datasets tend to have asymmetric (class-dependent) noise and suffer from high observer variability. In this paper, we systematically discuss and define the two common types of label noise in medical images - disagreement label noise from inconsistency expert opinions and single-target label noise from wrong diagnosis record. We then propose an uncertainty estimation-based framework to handle these two label noise amid the medical image classification task. We design a dual-uncertainty estimation approach to measure the disagreement label noise and single-target label noise via Direct Uncertainty Prediction and Monte-Carlo-Dropout. A boosting-based curriculum training procedure is later introduced for robust learning. We demonstrate the effectiveness of our method by conducting extensive experiments on three different diseases: skin lesions, prostate cancer, and retinal diseases. We also release a large re-engineered database that consists of annotations from more than ten ophthalmologists with an unbiased golden standard dataset for evaluation and benchmarking.

cs.CV↗

arXiv2021

Surrogate-assisted cooperative signal optimization for large-scale traffic networks

Reasonable setting of traffic signals can be very helpful in alleviating congestion in urban traffic networks. Meta-heuristic optimization algorithms have proved themselves to be able to find high-quality signal timing plans. However, they generally suffer from performance deterioration when solving large-scale traffic signal optimization problems due to the huge search space and limited computational budget. Directing against this issue, this study proposes a surrogate-assisted cooperative signal optimization (SCSO) method. Different from existing methods that directly deal with the entire traffic network, SCSO first decomposes it into a set of tractable sub-networks, and then achieves signal setting by cooperatively optimizing these sub-networks with a surrogate-assisted optimizer. The decomposition operation significantly narrows the search space of the whole traffic network, and the surrogate-assisted optimizer greatly lowers the computational burden by reducing the number of expensive traffic simulations. By taking Newman fast algorithm, radial basis function and a modified estimation of distribution algorithm as decomposer, surrogate model and optimizer, respectively, this study develops a concrete SCSO algorithm. To evaluate its effectiveness and efficiency, a large-scale traffic network involving crossroads and T-junctions is generated based on a real traffic network. Comparison with several existing meta-heuristic algorithms specially designed for traffic signal optimization demonstrates the superiority of SCSO in reducing the average delay time of vehicles.

cs.NE↗

arXiv2021

An embedded multichannel sound acquisition system for drone audition

Microphone array techniques can improve the acoustic sensing performance on drones, compared to the use of a single microphone. However, multichannel sound acquisition systems are not available in current commercial drone platforms. To encourage the research in drone audition, we present an embedded sound acquisition and recording system with eight microphones and a multichannel sound recorder mounted on a quadcopter. In addition to recording and storing locally the sound from multiple microphones simultaneously, the embedded system can connect wirelessly to a remote terminal to transfer audio files for further processing. This will be the first stage towards creating a fully embedded solution for drone audition. We present experimental results obtained by state-of-the-art drone audition algorithms applied to the sound recorded by the embedded system.

eess.AS↗

arXiv2021

Locally symmetric lattices for storage ring light sources

In this paper, a new lattice concept called the locally symmetric lattice is proposed for storage ring light sources. In this new lattice, beta functions are made locally symmetric about two mirror planes of the lattice cell, and the phase advances between the two mirror planes satisfy the condition of nonlinear dynamics cancellation. There are two kinds of locally symmetric lattices, corresponding to two symmetric representations of lattice cell. In a locally symmetric lattice, main nonlinear effects caused by sextupoles can be effectively cancelled within one lattice cell, and generally there can also be many knobs of sextupoles available for further optimizing the nonlinear dynamics. Two kinds of locally symmetric lattices are designed for a 2.2 GeV diffraction-limited storage ring to demonstrate the lattice concept.

physics.acc-ph↗

arXiv2020

A Novel Graphic Bending Transformation on Benchmark

Classical benchmark problems utilize multiple transformation techniques to increase optimization difficulty, e.g., shift for anti centering effect and rotation for anti dimension sensitivity. Despite testing the transformation invariance, however, such operations do not really change the landscape's "shape", but rather than change the "view point". For instance, after rotated, ill conditional problems are turned around in terms of orientation but still keep proportional components, which, to some extent, does not create much obstacle in optimization. In this paper, inspired from image processing, we investigate a novel graphic conformal mapping transformation on benchmark problems to deform the function shape. The bending operation does not alter the function basic properties, e.g., a unimodal function can almost maintain its unimodality after bent, but can modify the shape of interested area in the search space. Experiments indicate the same optimizer spends more search budget and encounter more failures on the conformal bent functions than the rotated version. Several parameters of the proposed function are also analyzed to reveal performance sensitivity of the evolutionary algorithms.

cs.NE↗

arXiv2020

Lattice dynamics of topological Dirac semimetal LaAgSb$_{2}$ with charge density wave ordering

LaAgSb$_{2}$ is a rare material, which offers the opportunity to investigate the complex interplay between charge density wave (CDW) ordering and topology protected electronic band structure. As both of these phenomena are governed by the structural symmetries, a comprehensive study of the lattice dynamics is highly desirable. In this report, we present the results of temperature and pressure dependent Raman spectroscopy and x-ray diffraction in single crystalline LaAgSb$_{2}$. Our results confirm that Raman spectroscopy is a highly sensitive tool to probe CDW ordering phenomenon, particularly the low-temperature second CDW transition in LaAgSb$_{2}$, which appears as a very weak anomaly in most experiments. The crystal orientation-dependent measurements provide the evolution of Raman modes with crystallographic symmetries and can be further studied through group symmetry analysis. The low-temperature x-ray diffraction data show the emergence of structural modulations corresponding to the CDW instability. The combined high-pressure Raman spectroscopy and synchrotron x-ray diffraction reveal multiple structural phase transitions through lowering of crystalline symmetries, which are also expected to lead to electronic topological transitions.

cond-mat.mtrl-sci↗

arXiv2020

Disentangling Action Sequences: Discovering Correlated Samples

Disentanglement is a highly desirable property of representation due to its similarity with human's understanding and reasoning. This improves interpretability, enables the performance of down-stream tasks, and enables controllable generative models. However, this domain is challenged by the abstract notion and incomplete theories to support unsupervised disentanglement learning. We demonstrate the data itself, such as the orientation of images, plays a crucial role in disentanglement and instead of the factors, and the disentangled representations align the latent variables with the action sequences. We further introduce the concept of disentangling action sequences which facilitates the description of the behaviours of the existing disentangling approaches. An analogy for this process is to discover the commonality between the things and categorizing them. Furthermore, we analyze the inductive biases on the data and find that the latent information thresholds are correlated with the significance of the actions. For the supervised and unsupervised settings, we respectively introduce two methods to measure the thresholds. We further propose a novel framework, fractional variational autoencoder (FVAE), to disentangle the action sequences with different significance step-by-step. Experimental results on dSprites and 3D Chairs show that FVAE improves the stability of disentanglement.

cs.LG↗

arXiv2020

Spin-orbit coupling and linear crossings of dipolar magnons in van der Waals antiferromagnets

A magnon spin-orbit coupling, induced by the dipole-dipole interaction, is derived in monoclinic-stacked bilayer honeycomb spin lattice with perpendicular magnetic anisotropy and antiferromagnetic interlayer coupling. Linear crossings are predicted in the magnon spectrum around the band minimum in G valley, as well as in the high frequency range around the zone boundary. The linear crossings in K and K' valleys, which connect the acoustic and optical bands, can be gapped when the intralayer dipole-dipole or Kitaev interactions exceed the interlayer dipole-dipole interaction, resulting in a phase transition from semimetal to insulator. Our results are useful for analyzing the magnon spin dynamics and transport properties in van der Waals antiferromagnet.

cond-mat.mes-hall↗