arXiv subjects

Lei Huang

Publications and source records attributed to Lei Huang.

At least 199 records · Page 11Linked to original sources

arXiv2022

Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective

Despite recent stereo matching networks achieving impressive performance given sufficient training data, they suffer from domain shifts and generalize poorly to unseen domains. We argue that maintaining feature consistency between matching pixels is a vital factor for promoting the generalization capability of stereo matching networks, which has not been adequately considered. Here we address this issue by proposing a simple pixel-wise contrastive learning across the viewpoints. The stereo contrastive feature loss function explicitly constrains the consistency between learned features of matching pixel pairs which are observations of the same 3D points. A stereo selective whitening loss is further introduced to better preserve the stereo feature consistency across domains, which decorrelates stereo features from stereo viewpoint-specific style information. Counter-intuitively, the generalization of feature consistency between two viewpoints in the same scene translates to the generalization of stereo matching performance to unseen domains. Our method is generic in nature as it can be easily embedded into existing stereo networks and does not require access to the samples in the target domain. When trained on synthetic data and generalized to four real-world testing sets, our method achieves superior performance over several state-of-the-art networks.

cs.CV↗

arXiv2021

Armada: A Robust Latency-Sensitive Edge Cloud in Heterogeneous Edge-Dense Environments

Edge computing has enabled a large set of emerging edge applications by exploiting data proximity and offloading latency-sensitive and computation-intensive workloads to nearby edge servers. However, supporting edge application users at scale in wide-area environments poses challenges due to limited point-of-presence edge sites and constrained elasticity. In this paper, we introduce Armada: a densely-distributed edge cloud infrastructure that explores the use of dedicated and volunteer resources to serve geo-distributed users in heterogeneous environments. We describe the lightweight Armada architecture and optimization techniques including performance-aware edge selection, auto-scaling and load balancing on the edge, fault tolerance, and in-situ data access. We evaluate Armada in both real-world volunteer environments and emulated platforms to show how common edge applications, namely real-time object detection and face recognition, can be easily deployed on Armada serving distributed users at scale with low latency.

cs.DC↗

arXiv2021

The Variability of the Black-Hole Image in M87 at the Dynamical Time Scale

The black-hole images obtained with the Event Horizon Telescope (EHT) are expected to be variable at the dynamical timescale near their horizons. For the black hole at the center of the M87 galaxy, this timescale (5-61 days) is comparable to the 6-day extent of the 2017 EHT observations. Closure phases along baseline triangles are robust interferometric observables that are sensitive to the expected structural changes of the images but are free of station-based atmospheric and instrumental errors. We explored the day-to-day variability in closure phase measurements on all six linearly independent non-trivial baseline triangles that can be formed from the 2017 observations. We showed that three triangles exhibit very low day-to-day variability, with a dispersion of $\sim3-5^\circ$. The only triangles that exhibit substantially higher variability ($\sim90-180^\circ$) are the ones with baselines that cross visibility amplitude minima on the $u-v$ plane, as expected from theoretical modeling. We used two sets of General Relativistic magnetohydrodynamic simulations to explore the dependence of the predicted variability on various black-hole and accretion-flow parameters. We found that changing the magnetic field configuration, electron temperature model, or black-hole spin has a marginal effect on the model consistency with the observed level of variability. On the other hand, the most discriminating image characteristic of models is the fractional width of the bright ring of emission. Models that best reproduce the observed small level of variability are characterized by thin ring-like images with structures dominated by gravitational lensing effects and thus least affected by turbulence in the accreting plasmas.

astro-ph.HE↗

arXiv2021

Rot-Pro: Modeling Transitivity by Projection in Knowledge Graph Embedding

Knowledge graph embedding models learn the representations of entities and relations in the knowledge graphs for predicting missing links (relations) between entities. Their effectiveness are deeply affected by the ability of modeling and inferring different relation patterns such as symmetry, asymmetry, inversion, composition and transitivity. Although existing models are already able to model many of these relations patterns, transitivity, a very common relation pattern, is still not been fully supported. In this paper, we first theoretically show that the transitive relations can be modeled with projections. We then propose the Rot-Pro model which combines the projection and relational rotation together. We prove that Rot-Pro can infer all the above relation patterns. Experimental results show that the proposed Rot-Pro model effectively learns the transitivity pattern and achieves the state-of-the-art results on the link prediction task in the datasets containing transitive relations.

cs.AI↗

arXiv2021

KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks

Kronecker-factored Approximate Curvature (K-FAC) has recently been shown to converge faster in deep neural network (DNN) training than stochastic gradient descent (SGD); however, K-FAC's larger memory footprint hinders its applicability to large models. We present KAISA, a K-FAC-enabled, Adaptable, Improved, and ScAlable second-order optimizer framework that adapts the memory footprint, communication, and computation given specific models and hardware to improve performance and increase scalability. We quantify the tradeoffs between memory and communication cost and evaluate KAISA on large models, including ResNet-50, Mask R-CNN, U-Net, and BERT, on up to 128 NVIDIA A100 GPUs. Compared to the original optimizers, KAISA converges 18.1-36.3% faster across applications with the same global batch size. Under a fixed memory budget, KAISA converges 32.5% and 41.6% faster in ResNet-50 and BERT-Large, respectively. KAISA can balance memory and communication to achieve scaling efficiency equal to or better than the baseline optimizers. KAISA is open source and available at https://github.com/gpauloski/kfac_pytorch.

cs.LG↗

arXiv2021

S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration

Previous studies dominantly target at self-supervised learning on real-valued networks and have achieved many promising results. However, on the more challenging binary neural networks (BNNs), this task has not yet been fully explored in the community. In this paper, we focus on this more difficult scenario: learning networks where both weights and activations are binary, meanwhile, without any human annotated labels. We observe that the commonly used contrastive objective is not satisfying on BNNs for competitive accuracy, since the backbone network contains relatively limited capacity and representation ability. Hence instead of directly applying existing self-supervised methods, which cause a severe decline in performance, we present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution, to minimize the loss and obtain desirable accuracy. Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.5~15% on BNNs. We further reveal that it is difficult for BNNs to recover the similar predictive distributions as real-valued models when training without labels. Thus, how to calibrate them is key to address the degradation in performance. Extensive experiments are conducted on the large-scale ImageNet and downstream datasets. Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods. Code is available at https://github.com/szq0214/S2-BNN.

cs.CV↗

arXiv2021

One-Bit Target Detection in Collocated MIMO Radar and Performance Degradation Analysis

Target detection is an important problem in multiple-input multiple-output (MIMO) radar. Many existing target detection algorithms were proposed without taking into consideration the quantization error caused by analog-to-digital converters (ADCs). This paper addresses the problem of target detection for MIMO radar with one-bit ADCs and derives a Rao's test-based detector. The proposed method has several appealing features: 1) it is a closed-form detector; 2) it allows us to handle sign measurements straightforwardly; 3) there are closed-form approximations of the detector's distributions, which allow us to theoretically evaluate its performance. Moreover, the closed-form distributions allow us to study the performance degradation due to the one-bit ADCs, yielding an approximate $2$ dB loss in the low-signal-to-noise-ratio (SNR) regime compared to $\infty$-bit ADCs. Simulation results are included to showcase the advantage of the proposed detector and validate the accuracy of the theoretical results.

eess.SP↗

arXiv2021

Constraints on black-hole charges with the 2017 EHT observations of M87*

Our understanding of strong gravity near supermassive compact objects has recently improved thanks to the measurements made by the Event Horizon Telescope (EHT). We use here the M87* shadow size to infer constraints on the physical charges of a large variety of nonrotating or rotating black holes. For example, we show that the quality of the measurements is already sufficient to rule out that M87* is a highly charged dilaton black hole. Similarly, when considering black holes with two physical and independent charges, we are able to exclude considerable regions of the space of parameters for the doubly-charged dilaton and the Sen black holes.

gr-qc↗

arXiv2021

The Polarized Image of a Synchrotron Emitting Ring of Gas Orbiting a Black Hole

Synchrotron radiation from hot gas near a black hole results in a polarized image. The image polarization is determined by effects including the orientation of the magnetic field in the emitting region, relativistic motion of the gas, strong gravitational lensing by the black hole, and parallel transport in the curved spacetime. We explore these effects using a simple model of an axisymmetric, equatorial accretion disk around a Schwarzschild black hole. By using an approximate expression for the null geodesics derived by Beloborodov (2002) and conservation of the Walker-Penrose constant, we provide analytic estimates for the image polarization. We test this model using currently favored general relativistic magnetohydrodynamic simulations of M87*, using ring parameters given by the simulations. For a subset of these with modest Faraday effects, we show that the ring model broadly reproduces the polarimetric image morphology. Our model also predicts the polarization evolution for compact flaring regions, such as those observed from Sgr A* with GRAVITY. With suitably chosen parameters, our simple model can reproduce the EVPA pattern and relative polarized intensity in Event Horizon Telescope images of M87*. Under the physically motivated assumption that the magnetic field trails the fluid velocity, this comparison is consistent with the clockwise rotation inferred from total intensity images.

astro-ph.HE↗

arXiv2021

One-bit Spectrum Sensing with the Eigenvalue Moment Ratio Approach

One-bit analog-to-digital converter (ADC), performing signal sampling as an extreme simple comparator, is an overwhelming technology for spectrum sensing due to its low-cost, low-power consumptions and high sampling rate. In this letter, we propose a novel one-bit sensing approach based on the eigenvalue moment ratio (EMR), which has been proved to be highly efficient for conventional multi-antenna spectrum sensing in $\infty$-bit situation. Particularly, we determine the asymptotic distribution of one-bit EMR under null hypothesis via the central limited theorem (CLT), allowing us to perform spectrum sensing with one-bit samples directly. Theoretical and simulation analysis show the new approach can provide reasonably good sensing performance at a low hardware cost.

cs.IT↗

arXiv2021

Group Whitening: Balancing Learning Efficiency and Representational Capacity

Batch normalization (BN) is an important technique commonly incorporated into deep learning models to perform standardization within mini-batches. The merits of BN in improving a model's learning efficiency can be further amplified by applying whitening, while its drawbacks in estimating population statistics for inference can be avoided through group normalization (GN). This paper proposes group whitening (GW), which exploits the advantages of the whitening operation and avoids the disadvantages of normalization within mini-batches. In addition, we analyze the constraints imposed on features by normalization, and show how the batch size (group number) affects the performance of batch (group) normalized networks, from the perspective of model's representational capacity. This analysis provides theoretical guidance for applying GW in practice. Finally, we apply the proposed GW to ResNet and ResNeXt architectures and conduct experiments on the ImageNet and COCO benchmarks. Results show that GW consistently improves the performance of different architectures, with absolute gains of $1.02\%$ $\sim$ $1.49\%$ in top-1 accuracy on ImageNet and $1.82\%$ $\sim$ $3.21\%$ in bounding box AP on COCO.

cs.LG↗

arXiv2021

Slimmable Generative Adversarial Networks

Generative adversarial networks (GANs) have achieved remarkable progress in recent years, but the continuously growing scale of models makes them challenging to deploy widely in practical applications. In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power. In this paper, we introduce slimmable GANs (SlimGANs), which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime. Specifically, we leverage multiple discriminators that share partial parameters to train the slimmable generator. To facilitate the \textit{consistency} between generators of different widths, we present a stepwise inplace distillation technique that encourages narrow generators to learn from wide ones. As for class-conditional generation, we propose a sliceable conditional batch normalization that incorporates the label information into different widths. Our methods are validated, both quantitatively and qualitatively, by extensive experiments and a detailed ablation study.

cs.LG↗

arXiv2021

Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification

Chest X-rays are an important and accessible clinical imaging tool for the detection of many thoracic diseases. Over the past decade, deep learning, with a focus on the convolutional neural network (CNN), has become the most powerful computer-aided diagnosis technology for improving disease identification performance. However, training an effective and robust deep CNN usually requires a large amount of data with high annotation quality. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. Thus, existing public chest X-ray datasets usually adopt language pattern based methods to automatically mine labels from reports. However, this results in label uncertainty and inconsistency. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods from two perspectives to improve a single model's disease identification performance, rather than focusing on an ensemble of models. MODL integrates multiple models to obtain a soft label distribution for optimizing the single target model, which can reduce the effects of original label uncertainty. Moreover, KNNS aims to enhance the robustness of the target model to provide consistent predictions on images with similar medical findings. Extensive experiments on the public NIH Chest X-ray and CheXpert datasets show that our model achieves consistent improvements over the state-of-the-art methods.

cs.CV↗

arXiv2021

An Automatic Observation Management System of the GWAC Network I: System Architecture and Workflow

The GWAC-N is an observation network composed of multi-aperture and multi-field of view robotic optical telescopes. The main instruments are the GWAC-A. Besides, several robotic optical telescopes with narrower field of views provide fast follow-up multi-band capabilities to the GWAC-N. The primary scientific goal of the GWAC-N is to search for the optical counterparts of GRB that will be detected by the SVOM. The GWAC-N performs many other observing tasks including the follow-ups of ToO and both the detection and the monitoring of variable/periodic objects as well as optical transients. To handle all of those scientific cases, we designed 10 observation modes and 175 observation strategies, especially, a joint observation strategy with multiple telescopes of the GWAC-N for the follow-up of GW events. To perform these observations, we thus develop an AOM system in charge of the object management, the dynamic scheduling of the observation plan and its automatic broadcasting to the network management and finally the image management. The AOM combines the individual telescopes into a network and smoothly organizes all the associated operations. The system completely meets the requirements of the GWAC-N on all its science objectives. With its good portability, the AOM is scientifically and technically qualified for other general purposed telescope networks. As the GWAC-N extends and evolves, the AOM will greatly enhance the discovery potential for the GWAC-N. In the first paper of a series of publications, we present the scientific goals of the GWAC-N as well as the hardware, the software and the strategy setup to achieve the scientific objectives. The structure, the technical design, the implementation and performances of the AOM will be also described in details. In the end, we summarize the current status of the GWAC-N and prospect for the development plan in the near future.

astro-ph.IM↗

arXiv2021

EGFI: Drug-Drug Interaction Extraction and Generation with Fusion of Enriched Entity and Sentence Information

The rapid growth in literature accumulates diverse and yet comprehensive biomedical knowledge hidden to be mined such as drug interactions. However, it is difficult to extract the heterogeneous knowledge to retrieve or even discover the latest and novel knowledge in an efficient manner. To address such a problem, we propose EGFI for extracting and consolidating drug interactions from large-scale medical literature text data. Specifically, EGFI consists of two parts: classification and generation. In the classification part, EGFI encompasses the language model BioBERT which has been comprehensively pre-trained on biomedical corpus. In particular, we propose the multi-head attention mechanism and pack BiGRU to fuse multiple semantic information for rigorous context modeling. In the generation part, EGFI utilizes another pre-trained language model BioGPT-2 where the generation sentences are selected based on filtering rules. We evaluated the classification part on "DDIs 2013" dataset and "DTIs" dataset, achieving the FI score of 0.842 and 0.720 respectively. Moreover, we applied the classification part to distinguish high-quality generated sentences and verified with the exiting growth truth to confirm the filtered sentences. The generated sentences that are not recorded in DrugBank and DDIs 2013 dataset also demonstrate the potential of EGFI to identify novel drug relationships.

cs.CL↗

arXiv2020

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

People with diabetes are at risk of developing an eye disease called diabetic retinopathy (DR). This disease occurs when high blood glucose levels cause damage to blood vessels in the retina. Computer-aided DR diagnosis is a promising tool for early detection of DR and severity grading, due to the great success of deep learning. However, most current DR diagnosis systems do not achieve satisfactory performance or interpretability for ophthalmologists, due to the lack of training data with consistent and fine-grained annotations. To address this problem, we construct a large fine-grained annotated DR dataset containing 2,842 images (FGADR). This dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency. The proposed dataset will enable extensive studies on DR diagnosis. We set up three benchmark tasks for evaluation: 1. DR lesion segmentation; 2. DR grading by joint classification and segmentation; 3. Transfer learning for ocular multi-disease identification. Moreover, a novel inductive transfer learning method is introduced for the third task. Extensive experiments using different state-of-the-art methods are conducted on our FGADR dataset, which can serve as baselines for future research.

cs.CV↗

arXiv2020

The eigenvalue of the confined potential

Analytic solutions for the energy eigenvalues are obtained from a confined potentials of the form $br$ in 3 dimensions. The confinement is effected by linear term which is a very important part in Cornell potential. The analytic eigenvalues and numerical solutions are exactly matched.

quant-ph↗

arXiv2020

Effects of the Nonlinearity in Activation Functions on the Performance of Deep Learning Models

The nonlinearity of activation functions used in deep learning models are crucial for the success of predictive models. There are several commonly used simple nonlinear functions, including Rectified Linear Unit (ReLU) and Leaky-ReLU (L-ReLU). In practice, these functions remarkably enhance the model accuracy. However, there is limited insight into the functionality of these nonlinear activation functions in terms of why certain models perform better than others. Here, we investigate the model performance when using ReLU or L-ReLU as activation functions in different model architectures and data domains. Interestingly, we found that the application of L-ReLU is mostly effective when the number of trainable parameters in a model is relatively small. Furthermore, we found that the image classification models seem to perform well with L-ReLU in fully connected layers, especially when pre-trained models such as the VGG-16 are used for the transfer learning.

cs.LG↗