arXiv subjects

Fei Wang

Publications and source records attributed to Fei Wang.

At least 577 records · Page 32Linked to original sources

arXiv2021

ReSSL: Relational Self-Supervised Learning with Weak Augmentation

Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most of methods mainly focus on the instance level information (\ie, the different augmented images of the same instance should have the same feature or cluster into the same class), but there is a lack of attention on the relationships between different instances. In this paper, we introduced a novel SSL paradigm, which we term as relational self-supervised learning (ReSSL) framework that learns representations by modeling the relationship between different instances. Specifically, our proposed method employs sharpened distribution of pairwise similarities among different instances as \textit{relation} metric, which is thus utilized to match the feature embeddings of different augmentations. Moreover, to boost the performance, we argue that weak augmentations matter to represent a more reliable relation, and leverage momentum strategy for practical efficiency. Experimental results show that our proposed ReSSL significantly outperforms the previous state-of-the-art algorithms in terms of both performance and training efficiency. Code is available at \url{https://github.com/KyleZheng1997/ReSSL}.

cs.CV↗

arXiv2021

GUT-scale constrained SUSY in light of E989 muon g-2 measurement

The new FNAL result of the muon $g-2$, combined with the BNL result, shows a 4.2$σ$ deviation from the SM. We use the new data of the muon $g-2$ to revisit several GUT-scale constrained SUSY models with the constraints from the LHC searches, the dark matter detection, the flavor data and the electroweak vacuum stability. We first demonstrate the tension between the muon $g-2$ and other experimental measurements in the CMSSM/mSUGRA. Then after discussing the possible ways to alleviate such a tension and showing the muon $g-2$ in pMSSM under relevant experimental constraints, we survey several extensions of the CMSSM/mSUGRA with different types of universal boundary conditions at the GUT scale. Finally, we briefly discuss the muon $g-2$ in other popular SUSY breaking mechanisms, namely the GMSB and AMSB mechanisms and their extensions.

hep-ph↗

arXiv2021

On the Euler+Prandtl expansion for the Navier-Stokes equations

We establish the validity of the Euler$+$Prandtl approximation for solutions of the Navier-Stokes equations in the half plane with Dirichlet boundary conditions, in the vanishing viscosity limit, for initial data which are analytic only near the boundary, and Sobolev smooth away from the boundary. Our proof does not require higher order correctors, and works directly by estimating an $L^{1}$-type norm for the vorticity of the error term in the expansion Navier-Stokes$-($Euler$+$Prandtl$)$. An important ingredient in the proof is the propagation of local analyticity for the Euler equation, a result of independent interest.

math.AP↗

arXiv2021

MCMC Confidence Intervals and Biases

The recent paper "Simple confidence intervals for MCMC without CLTs" by J.S. Rosenthal, showed the derivation of a simple MCMC confidence interval using only Chebyshev's inequality, not CLT. That result required certain assumptions about how the estimator bias and variance grow with the number of iterations $n$. In particular, the bias is $o(1/\sqrt{n})$. This assumption seemed mild. It is generally believed that the estimator bias will be $O(1/n)$ and hence $o(1/\sqrt{n})$. However, questions were raised by researchers about how to verify this assumption. Indeed, we show that this assumption might not always hold. In this paper, we seek to simplify and weaken the assumptions in the previously mentioned paper, to make MCMC confidence intervals without CLTs more widely applicable.

math.ST↗

arXiv2021

MFR 2021: Masked Face Recognition Competition

This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 valid solutions. The competition is designed to motivate solutions aiming at enhancing the face recognition accuracy of masked faces. Moreover, the competition considered the deployability of the proposed solutions by taking the compactness of the face recognition models into account. A private dataset representing a collaborative, multi-session, real masked, capture scenario is used to evaluate the submitted solutions. In comparison to one of the top-performing academic face recognition solutions, 10 out of the 18 submitted solutions did score higher masked face verification accuracy.

cs.CV↗

arXiv2021

Even-Odd Layer-Dependent Anomalous Hall Effect in Topological Magnet MnBi2Te4 Thin Films

A central theme in condensed matter physics is to create and understand the exotic states of matter by incorporating magnetism into topological materials. One prime example is the quantum anomalous Hall (QAH) state. Recently, MnBi2Te4 has been demonstrated to be an intrinsic magnetic topological insulator and the QAH effect was observed in exfoliated MnBi2Te4 flakes. Here, we used molecular beam epitaxy (MBE) to grow MnBi2Te4 films with thickness down to 1 septuple layer (SL) and performed thickness-dependent transport measurements. We observed a non-square hysteresis loop in the antiferromagnetic state for films with thickness greater than 2 SL. The hysteresis loop can be separated into two AH components. Through careful analysis, we demonstrated that one AH component with the larger coercive field is from the dominant MnBi2Te4 phase, while the other AH component with the smaller coercive field is from the minor Mn-doped Bi2Te3 phase in the samples. The extracted AH component of the MnBi2Te4 phase shows a clear even-odd layer-dependent behavior, a signature of antiferromagnetic thin films. Our studies reveal insights on how to optimize the MBE growth conditions to improve the quality of MnBi2Te4 films, in which the QAH and other exotic states are predicted.

cond-mat.mtrl-sci↗

arXiv2021

S-LIME: Stabilized-LIME for Model Explanation

An increasing number of machine learning models have been deployed in domains with high stakes such as finance and healthcare. Despite their superior performances, many models are black boxes in nature which are hard to explain. There are growing efforts for researchers to develop methods to interpret these black-box models. Post hoc explanations based on perturbations, such as LIME, are widely used approaches to interpret a machine learning model after it has been built. This class of methods has been shown to exhibit large instability, posing serious challenges to the effectiveness of the method itself and harming user trust. In this paper, we propose S-LIME, which utilizes a hypothesis testing framework based on central limit theorem for determining the number of perturbation points needed to guarantee stability of the resulting explanation. Experiments on both simulated and real world data sets are provided to demonstrate the effectiveness of our method.

stat.ML↗

arXiv2021

K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets

In one-shot weight sharing for NAS, the weights of each operation (at each layer) are supposed to be identical for all architectures (paths) in the supernet. However, this rules out the possibility of adjusting operation weights to cater for different paths, which limits the reliability of the evaluation results. In this paper, instead of counting on a single supernet, we introduce $K$-shot supernets and take their weights for each operation as a dictionary. The operation weight for each path is represented as a convex combination of items in a dictionary with a simplex code. This enables a matrix approximation of the stand-alone weight matrix with a higher rank ($K>1$). A \textit{simplex-net} is introduced to produce architecture-customized code for each path. As a result, all paths can adaptively learn how to share weights in the $K$-shot supernets and acquire corresponding weights for better evaluation. $K$-shot supernets and simplex-net can be iteratively trained, and we further extend the search to the channel dimension. Extensive experiments on benchmark datasets validate that K-shot NAS significantly improves the evaluation accuracy of paths and thus brings in impressive performance improvements.

cs.CV↗

arXiv2021

Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility

Bipartite ranking, which aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data, is widely adopted in various applications where sample prioritization is needed. Recently, there have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups defined by sensitive attributes. While there could be trade-off between fairness and performance, in this paper we propose a model agnostic post-processing framework for balancing them in the bipartite ranking scenario. Specifically, we maximize a weighted sum of the utility and fairness by directly adjusting the relative ordering of samples across groups. By formulating this problem as the identification of an optimal warping path across different protected groups, we propose a non-parametric method to search for such an optimal path through a dynamic programming process. Our method is compatible with various classification models and applicable to a variety of ranking fairness metrics. Comprehensive experiments on a suite of benchmark data sets and two real-world patient electronic health record repositories show that our method can achieve a great balance between the algorithm utility and ranking fairness. Furthermore, we experimentally verify the robustness of our method when faced with the fewer training samples and the difference between training and testing ranking score distributions.

cs.LG↗

arXiv2021

Why Attentions May Not Be Interpretable?

Attention-based methods have played important roles in model interpretations, where the calculated attention weights are expected to highlight the critical parts of inputs~(e.g., keywords in sentences). However, recent research found that attention-as-importance interpretations often do not work as we expected. For example, learned attention weights sometimes highlight less meaningful tokens like "[SEP]", ",", and ".", and are frequently uncorrelated with other feature importance indicators like gradient-based measures. A recent debate over whether attention is an explanation or not has drawn considerable interest. In this paper, we demonstrate that one root cause of this phenomenon is the combinatorial shortcuts, which means that, in addition to the highlighted parts, the attention weights themselves may carry extra information that could be utilized by downstream models after attention layers. As a result, the attention weights are no longer pure importance indicators. We theoretically analyze combinatorial shortcuts, design one intuitive experiment to show their existence, and propose two methods to mitigate this issue. We conduct empirical studies on attention-based interpretation models. The results show that the proposed methods can effectively improve the interpretability of attention mechanisms.

stat.ML↗

arXiv2021

Impact of Temperature and Relative Humidity on the Transmission of COVID-19: A Modeling Study in China and the United States

Objectives: We aim to assess the impact of temperature and relative humidity on the transmission of COVID-19 across communities after accounting for community-level factors such as demographics, socioeconomic status, and human mobility status. Design: A retrospective cross-sectional regression analysis via the Fama-MacBeth procedure is adopted. Setting: We use the data for COVID-19 daily symptom-onset cases for 100 Chinese cities and COVID-19 daily confirmed cases for 1,005 U.S. counties. Participants: A total of 69,498 cases in China and 740,843 cases in the U.S. are used for calculating the effective reproductive numbers. Primary outcome measures: Regression analysis of the impact of temperature and relative humidity on the effective reproductive number (R value). Results: Statistically significant negative correlations are found between temperature/relative humidity and the effective reproductive number (R value) in both China and the U.S. Conclusions: Higher temperature and higher relative humidity potentially suppress the transmission of COVID-19. Specifically, an increase in temperature by 1 degree Celsius is associated with a reduction in the R value of COVID-19 by 0.026 (95% CI [-0.0395,-0.0125]) in China and by 0.020 (95% CI [-0.0311, -0.0096]) in the U.S.; an increase in relative humidity by 1% is associated with a reduction in the R value by 0.0076 (95% CI [-0.0108,-0.0045]) in China and by 0.0080 (95% CI [-0.0150,-0.0010]) in the U.S. Therefore, the potential impact of temperature/relative humidity on the effective reproductive number alone is not strong enough to stop the pandemic.

q-bio.PE↗

arXiv2021

BCNet: Searching for Network Width with Bilaterally Coupled Network

Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constraints. To fulfill the searching, a one-shot supernet is usually leveraged to efficiently evaluate the performance \wrt~different network widths. However, current methods mainly follow a \textit{unilaterally augmented} (UA) principle for the evaluation of each width, which induces the training unfairness of channels in supernet. In this paper, we introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Besides, we leverage a stochastic complementary strategy for training the BCNet, and propose a prior initial population sampling method to boost the performance of the evolutionary search. Extensive experiments on benchmark CIFAR-10 and ImageNet datasets indicate that our method can achieve state-of-the-art or competing performance over other baseline methods. Moreover, our method turns out to further boost the performance of NAS models by refining their network widths. For example, with the same FLOPs budget, our obtained EfficientNet-B0 achieves 77.36\% Top-1 accuracy on ImageNet dataset, surpassing the performance of original setting by 0.48\%.

cs.CV↗

arXiv2021

CODER: Knowledge infused cross-lingual medical term embedding for term normalization

This paper proposes CODER: contrastive learning on knowledge graphs for cross-lingual medical term representation. CODER is designed for medical term normalization by providing close vector representations for different terms that represent the same or similar medical concepts with cross-lingual support. We train CODER via contrastive learning on a medical knowledge graph (KG) named the Unified Medical Language System, where similarities are calculated utilizing both terms and relation triplets from KG. Training with relations injects medical knowledge into embeddings and aims to provide potentially better machine learning features. We evaluate CODER in zero-shot term normalization, semantic similarity, and relation classification benchmarks, which show that CODERoutperforms various state-of-the-art biomedical word embedding, concept embeddings, and contextual embeddings. Our codes and models are available at https://github.com/GanjinZero/CODER.

cs.CL↗

arXiv2021

Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentation

Language-queried video actor segmentation aims to predict the pixel-level mask of the actor which performs the actions described by a natural language query in the target frames. Existing methods adopt 3D CNNs over the video clip as a general encoder to extract a mixed spatio-temporal feature for the target frame. Though 3D convolutions are amenable to recognizing which actor is performing the queried actions, it also inevitably introduces misaligned spatial information from adjacent frames, which confuses features of the target frame and yields inaccurate segmentation. Therefore, we propose a collaborative spatial-temporal encoder-decoder framework which contains a 3D temporal encoder over the video clip to recognize the queried actions, and a 2D spatial encoder over the target frame to accurately segment the queried actors. In the decoder, a Language-Guided Feature Selection (LGFS) module is proposed to flexibly integrate spatial and temporal features from the two encoders. We also propose a Cross-Modal Adaptive Modulation (CMAM) module to dynamically recombine spatial- and temporal-relevant linguistic features for multimodal feature interaction in each stage of the two encoders. Our method achieves new state-of-the-art performance on two popular benchmarks with less computational overhead than previous approaches.

cs.CV↗

arXiv2021

Reformulating HOI Detection as Adaptive Set Prediction

Determining which image regions to concentrate on is critical for Human-Object Interaction (HOI) detection. Conventional HOI detectors focus on either detected human and object pairs or pre-defined interaction locations, which limits learning of the effective features. In this paper, we reformulate HOI detection as an adaptive set prediction problem, with this novel formulation, we propose an Adaptive Set-based one-stage framework (AS-Net) with parallel instances and interaction branches. To attain this, we map a trainable interaction query set to an interaction prediction set with a transformer. Each query adaptively aggregates the interaction-relevant features from global contexts through multi-head co-attention. Besides, the training process is supervised adaptively by matching each ground truth with the interaction prediction. Furthermore, we design an effective instance-aware attention module to introduce instructive features from the instance branch into the interaction branch. Our method outperforms previous state-of-the-art methods without any extra human pose and language features on three challenging HOI detection datasets. Especially, we achieve over $31\%$ relative improvement on a large-scale HICO-DET dataset. Code is available at https://github.com/yoyomimi/AS-Net.

cs.CV↗

arXiv2021

Retrieving Complex Tables with Multi-Granular Graph Representation Learning

The task of natural language table retrieval (NLTR) seeks to retrieve semantically relevant tables based on natural language queries. Existing learning systems for this task often treat tables as plain text based on the assumption that tables are structured as dataframes. However, tables can have complex layouts which indicate diverse dependencies between subtable structures, such as nested headers. As a result, queries may refer to different spans of relevant content that is distributed across these structures. Moreover, such systems fail to generalize to novel scenarios beyond those seen in the training set. Prior methods are still distant from a generalizable solution to the NLTR problem, as they fall short in handling complex table layouts or queries over multiple granularities. To address these issues, we propose Graph-based Table Retrieval (GTR), a generalizable NLTR framework with multi-granular graph representation learning. In our framework, a table is first converted into a tabular graph, with cell nodes, row nodes and column nodes to capture content at different granularities. Then the tabular graph is input to a Graph Transformer model that can capture both table cell content and the layout structures. To enhance the robustness and generalizability of the model, we further incorporate a self-supervised pre-training task based on graph-context matching. Experimental results on two benchmarks show that our method leads to significant improvements over the current state-of-the-art systems. Further experiments demonstrate promising performance of our method on cross-dataset generalization, and enhanced capability of handling complex tables and fulfilling diverse query intents. Code and data are available at https://github.com/FeiWang96/GTR.

cs.IR↗

arXiv2021

Research on Optimization Method of Multi-scale Fish Target Fast Detection Network

The fish target detection algorithm lacks a good quality data set, and the algorithm achieves real-time detection with lower power consumption on embedded devices, and it is difficult to balance the calculation speed and identification ability. To this end, this paper collected and annotated a data set named "Aquarium Fish" of 84 fishes containing 10042 images, and based on this data set, proposed a multi-scale input fast fish target detection network (BTP-yoloV3) and its optimization method. The experiment uses Depthwise convolution to redesign the backbone of the yoloV4 network, which reduces the amount of calculation by 94.1%, and the test accuracy is 92.34%. Then, the training model is enhanced with MixUp, CutMix, and mosaic to increase the test accuracy by 1.27%; Finally, use the mish, swish, and ELU activation functions to increase the test accuracy by 0.76%. As a result, the accuracy of testing the network with 2000 fish images reached 94.37%, and the computational complexity of the network BFLOPS was only 5.47. Comparing the YoloV3~4, MobileNetV2-yoloV3, and YoloV3-tiny networks of migration learning on this data set. The results show that BTP-Yolov3 has smaller model parameters, faster calculation speed, and lower energy consumption during operation while ensuring the calculation accuracy. It provides a certain reference value for the practical application of neural network.

cs.CV↗

arXiv2021

Switchable Rashba Anisotropy in Two-dimensional Hybrid Organic-Inorganic Perovskite by Hybrid Improper Ferroelectricity

Two-dimensional (2D) hybrid organic-inorganic perovskites (HOIPs) are introducing new directions in the 2D materials landscape. The coexistence of ferroelectricity and spin-orbit interactions play a key role in their optoelectronic properties. We perform a detailed study on a recently synthesized ferroelectric 2D-HOIP, (AMP)PbI$_4$ (AMP = 4-aminomethyl-piperidinium). The calculated polarization and Rashba parameter are in excellent agreement with experimental values. We report a striking new effect, i.e., an extraordinarily large Rashba anisotropy that is tunable by ferroelectric polarization: as polarization is reversed, not only the spin texture chirality is inverted, but also the major and minor axes of the Rashba anisotropy ellipse in k-space are interchanged - a pseudo rotation. A $k \cdot p$ model Hamiltonian and symmetry-mode analysis reveal a quadrilinear coupling between the cation-rotation modes responsible for the Rashba ellipse pseudo-rotation, the framework rotation, and the polarization. These findings may provide new avenues for spin-optoelectronic devices such as spin valves or spin FETs.

cond-mat.mtrl-sci↗