arXiv subjects

Lei Feng

Publications and source records attributed to Lei Feng.

At least 163 records · Page 9Linked to original sources

arXiv2023

Mitigating Memorization of Noisy Labels by Clipping the Model Prediction

In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks. Cross Entropy (CE) loss has been shown to be not robust to noisy labels due to its unboundedness. To alleviate this issue, existing works typically design specialized robust losses with the symmetric condition, which usually lead to the underfitting issue. In this paper, our key idea is to induce a loss bound at the logit level, thus universally enhancing the noise robustness of existing losses. Specifically, we propose logit clipping (LogitClip), which clamps the norm of the logit vector to ensure that it is upper bounded by a constant. In this manner, CE loss equipped with our LogitClip method is effectively bounded, mitigating the overfitting to examples with noisy labels. Moreover, we present theoretical analyses to certify the noise-tolerant ability of LogitClip. Extensive experiments show that LogitClip not only significantly improves the noise robustness of CE loss, but also broadly enhances the generalization performance of popular robust losses.

cs.LG↗

arXiv2023

A Generalized Unbiased Risk Estimator for Learning with Augmented Classes

In contrast to the standard learning paradigm where all classes can be observed in training data, learning with augmented classes (LAC) tackles the problem where augmented classes unobserved in the training data may emerge in the test phase. Previous research showed that given unlabeled data, an unbiased risk estimator (URE) can be derived, which can be minimized for LAC with theoretical guarantees. However, this URE is only restricted to the specific type of one-versus-rest loss functions for multi-class classification, making it not flexible enough when the loss needs to be changed with the dataset in practice. In this paper, we propose a generalized URE that can be equipped with arbitrary loss functions while maintaining the theoretical guarantees, given unlabeled data for LAC. To alleviate the issue of negative empirical risk commonly encountered by previous studies, we further propose a novel risk-penalty regularization term. Experiments demonstrate the effectiveness of our proposed method.

cs.LG↗

arXiv2023

ALIM: Adjusting Label Importance Mechanism for Noisy Partial Label Learning

Noisy partial label learning (noisy PLL) is an important branch of weakly supervised learning. Unlike PLL where the ground-truth label must conceal in the candidate label set, noisy PLL relaxes this constraint and allows the ground-truth label may not be in the candidate label set. To address this challenging problem, most of the existing works attempt to detect noisy samples and estimate the ground-truth label for each noisy sample. However, detection errors are unavoidable. These errors can accumulate during training and continuously affect model optimization. To this end, we propose a novel framework for noisy PLL with theoretical guarantees, called ``Adjusting Label Importance Mechanism (ALIM)''. It aims to reduce the negative impact of detection errors by trading off the initial candidate set and model outputs. ALIM is a plug-in strategy that can be integrated with existing PLL approaches. Experimental results on benchmark datasets demonstrate that our method can achieve state-of-the-art performance on noisy PLL. \textcolor[rgb]{0.93,0.0,0.47}{Our code can be found in Supplementary Material}.

cs.CV↗

arXiv2023

Learning Cross-domain Semantic-Visual Relationships for Transductive Zero-Shot Learning

Zero-Shot Learning (ZSL) learns models for recognizing new classes. One of the main challenges in ZSL is the domain discrepancy caused by the category inconsistency between training and testing data. Domain adaptation is the most intuitive way to address this challenge. However, existing domain adaptation techniques cannot be directly applied into ZSL due to the disjoint label space between source and target domains. This work proposes the Transferrable Semantic-Visual Relation (TSVR) approach towards transductive ZSL. TSVR redefines image recognition as predicting the similarity/dissimilarity labels for semantic-visual fusions consisting of class attributes and visual features. After the above transformation, the source and target domains can have the same label space, which hence enables to quantify domain discrepancy. For the redefined problem, the number of similar semantic-visual pairs is significantly smaller than that of dissimilar ones. To this end, we further propose to use Domain-Specific Batch Normalization to align the domain discrepancy.

cs.CV↗

arXiv2023

COMET: Convolutional Dimension Interaction for Collaborative Filtering

Representation learning-based recommendation models play a dominant role among recommendation techniques. However, most of the existing methods assume both historical interactions and embedding dimensions are independent of each other, and thus regrettably ignore the high-order interaction information among historical interactions and embedding dimensions. In this paper, we propose a novel representation learning-based model called COMET (COnvolutional diMEnsion inTeraction), which simultaneously models the high-order interaction patterns among historical interactions and embedding dimensions. To be specific, COMET stacks the embeddings of historical interactions horizontally at first, which results in two "embedding maps". In this way, internal interactions and dimensional interactions can be exploited by convolutional neural networks (CNN) with kernels of different sizes simultaneously. A fully-connected multi-layer perceptron (MLP) is then applied to obtain two interaction vectors. Lastly, the representations of users and items are enriched by the learnt interaction vectors, which can further be used to produce the final prediction. Extensive experiments and ablation studies on various public implicit feedback datasets clearly demonstrate the effectiveness and rationality of our proposed method.

cs.IR↗

arXiv2023

Fine-Grained Classification with Noisy Labels

Learning with noisy labels (LNL) aims to ensure model generalization given a label-corrupted training set. In this work, we investigate a rarely studied scenario of LNL on fine-grained datasets (LNL-FG), which is more practical and challenging as large inter-class ambiguities among fine-grained classes cause more noisy labels. We empirically show that existing methods that work well for LNL fail to achieve satisfying performance for LNL-FG, arising the practical need of effective solutions for LNL-FG. To this end, we propose a novel framework called stochastic noise-tolerated supervised contrastive learning (SNSCL) that confronts label noise by encouraging distinguishable representation. Specifically, we design a noise-tolerated supervised contrastive learning loss that incorporates a weight-aware mechanism for noisy label correction and selectively updating momentum queue lists. By this mechanism, we mitigate the effects of noisy anchors and avoid inserting noisy labels into the momentum-updated queue. Besides, to avoid manually-defined augmentation strategies in contrastive learning, we propose an efficient stochastic module that samples feature embeddings from a generated distribution, which can also enhance the representation ability of deep models. SNSCL is general and compatible with prevailing robust LNL strategies to improve their performance for LNL-FG. Extensive experiments demonstrate the effectiveness of SNSCL.

cs.CV↗

arXiv2023

NMSSM neutralino dark matter for CDF II $W$-boson mass and muon $g-2$ and the promising prospect of direct detection

Two experiments from the Fermilab, E989 and CDF II, have reported two anomalies for muon $g-2$ and $W$-boson mass that may indicate the new physics at the low energy scale. Here we examine the possibility of a common origin of these two anomalies in the Next-to-Minimal Supersymmetric Standard Model. Considering various experimental and astrophysical constraints such as the Higgs mass, collider data, flavor physics, dark matter relic density, and direct detection experiments, we find that lighter electroweakinos and sleptons can generate sufficient contributions to muon $g-2$ and $m_W$. Moreover, the corresponding bino-like neutralino dark matter mass is in the $\sim 180-280$ GeV range. Interestingly, the favored DM mass region can soon be entirely probed by ongoing direct detection experiments like PandaX-4T, XENONnT, LUX-ZEPLIN, and DARWIN.

hep-ph↗

arXiv2023

In situ Biological Particle Analyzer based on Digital Inline Holography

Obtaining in situ measurements of biological microparticles is crucial for both scientific research and numerous industrial applications (e.g., early detection of harmful algal blooms, monitoring yeast during fermentation). However, existing methods are limited to offer timely diagnostics of these particles with sufficient accuracy and information. Here, we introduce a novel method for real-time, in situ analysis using machine learning assisted digital inline holography (DIH). Our machine learning model uses a customized YOLO v5 architecture specialized for the detection and classification of small biological particles. We demonstrate the effectiveness of our method in the analysis of 10 plankton species with equivalent high accuracy and significantly reduced processing time compared to previous methods. We also applied our method to differentiate yeast cells under four metabolic states and from two strains. Our results show that the proposed method can accurately detect and differentiate cellular and subcellular features related to metabolic states and strains. This study demonstrates the potential of machine learning driven DIH approach as a sensitive and versatile diagnostic tool for real-time, in situ analysis of both biotic and abiotic particles. This method can be readily deployed in a distributive manner for scientific research and manufacturing on an industrial scale.

physics.bio-ph↗

arXiv2022

Search for gamma-ray spectral lines with the DArk Matter Particle Explorer

The DArk Matter Particle Explorer (DAMPE) is well suitable for searching for monochromatic and sharp $γ$-ray structures in the GeV$-$TeV range thanks to its unprecedented high energy resolution. In this work, we search for $γ$-ray line structures using five years of DAMPE data. To improve the sensitivity, we develop two types of dedicated data sets (including the BgoOnly data which is the first time to be used in the data analysis for the calorimeter-based gamma-ray observatories) and adopt the signal-to-noise ratio optimized regions of interest (ROIs) for different DM density profiles. No line signals or candidates are found between 10 and 300 GeV in the Galaxy. The constraints on the velocity-averaged cross section for $χχ\to γγ$ and the decay lifetime for $χ\to γν$, both at 95% confidence level, have been calculated and the systematic uncertainties have been taken into account. Comparing to the previous Fermi-LAT results, though DAMPE has an acceptance smaller by a factor of $\sim 10$, similar constraints on the DM parameters are achieved and below 100 GeV the lower limits on the decay lifetime are even stronger by a factor of a few. Our results demonstrate the potential of high-energy-resolution observations on dark matter detection.

astro-ph.HE↗

arXiv2022

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL -- representation learning and label disambiguation -- in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disambiguation algorithm. PiCO produces closely aligned representations for examples from the same classes and facilitates label disambiguation. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. Moreover, we study a challenging yet practical noisy partial label learning setup, where the ground-truth may not be included in the candidate set. To remedy this problem, we present an extension PiCO+ that performs distance-based clean sample selection and learns robust classifiers by a semi-supervised contrastive learning algorithm. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning.

cs.LG↗

arXiv2022

On the Robustness of Average Losses for Partial-Label Learning

Partial-label learning (PLL) utilizes instances with PLs, where a PL includes several candidate labels but only one is the true label (TL). In PLL, identification-based strategy (IBS) purifies each PL on the fly to select the (most likely) TL for training; average-based strategy (ABS) treats all candidate labels equally for training and let trained models be able to predict TL. Although PLL research has focused on IBS for better performance, ABS is also worthy of study since modern IBS behaves like ABS in the beginning of training to prepare for PL purification and TL selection. In this paper, we analyze why ABS was unsatisfactory and propose how to improve it. Theoretically, we formalize five problem settings of PLL and prove that average PL losses (APLLs) with bounded multi-class losses are always robust, while APLLs with unbounded losses may be non-robust, which is the first robustness analysis for PLL. Experimentally, we have two promising findings: ABS using bounded losses can match/exceed state-of-the-art performance of IBS using unbounded losses; after using robust APLLs to warm start, IBS can further improve upon itself. Our work draws attention to ABS research, which can in turn boost IBS and push forward the whole PLL.

cs.LG↗

arXiv2022

Continuous Symmetry Breaking in a Trapped-Ion Spin Chain

One-dimensional systems exhibiting a continuous symmetry can host quantum phases of matter with true long-range order only in the presence of sufficiently long-range interactions. In most physical systems, however, the interactions are short-ranged, hindering the emergence of such phases in one dimension. Here we use a one-dimensional trapped-ion quantum simulator to prepare states with long-range spin order that extends over the system size of up to $23$ spins and is characteristic of the continuous symmetry-breaking phase of matter. Our preparation relies on simultaneous control over an array of tightly focused individual-addressing laser beams, generating long-range spin-spin interactions. We also observe a disordered phase with frustrated correlations. We further study the phases at different ranges of interaction and the out-of-equilibrium response to symmetry-breaking perturbations. This work opens an avenue to study new quantum phases and out-of-equilibrium dynamics in low-dimensional systems.

quant-ph↗

arXiv2022

The Possibility of Mirror Planet as Planet Nine in Solar System

A series of dynamical anomalies in the orbits of distant trans-Neptunian objects points to a new celestial body (usually named Planet Nine) in the solar system. In this draft, we point out that a mirror planet captured from the outer solar system or formed in the solar system is also a possible candidate. The introduction of the mirror matter model is due to an unbroken parity symmetry and is a potential explanation for dark matter. This mirror planet has null or fainter electromagnetic counterparts with a smaller optical radius and might be explored through gravitational effects.

astro-ph.EP↗

arXiv2022

Can Adversarial Training Be Manipulated By Non-Robust Features?

Adversarial training, originally designed to resist test-time adversarial examples, has shown to be promising in mitigating training-time availability attacks. This defense ability, however, is challenged in this paper. We identify a novel threat model named stability attack, which aims to hinder robust availability by slightly manipulating the training data. Under this threat, we show that adversarial training using a conventional defense budget $ε$ provably fails to provide test robustness in a simple statistical setting, where the non-robust features of the training data can be reinforced by $ε$-bounded perturbation. Further, we analyze the necessity of enlarging the defense budget to counter stability attacks. Finally, comprehensive experiments demonstrate that stability attacks are harmful on benchmark datasets, and thus the adaptive defense is necessary to maintain robustness. Our code is available at https://github.com/TLMichael/Hypocritical-Perturbation.

cs.LG↗

arXiv2022

Is the $W$-boson mass enhanced by the axion-like particle, dark photon, or chameleon dark energy?

The $W$-boson mass ($m_{W}=80.4335 \pm 0.0094 \mathrm{GeV}$) measured by the Collider Detector at Fermilab collaboration is greater than the standard model (SM) prediction at a confidence level of $7σ$, strongly suggesting the presence of new particles or fields. In the literature, various new particles and/or fields have been introduced to explain the astrophysical and experimental data, and their presence, in principle, may also enhance the $W$-boson mass. In this study, we investigate axion-like particle (ALP), dark photon (DP), and chameleon dark energy (DE) models for a solution to the $W$-boson mass excess. We find that the ALP and DP interpretations have been significantly narrowed down by global electroweak fits. The possibility of attributing the $W-$boson mass anomaly to the chameleon DE is ruled out by other experiments.

hep-ph↗

arXiv2022

Experimental Implementation of an Efficient Test of Quantumness

A test of quantumness is a protocol where a classical user issues challenges to a quantum device to determine if it exhibits non-classical behavior, under certain cryptographic assumptions. Recent attempts to implement such tests on current quantum computers rely on either interactive challenges with efficient verification, or non-interactive challenges with inefficient (exponential time) verification. In this paper, we execute an efficient non-interactive test of quantumness on an ion-trap quantum computer. Our results significantly exceed the bound for a classical device's success.

quant-ph↗

arXiv2022

SoLar: Sinkhorn Label Refinery for Imbalanced Partial-Label Learning

Partial-label learning (PLL) is a peculiar weakly-supervised learning task where the training samples are generally associated with a set of candidate labels instead of single ground truth. While a variety of label disambiguation methods have been proposed in this domain, they normally assume a class-balanced scenario that may not hold in many real-world applications. Empirically, we observe degenerated performance of the prior methods when facing the combinatorial challenge from the long-tailed distribution and partial-labeling. In this work, we first identify the major reasons that the prior work failed. We subsequently propose SoLar, a novel Optimal Transport-based framework that allows to refine the disambiguated labels towards matching the marginal class prior distribution. SoLar additionally incorporates a new and systematic mechanism for estimating the long-tailed class prior distribution under the PLL setup. Through extensive experiments, SoLar exhibits substantially superior results on standardized benchmarks compared to the previous state-of-the-art PLL methods. Code and data are available at: https://github.com/hbzju/SoLar .

cs.LG↗

arXiv2022

Demonstration of three- and four-body interactions between trapped-ion spins

Quantum processors use the native interactions between effective spins to simulate Hamiltonians or execute quantum gates. In most processors, the native interactions are pairwise, limiting the efficiency of controlling entanglement between many qubits. Here we experimentally demonstrate a new class of native interactions between trapped-ion qubits, extending conventional pairwise interactions to higher order. We realize three- and four-body spin interactions as examples, showing that high-order spin polynomials may serve as a new toolbox for quantum information applications.

quant-ph↗