arXiv subjects

Long Yan

Publications and source records attributed to Long Yan.

16 recordsLinked to original sources

arXiv2024

Data-Free Generalized Zero-Shot Learning

Deep learning models have the ability to extract rich knowledge from large-scale datasets. However, the sharing of data has become increasingly challenging due to concerns regarding data copyright and privacy. Consequently, this hampers the effective transfer of knowledge from existing data to novel downstream tasks and concepts. Zero-shot learning (ZSL) approaches aim to recognize new classes by transferring semantic knowledge learned from base classes. However, traditional generative ZSL methods often require access to real images from base classes and rely on manually annotated attributes, which presents challenges in terms of data restrictions and model scalability. To this end, this paper tackles a challenging and practical problem dubbed as data-free zero-shot learning (DFZSL), where only the CLIP-based base classes data pre-trained classifier is available for zero-shot classification. Specifically, we propose a generic framework for DFZSL, which consists of three main components. Firstly, to recover the virtual features of the base data, we model the CLIP features of base class images as samples from a von Mises-Fisher (vMF) distribution based on the pre-trained classifier. Secondly, we leverage the text features of CLIP as low-cost semantic information and propose a feature-language prompt tuning (FLPT) method to further align the virtual image features and textual features. Thirdly, we train a conditional generative model using the well-aligned virtual image features and corresponding semantic text features, enabling the generation of new classes features and achieve better zero-shot generalization. Our framework has been evaluated on five commonly used benchmarks for generalized ZSL, as well as 11 benchmarks for the base-to-new ZSL. The results demonstrate the superiority and effectiveness of our approach. Our code is available in https://github.com/ylong4/DFZSL

cs.CV↗

arXiv2023

Frame Pairwise Distance Loss for Weakly-supervised Sound Event Detection

Weakly-supervised learning has emerged as a promising approach to leverage limited labeled data in various domains by bridging the gap between fully supervised methods and unsupervised techniques. Acquisition of strong annotations for detecting sound events is prohibitively expensive, making weakly supervised learning a more cost-effective and broadly applicable alternative. In order to enhance the recognition rate of the learning of detection of weakly-supervised sound events, we introduce a Frame Pairwise Distance (FPD) loss branch, complemented with a minimal amount of synthesized data. The corresponding sampling and label processing strategies are also proposed. Two distinct distance metrics are employed to evaluate the proposed approach. Finally, the method is validated on the DCASE 2023 task4 dataset. The obtained experimental results corroborated the efficacy of this approach.

cs.HC↗

arXiv2023

Semi-supervised Sound Event Detection with Local and Global Consistency Regularization

Learning meaningful frame-wise features on a partially labeled dataset is crucial to semi-supervised sound event detection. Prior works either maintain consistency on frame-level predictions or seek feature-level similarity among neighboring frames, which cannot exploit the potential of unlabeled data. In this work, we design a Local and Global Consistency (LGC) regularization scheme to enhance the model on both label- and feature-level. The audio CutMix is introduced to change the contextual information of clips. Then, the local consistency is adopted to encourage the model to leverage local features for frame-level predictions, and the global consistency is applied to force features to align with global prototypes through a specially designed contrastive loss. Experiments on the DESED dataset indicate the superiority of LGC, surpassing its respective competitors largely with the same settings as the baseline system. Besides, combining LGC with existing methods can obtain further improvements. The code will be released soon.

eess.AS↗

arXiv2023

Audio Generation with Multiple Conditional Diffusion Model

Text-based audio generation models have limitations as they cannot encompass all the information in audio, leading to restricted controllability when relying solely on text. To address this issue, we propose a novel model that enhances the controllability of existing pre-trained text-to-audio models by incorporating additional conditions including content (timestamp) and style (pitch contour and energy contour) as supplements to the text. This approach achieves fine-grained control over the temporal order, pitch, and energy of generated audio. To preserve the diversity of generation, we employ a trainable control condition encoder that is enhanced by a large language model and a trainable Fusion-Net to encode and fuse the additional conditions while keeping the weights of the pre-trained text-to-audio model frozen. Due to the lack of suitable datasets and evaluation metrics, we consolidate existing datasets into a new dataset comprising the audio and corresponding conditions and use a series of evaluation metrics to evaluate the controllability performance. Experimental results demonstrate that our model successfully achieves fine-grained control to accomplish controllable audio generation. Audio samples and our dataset are publicly available at https://conditionaudiogen.github.io/conditionaudiogen/

cs.SD↗

arXiv2022

A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4

In this paper, we describe in detail our system for DCASE 2022 Task4. The system combines two considerably different models: an end-to-end Sound Event Detection Transformer (SEDT) and a frame-wise model, Metric Learning and Focal Loss CNN (MLFL-CNN). The former is an event-wise model which learns event-level representations and predicts sound event categories and boundaries directly, while the latter is based on the widely adopted frame-classification scheme, under which each frame is classified into event categories and event boundaries are obtained by post-processing such as thresholding and smoothing. For SEDT, self-supervised pre-training using unlabeled data is applied, and semi-supervised learning is adopted by using an online teacher, which is updated from the student model using the Exponential Moving Average (EMA) strategy and generates reliable pseudo labels for weakly-labeled and unlabeled data. For the frame-wise model, the ICT-TOSHIBA system of DCASE 2021 Task 4 is used. Experimental results show that the hybrid system considerably outperforms either individual model and achieves psds1 of 0.420 and psds2 of 0.783 on the validation set without external data. The code is available at https://github.com/965694547/Hybrid-system-of-frame-wise-model-and-SEDT.

cs.SD↗

arXiv2021

SP-SEDT: Self-supervised Pre-training for Sound Event Detection Transformer

Recently, an event-based end-to-end model (SEDT) has been proposed for sound event detection (SED) and achieves competitive performance. However, compared with the frame-based model, it requires more training data with temporal annotations to improve the localization ability. Synthetic data is an alternative, but it suffers from a great domain gap with real recordings. Inspired by the great success of UP-DETR in object detection, we propose to self-supervisedly pre-train SEDT (SP-SEDT) by detecting random patches (only cropped along the time axis). Experiments on the DCASE2019 task4 dataset show the proposed SP-SEDT can outperform fine-tuned frame-based model. The ablation study is also conducted to investigate the impact of different loss functions and patch size.

cs.SD↗

arXiv2021

Couple Learning for semi-supervised sound event detection

The recently proposed Mean Teacher method, which exploits large-scale unlabeled data in a self-ensembling manner, has achieved state-of-the-art results in several semi-supervised learning benchmarks. Spurred by current achievements, this paper proposes an effective Couple Learning method that combines a well-trained model and a Mean Teacher model. The suggested pseudo-labels generated model (PLG) increases strongly- and weakly-labeled data to improve the Mean Teacher method-s performance. Moreover, the Mean Teacher-s consistency cost reduces the noise impact in the pseudo-labels introduced by detection errors. The experimental results on Task 4 of the DCASE2020 challenge demonstrate the superiority of the proposed method, achieving about 44.25% F1-score on the public evaluation set, significantly outperforming the baseline system-s 32.39%. At the same time, we also propose a simple and effective experiment called the Variable Order Input (VOI) experiment, which proves the significance of the Couple Learning method. Our developed Couple Learning code is available on GitHub.

cs.LG↗

arXiv2021

Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level prediction, converting it into a sequence multi-label classification problem. A critical issue with the frame-based model is that it pursues the best frame-level prediction rather than the best event-level prediction. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the one-dimensional Detection Transformer (1D-DETR), inspired by Detection Transformer for image object detection. Furthermore, given the characteristics of SED, the audio query branch and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form Sound Event Detection Transformer (SEDT). To our knowledge, SEDT is the first event-based and end-to-end SED model. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both show that SEDT can achieve competitive performance.

cs.SD↗

arXiv2021

Core shell NaErF4 at NaYF4 upconversion nanoparticles qualify a NIR speckle wavemeter by a visible CCD

Speckle patterns have been widely confirmed that can be utilized to reconstruct the wavelength information. In order to achieve higher resolution, a varies of optical diffusing waveguides have been investigated with a focus on their wavelength sensitivity. However, it has been a challenge to reach the balance among cost, volumes, resolution, and stability. In this work, we designed a compact cylindrical random scattering waveguide (CRSW) as the light diffuser only by mixing TiO2 particles and ultra-violate adhesive. The speckle patterns are generated by the light multiple scattering in the CRSW. Importantly, the thin layer of upconversion nanoparticles (UCNPs) were sprayed on the end face of the CRSW. This allows the near infrared (NIR) light to be converted to the visible light, breaking the imaging limitation of visible cameras in the NIR range. We further designed a convolution neural network (CNN) to recognize the wavelength of the speckle patterns with good robustness and excellent ability of transfer learning, resulting in the achievement of a high resolution of 20 kHz ( 0.16 fm) at around 1550 nm with temperature resistance of 2 celsius. Our results provide a low-cost, compact, and simple NIR wavemeter in particular with the ultra high resolution and good temperature stability.

physics.optics↗

arXiv2019

Novel two-dimensional Ca-Cl crystals with metallicity, piezoelectric effect and room-temperature ferromagnetism

Recently we have reported the direct observation of two-dimensional (2D) Ca-Cl crystals on reduced graphene oxide (rGO) membranes, in which the calcium ions are only about monovalent (i.e. ~+1) and metallic rather than insulating properties are displayed by those CaCl crystals. Here, we report the experimental observation and demonstration of the formation of graphene-Ca-Cl heterojunction owing to the metallicity of 2D Ca-Cl crystals, unexpected piezoelectric effect, room-temperature ferromagnetism, as well as the distinct hydrogen storage and release capability of the Ca-Cl crystals in rGO membranes. Theoretical studies show that the formation of those abnormal crystals is attributed to the strong cation-pi interactions of the Ca2+ with the aromatic rings in the graphitic surfaces. Since strong cation-pi interactions also exist between other metal ions (such as Mg2+, Fe2+, Co2+, Cu2+, Cd2+, Cr2+ and Pb2+) and graphitic surfaces, similar 2D crystals with abnormal valence state of the metal cations and corresponding abnormal properties as well as novel applications are highly expected. Those findings further show the realistically potential applications of such abnormal CaCl material with unusual electronic properties in designing novel transistors and magnetic devices, hydrogen storage, catalyzer, high-performance conducting electrodes and sensors, with a size down to atomic scale.

physics.chem-ph↗

arXiv2018

Two-dimensional Ca-Cl crystals under ambient conditions observed directly by cryo-electron microscopy

Recently, we report the direct observation, under ambient conditions, of Na2Cl and Na3Cl as two-dimensional (2D) Na-Cl crystals, together with regular NaCl, on reduced graphene oxide membranes and on the surfaces of natural graphite powders from salt solutions far below the saturated concentration. However, what are these abnormal stoichiometries for high valence ions, such as calcium ions and copper ions still remain unknown. Here, using cryo-electron microscopy, we report the direct observation of two-dimensional (2D) Ca-Cl crystals on reduced graphene oxide (rGO) membranes, in which the calcium ions are only monovalent (i.e. +1). Remarkably, metallic properties rather than insulating are displayed by those CaCl crystals. We note that such CaCl crystals are obtained by simply incubating rGO membranes in salt solutions below the saturated concentration, under ambient conditions. Theoretical studies show that the formation of those abnormal crystals is attributed to the strong cation-pi interactions of the Ca2+ ions with the aromatic rings in the graphitic surfaces. Since strong cation-pi interactions also exist between other metal ions (such as Mg2+, Fe2+, Co2+, Cu2+, Cd2+, Cr2+ and Pb2+) and graphitic surfaces, similar 2D crystals with abnormal valence state of the metal cations and corresponding abnormal properties are highly expected. The 2D crystals with monovalent calcium ions show unusual electronic properties, and can be applicated in catalyzer, hydrogen storage, high-performance conducting electrodes and sensors. These findings also produce functionalized graphene including compact "graphene-metallic CaCl-insulating CaCl2" junction that can serve as transistors down to the atomic scale, and other devices for magnetic, optical and mechanical applications.

physics.chem-ph↗

arXiv2018

Unexpectedly high salt accumulation inside carbon nanotubes soaked in very dilute salt solutions

We experimentally demonstrate the formation of salt aggregations with unexpectedly high concentration inside multi-walled carbon nanotubes (CNTs) soaked only in dilute salt solutions and even in solutions containing only traces of salts. This finding suggests the blocking of fluid across CNTs by the salt aggregations when CNTs are soaked in a dilute salt solution with the concentration of seawater or even lower, which may open new avenues for the development of novel CNT-based desalination techniques. The high salt accumulation of CNTs also provides a new CNT-based strategy for the collection/extraction of noble metal salts in solutions containing traces of noble metal salts. Theoretical analyses reveal that this high salt accumulation inside CNTs can be mainly attributed to the strong hydrated cation-pi interactions of hydrated cations and pi electrons in the aromatic rings of CNTs.

cond-mat.mtrl-sci↗

arXiv2018

Observation of Meisser effect in Kapton Tapes

Composed of organic compounds, Kapton tape is regularly used as a non-conducting and non-magnetic material in cryogenic experiments. Here we report the discovery of Meisser effect at transition temperature (Tc) of ~ 3.8 K and ~ 8.0 K in Kapton tapes. New organic superconducting materials may be further explored in the tapes.

cond-mat.supr-con↗

arXiv2017

NaCl crystal from salt solution with far below saturated concentration under ambient condition

Under ambient conditions, we directly observed NaCl crystals experimentally in the rGO membranes soaked in the salt solution with concentration below and far below the saturated concentration. Moreover, in most probability, the NaCl crystals show stoichiometries behavior. We attribute this unexpected crystallization to the cation-{\pi} interactions between the ions and the aromatic rings of the rGO.

physics.chem-ph↗

arXiv2016

Saturation of ion irradiation effects in Cr2AlC

Cr2AlC materials were irradiated with 7 MeV Xe26+ ions and 500 keV He2+ ions at room temperature. A structural transition with an increased c lattice parameter and a decreased a lattice parameter occurs after irradiation to doses above 1 dpa. Nevertheless, the modified structure is stable up to the dose of 5.2 dpa without obvious lattice disorder. The three samples irradiated to doses above 1 dpa have comparable lattice parameters and hardness values, suggesting a saturation of irradiation effects in Cr2AlC. The structural transition and irradiation effects saturation are ascribed to irradiation-induced antisite defects (CrAl and AlCr) and C interstitials, which is supported by the calculations of the formation energies of various defects in Cr2AlC. The irradiation-induced antisite defects and C interstitials may be critical to understand the excellent resistance to irradiation-induced amorphization of MAX phases.

cond-mat.mtrl-sci↗

arXiv2014

Irradiation effects in the Ni-17Mo-7Cr alloy bombarded with MeV Au ions

Irradiation effects in Ni-17Mo-7Cr alloy, which is an newly developed structural material for molten salt reactor (MSR), have been systematically investigated by using 3MeV Au ions at different fluences, corresponding to dpa number (displacement per atom) of 1~ 30. GIXRD measurement indicates that the microstrain of the irradiated samples increased from 0.14% to 0.22% as dpa increased from 1 to 30. In the meanwhile, nanoindentation results reveal the Ni-17Mo-7Cr alloy underwent radiation-induced hardening first and then softening at dpa of 30. The swelling rate of Ni-17Mo-7Cr alloy was found around 1.3% at 30 dpa, which means only 0.04% per dpa. Besides, Raman spectra shows that carbon segregation appeared after Au ions irradiation. Our results are very helpful for understanding irradiation damages in Nickel-base alloys, especially for those in purpose of being used in future MSR nuclear energy system.

cond-mat.mtrl-sci↗