arXiv Science⌕ Search

arXiv subjects

Jie Yang

Publications and source records attributed to Jie Yang.

At least 343 records · Page 19Linked to original sources
arXiv2023

MobileUtr: Revisiting the relationship between light-weight CNN and Transformer for efficient medical image segmentation

Due to the scarcity and specific imaging characteristics in medical images, light-weighting Vision Transformers (ViTs) for efficient medical image segmentation is a significant challenge, and current studies have not yet paid attention to this issue. This work revisits the relationship between CNNs and Transformers in lightweight universal networks for medical image segmentation, aiming to integrate the advantages of both worlds at the infrastructure design level. In order to leverage the inductive bias inherent in CNNs, we abstract a Transformer-like lightweight CNNs block (ConvUtr) as the patch embeddings of ViTs, feeding Transformer with denoised, non-redundant and highly condensed semantic information. Moreover, an adaptive Local-Global-Local (LGL) block is introduced to facilitate efficient local-to-global information flow exchange, maximizing Transformer's global context information extraction capabilities. Finally, we build an efficient medical image segmentation model (MobileUtr) based on CNN and Transformer. Extensive experiments on five public medical image datasets with three different modalities demonstrate the superiority of MobileUtr over the state-of-the-art methods, while boasting lighter weights and lower computational cost. Code is available at https://github.com/FengheTan9/MobileUtr.

eess.IV↗
arXiv2023

TIDE: Test Time Few Shot Object Detection

Few-shot object detection (FSOD) aims to extract semantic knowledge from limited object instances of novel categories within a target domain. Recent advances in FSOD focus on fine-tuning the base model based on a few objects via meta-learning or data augmentation. Despite their success, the majority of them are grounded with parametric readjustment to generalize on novel objects, which face considerable challenges in Industry 5.0, such as (i) a certain amount of fine-tuning time is required, and (ii) the parameters of the constructed model being unavailable due to the privilege protection, making the fine-tuning fail. Such constraints naturally limit its application in scenarios with real-time configuration requirements or within black-box settings. To tackle the challenges mentioned above, we formalize a novel FSOD task, referred to as Test TIme Few Shot DEtection (TIDE), where the model is un-tuned in the configuration procedure. To that end, we introduce an asymmetric architecture for learning a support-instance-guided dynamic category classifier. Further, a cross-attention module and a multi-scale resizer are provided to enhance the model performance. Experimental results on multiple few-shot object detection platforms reveal that the proposed TIDE significantly outperforms existing contemporary methods. The implementation codes are available at https://github.com/deku-0621/TIDE

cs.CV↗
arXiv2023

An Ultra-fast Quantum Random Number Generation Scheme Based on Laser Phase Noise

Based on the intrinsic random property of quantum mechanics, quantum random number generators allow for access of truly unpredictable random sequence and are now heading towards high performance and small miniaturization, among which a popular scheme is based on the laser phase noise. However, this scheme is generally limited in speed and implementation complexity, especially for chip integration. In this work, a general physical model based on wiener process for such schemes is introduced, which provides an approach to clearly explain the limitation on the generation rate and comprehensively optimize the system performance. We present an insight to exploit the potential bandwidth of the quantum entropy source that contains plentiful quantum randomness with a simple spectral filtering method and experimentally boost the bandwidth of the corresponding quantum entropy source to 20 GHz, based on which an ultra-fast generation rate of 218 Gbps is demonstrated, setting a new record for laser phase noise based schemes by one order of magnitude. Our proposal significantly enhances the ceiling speed of such schemes without requiring extra complex hardware, thus effectively benefits the corresponding chip integration with high performance and low implementation cost, which paves the way for its large-scale applications.

quant-ph↗
arXiv2023

Monitoring the evolution of relative product populations at early times during a photochemical reaction

Identifying multiple rival reaction products and transient species formed during ultrafast photochemical reactions and determining their time-evolving relative populations are key steps towards understanding and predicting photochemical outcomes. Yet, most contemporary ultrafast studies struggle with clearly identifying and quantifying competing molecular structures/species amongst the emerging reaction products. Here, we show that mega-electronvolt ultrafast electron diffraction in combination with ab initio molecular dynamics calculations offer a powerful route to determining time-resolved populations of the various isomeric products formed after UV (266 nm) excitation of the five-membered heterocyclic molecule 2(5H)-thiophenone. This strategy provides experimental validation of the predicted high (~50%) yield of an episulfide isomer containing a strained 3-membered ring within ~1 ps of photoexcitation and highlights the rapidity of interconversion between the rival highly vibrationally excited photoproducts in their ground electronic state.

physics.chem-ph↗
arXiv2023

YATO: Yet Another deep learning based Text analysis Open toolkit

We introduce YATO, an open-source, easy-to-use toolkit for text analysis with deep learning. Different from existing heavily engineered toolkits and platforms, YATO is lightweight and user-friendly for researchers from cross-disciplinary areas. Designed in a hierarchical structure, YATO supports free combinations of three types of widely used features including 1) traditional neural networks (CNN, RNN, etc.); 2) pre-trained language models (BERT, RoBERTa, ELECTRA, etc.); and 3) user-customized neural features via a simple configurable file. Benefiting from the advantages of flexibility and ease of use, YATO can facilitate fast reproduction and refinement of state-of-the-art NLP models, and promote the cross-disciplinary applications of NLP techniques. The code, examples, and documentation are publicly available at https://github.com/jiesutd/YATO. A demo video is also available at https://www.youtube.com/playlist?list=PLJ0mhzMcRuDUlTkzBfAftOqiJRxYTTjXH.

cs.CL↗
arXiv2023

ClusVPR: Efficient Visual Place Recognition with Clustering-based Weighted Transformer

Visual place recognition (VPR) is a highly challenging task that has a wide range of applications, including robot navigation and self-driving vehicles. VPR is particularly difficult due to the presence of duplicate regions and the lack of attention to small objects in complex scenes, resulting in recognition deviations. In this paper, we present ClusVPR, a novel approach that tackles the specific issues of redundant information in duplicate regions and representations of small objects. Different from existing methods that rely on Convolutional Neural Networks (CNNs) for feature map generation, ClusVPR introduces a unique paradigm called Clustering-based Weighted Transformer Network (CWTNet). CWTNet leverages the power of clustering-based weighted feature maps and integrates global dependencies to effectively address visual deviations encountered in large-scale VPR problems. We also introduce the optimized-VLAD (OptLAD) layer that significantly reduces the number of parameters and enhances model efficiency. This layer is specifically designed to aggregate the information obtained from scale-wise image patches. Additionally, our pyramid self-supervised strategy focuses on extracting representative and diverse information from scale-wise image patches instead of entire images, which is crucial for capturing representative and diverse information in VPR. Extensive experiments on four VPR datasets show our model's superior performance compared to existing models while being less complex.

cs.CV↗
arXiv2023

Eliminating Contextual Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion

The recent success of text-to-image generation diffusion models has also revolutionized semantic image editing, enabling the manipulation of images based on query/target texts. Despite these advancements, a significant challenge lies in the potential introduction of contextual prior bias in pre-trained models during image editing, e.g., making unexpected modifications to inappropriate regions. To address this issue, we present a novel approach called Dual-Cycle Diffusion, which generates an unbiased mask to guide image editing. The proposed model incorporates a Bias Elimination Cycle that consists of both a forward path and an inverted path, each featuring a Structural Consistency Cycle to ensure the preservation of image content during the editing process. The forward path utilizes the pre-trained model to produce the edited image, while the inverted path converts the result back to the source image. The unbiased mask is generated by comparing differences between the processed source image and the edited image to ensure that both conform to the same distribution. Our experiments demonstrate the effectiveness of the proposed method, as it significantly improves the D-CLIP score from 0.272 to 0.283. The code will be available at https://github.com/JohnDreamer/DualCycleDiffsion.

cs.CV↗
arXiv2023

Van der Waals Spin-Orbit Torque Antiferromagnetic Memory

The technique of conventional ferromagnet/heavy-metal spin-orbit torque (SOT) offers significant potential for enhancing the efficiency of magnetic memories. However, it faces fundamental physical limitations, including hunting effects from the metallic layer, broken symmetry for enabling antidamping switching, spin scattering caused by interfacial defects, and sensitivity to stray magnetic fields. To address these issues, we here propose a van der Waals (vdW) field-free SOT antiferromagnetic memory using a vdW bilayer LaBr$_2$ (an antiferromagnet with perpendicular magnetic anisotropy) and a monolayer T$_d$ phase WTe$_2$ (a Weyl semimetal with broken inversion symmetry). By systematically employing density functional theory in conjunction with non-equilibrium Green's function methods and macrospin simulations, we demonstrate that the proposed vdW SOT devices exhibit remarkably low critical current density approximately 10 MA/cm$^2$ and rapid field-free magnetization switching in 250 ps. This facilitates excellent write performance with extremely low energy consumption. Furthermore, the device shows a significantly low read error rate, as evidenced by a high tunnel magnetoresistance ratio of up to 4250%. The superior write and read performance originates from the unique strong on-site (insulating phase) and off-site (magnetic phase) Coulomb interactions in electride LaBr$_2$, a large non-zero z-component polarization in WTe$_2$, and the proximity effect between them.

cond-mat.mtrl-sci↗
arXiv2023

MMA-Net: Multiple Morphology-Aware Network for Automated Cobb Angle Measurement

Scoliosis diagnosis and assessment depend largely on the measurement of the Cobb angle in spine X-ray images. With the emergence of deep learning techniques that employ landmark detection, tilt prediction, and spine segmentation, automated Cobb angle measurement has become increasingly popular. However, these methods encounter difficulties such as high noise sensitivity, intricate computational procedures, and exclusive reliance on a single type of morphological information. In this paper, we introduce the Multiple Morphology-Aware Network (MMA-Net), a novel framework that improves Cobb angle measurement accuracy by integrating multiple spine morphology as attention information. In the MMA-Net, we first feed spine X-ray images into the segmentation network to produce multiple morphological information (spine region, centerline, and boundary) and then concatenate the original X-ray image with the resulting segmentation maps as input for the regression module to perform precise Cobb angle measurement. Furthermore, we devise joint loss functions for our segmentation and regression network training, respectively. We evaluate our method on the AASCE challenge dataset and achieve superior performance with the SMAPE of 7.28% and the MAE of 3.18°, indicating a strong competitiveness compared to other outstanding methods. Consequently, we can offer clinicians automated, efficient, and reliable Cobb angle measurement.

eess.IV↗
arXiv2023

Dynamic Frame Interpolation in Wavelet Domain

Video frame interpolation is an important low-level vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. However, the spatial redundancy when synthesizing the target frame has not been fully explored, that can result in lots of inefficient computation. On the other hand, the computation compression degree in frame interpolation is highly dependent on both texture distribution and scene motion, which demands to understand the spatial-temporal information of each input frame pair for a better compression degree selection. In this work, we propose a novel two-stage frame interpolation framework termed WaveletVFI to address above problems. It first estimates intermediate optical flow with a lightweight motion perception network, and then a wavelet synthesis network uses flow aligned context features to predict multi-scale wavelet coefficients with sparse convolution for efficient target frame reconstruction, where the sparse valid masks that control computation in each scale are determined by a crucial threshold ratio. Instead of setting a fixed value like previous methods, we find that embedding a classifier in the motion perception network to learn a dynamic threshold for each sample can achieve more computation reduction with almost no loss of accuracy. On the common high resolution and animation frame interpolation benchmarks, proposed WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts. Code is available at https://github.com/ltkong218/WaveletVFI.

cs.CV↗
arXiv2023

Over-Sampling Strategy in Feature Space for Graphs based Class-imbalanced Bot Detection

The presence of a large number of bots in Online Social Networks (OSN) leads to undesirable social effects. Graph neural networks (GNNs) are effective in detecting bots as they utilize user interactions. However, class-imbalanced issues can affect bot detection performance. To address this, we propose an over-sampling strategy for GNNs (OS-GNN) that generates samples for the minority class without edge synthesis. First, node features are mapped to a feature space through neighborhood aggregation. Then, we generate samples for the minority class in the feature space. Finally, the augmented features are used to train the classifiers. This framework is general and can be easily extended into different GNN architectures. The proposed framework is evaluated using three real-world bot detection benchmark datasets, and it consistently exhibits superiority over the baselines.

cs.CV↗
arXiv2023

A General Equivalence Theorem for Crossover Designs under Generalized Linear Models

With the help of Generalized Estimating Equations, we identify locally D-optimal crossover designs for generalized linear models. We adopt the variance of parameters of interest as the objective function, which is minimized using constrained optimization to obtain optimal crossover designs. In this case, the traditional general equivalence theorem could not be used directly to check the optimality of obtained designs. In this manuscript, we derive a corresponding general equivalence theorem for crossover designs under generalized linear models.

stat.ME↗
arXiv2023

Pseudo entropy of primary operators in $T\bar{T}$/$J\bar{T}$-deformed CFTs

In this work, we investigate the time evolution of the pseudo-(Rényi) entropy after local primary operator quenches in 2D CFTs with $T\bar T/J\bar T$-deformation. Using perturbation theory, we analyze the corrections to the second pseudo-Rényi entropy at the late time, which exhibit a universal form, while its early-time behavior is model-dependent. Moreover, we uncover nontrivial time-dependent effects arising from the first-order deformation of the $k^{\rm th}$ pseudo-Rényi entropy at the late time. Additionally, drawing inspiration from the gravitational side, specifically the gluing of two cutoff AdS geometries, we investigate the $k^{\rm th}$ pseudo-Rényi entropy for vacuum states characterized by distinct $T\bar{T}$-deformation parameters, as well as for primary states acting on different deformed vacuum states. Our findings reveal additional corrections compared to the results of pseudo-Rényi entropy for globally deformed vacuum states.

hep-th↗
arXiv2023

Neural Network-Based Histologic Remission Prediction In Ulcerative Colitis

BACKGROUND & AIMS: Histological remission (HR) is advocated and considered as a new therapeutic target in ulcerative colitis (UC). Diagnosis of histologic remission currently relies on biopsy; during this process, patients are at risk for bleeding, infection, and post-biopsy fibrosis. In addition, histologic response scoring is complex and time-consuming, and there is heterogeneity among pathologists. Endocytoscopy (EC) is a novel ultra-high magnification endoscopic technique that can provide excellent in vivo assessment of glands. Based on the EC technique, we propose a neural network model that can assess histological disease activity in UC using EC images to address the above issues. The experiment results demonstrate that the proposed method can assist patients in precise treatment and prognostic assessment. METHODS: We construct a neural network model for UC evaluation. A total of 5105 images of 154 intestinal segments from 87 patients undergoing EC treatment at a center in China between March 2022 and March 2023 are scored according to the Geboes score. Subsequently, 103 intestinal segments are used as the training set, 16 intestinal segments are used as the validation set for neural network training, and the remaining 35 intestinal segments are used as the test set to measure the model performance together with the validation set. RESULTS: By treating HR as a negative category and histologic activity as a positive category, the proposed neural network model can achieve an accuracy of 0.9, a specificity of 0.95, a sensitivity of 0.75, and an area under the curve (AUC) of 0.81. CONCLUSION: We develop a specific neural network model that can distinguish histologic remission/activity in EC images of UC, which helps to accelerate clinical histological diagnosis. keywords: ulcerative colitis; Endocytoscopy; Geboes score; neural network.

cs.CV↗
arXiv2023

ARTIST: ARTificial Intelligence for Simplified Text

Complex text is a major barrier for many citizens when accessing public information and knowledge. While often done manually, Text Simplification is a key Natural Language Processing task that aims for reducing the linguistic complexity of a text while preserving the original meaning. Recent advances in Generative Artificial Intelligence (AI) have enabled automatic text simplification both on the lexical and syntactical levels. However, as applications often focus on English, little is understood about the effectiveness of Generative AI techniques on low-resource languages such as Dutch. For this reason, we carry out empirical studies to understand the benefits and limitations of applying generative technologies for text simplification and provide the following outcomes: 1) the design and implementation for a configurable text simplification pipeline that orchestrates state-of-the-art generative text simplification models, domain and reader adaptation, and visualisation modules; 2) insights and lessons learned, showing the strengths of automatic text simplification while exposing the challenges in handling cultural and commonsense knowledge. These outcomes represent a first step in the exploration of Dutch text simplification and shed light on future endeavours both for research and practice.

cs.CL↗
arXiv2023

Neural Interactive Keypoint Detection

This work proposes an end-to-end neural interactive keypoint detection framework named Click-Pose, which can significantly reduce more than 10 times labeling costs of 2D keypoint annotation compared with manual-only annotation. Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process. Specifically, we design the pose error modeling strategy that inputs the ground truth pose combined with four typical pose errors into the decoder and trains the model to reconstruct the correct poses, which enhances the self-correction ability of the model. Then, we attach an interactive human-feedback loop that allows receiving users' clicks to correct one or several predicted keypoints and iteratively utilizes the decoder to update all other keypoints with a minimum number of clicks (NoC) for efficient annotation. We validate Click-Pose in in-domain, out-of-domain scenes, and a new task of keypoint adaptation. For annotation, Click-Pose only needs 1.97 and 6.45 NoC@95 (at precision 95%) on COCO and Human-Art, reducing 31.4% and 36.3% efforts than the SOTA model (ViTPose) with manual correction, respectively. Besides, without user clicks, Click-Pose surpasses the previous end-to-end model by 1.4 AP on COCO and 3.0 AP on Human-Art. The code is available at https://github.com/IDEA-Research/Click-Pose.

cs.CV↗
arXiv2023

Tackling Vision Language Tasks Through Learning Inner Monologues

Visual language tasks require AI models to comprehend and reason with both visual and textual content. Driven by the power of Large Language Models (LLMs), two prominent methods have emerged: (1) the hybrid integration between LLMs and Vision-Language Models (VLMs), where visual inputs are firstly converted into language descriptions by VLMs, serving as inputs for LLMs to generate final answer(s); (2) visual feature alignment in language space, where visual inputs are encoded as embeddings and projected to LLMs' language space via further supervised fine-tuning. The first approach provides light training costs and interpretability but is hard to be optimized in an end-to-end fashion. The second approach presents decent performance, but feature alignment usually requires large amounts of training data and lacks interpretability. To tackle this dilemma, we propose a novel approach, Inner Monologue Multi-Modal Optimization (IMMO), to solve complex vision language problems by simulating inner monologue processes, a cognitive process in which an individual engages in silent verbal communication with themselves. We enable LLMs and VLMs to interact through natural language conversation and propose to use a two-stage training process to learn how to do the inner monologue (self-asking questions and answering questions). IMMO is evaluated on two popular tasks and the results suggest by emulating the cognitive phenomenon of internal dialogue, our approach can enhance reasoning and explanation abilities, contributing to the more effective fusion of vision and language models. More importantly, instead of using predefined human-crafted monologues, IMMO learns this process within the deep learning models, promising wider applicability to many different AI problems beyond vision language tasks.

cs.CL↗
arXiv2023

MC-DRE: Multi-Aspect Cross Integration for Drug Event/Entity Extraction

Extracting meaningful drug-related information chunks, such as adverse drug events (ADE), is crucial for preventing morbidity and saving many lives. Most ADEs are reported via an unstructured conversation with the medical context, so applying a general entity recognition approach is not sufficient enough. In this paper, we propose a new multi-aspect cross-integration framework for drug entity/event detection by capturing and aligning different context/language/knowledge properties from drug-related documents. We first construct multi-aspect encoders to describe semantic, syntactic, and medical document contextual information by conducting those slot tagging tasks, main drug entity/event detection, part-of-speech tagging, and general medical named entity recognition. Then, each encoder conducts cross-integration with other contextual information in three ways: the key-value cross, attention cross, and feedforward cross, so the multi-encoders are integrated in depth. Our model outperforms all SOTA on two widely used tasks, flat entity detection and discontinuous event extraction.

cs.CL↗