arXiv subjects
Publications and source records attributed to Jie Li.
This technical report describes ChinaTelecom system for Track 1 (closed) of the VoxCeleb2023 Speaker Recognition Challenge (VoxSRC 2023). Our system consists of several ResNet variants trained only on VoxCeleb2, which were fused for better performance later. Score calibration was also applied for each variant and the fused system. The final submission achieved minDCF of 0.1066 and EER of 1.980%.
Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects, which plays a curtail role in high-level semantic understanding tasks. However, most works pursue designing better architectures to learn overall features more efficiently, while ignoring the long-tail nature of interaction-object pair categories. In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL). Firstly, a novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images. In this stage, virtual images are generated based on prompts with specific characterizations and selected by multi-filtering processes. Secondly, we use both virtual and real images to train the model with the teacher-student framework. Considering the initial labels of some virtual images are inaccurate and inadequate, we devise an Adaptive Matching-and-Filtering (AMF) module to construct pseudo-labels. Our method is independent of the internal structure of HOI detectors, so it can be combined with off-the-shelf methods by training merely 10 additional epochs. With the assistance of our method, multiple methods obtain significant improvements, and new state-of-the-art results are achieved on two benchmarks.
In this paper, we study the Feldman-Katok metric. We give entropy formulas by replacing Bowen metric with Feldman-Katok metric. Some relative topics are also discussed.
To better care for the elderly and disabled, it is essential for service robots to have an effective fusion method of object detection and grasp estimation. However, limited research has been observed on the combination of object detection and grasp estimation. To overcome this technical difficulty, a novel integrated method of detection-grasping for specific object based on the box coordinate matching is proposed in this paper. Firstly, the SOLOv2 instance segmentation model is improved by adding channel attention module (CAM) and spatial attention module (SAM). Then, the atrous spatial pyramid pooling (ASPP) and CAM are added to the generative residual convolutional neural network (GR-CNN) model to optimize grasp estimation. Furthermore, a detection-grasping integrated algorithm based on box coordinate matching (DG-BCM) is proposed to obtain the fusion model of object detection and grasp estimation. For verification, experiments on object detection and grasp estimation are conducted separately to verify the superiority of improved models. Additionally, grasping tasks for several specific objects are implemented on a simulation platform, demonstrating the feasibility and effectiveness of DG-BCM algorithm proposed in this paper.
Light-matter interaction is crucial to both understanding fundamental phenomena and developing versatile applications. Strong coupling, robustness, and controllability are the three most important aspects in realizing light-matter interactions. Topological and non-Hermitian photonics, have provided frameworks for robustness and extensive control freedom, respectively. How to engineer the properties of the edge state such as photonic density of state, scattering parameters by using non-Hermitian engineering while ensuring topological protection has not been fully studied. Here we construct a parity-time-symmetric dimerized photonic lattice and generate complex-valued edge states via spontaneous PT-symmetry breaking. The enhanced strong coupling between the topological photonic edge mode and magnon mode in a ferromagnetic spin ensemble is demonstrated. Our research reveals the subtle non-Hermitian topological edge states and provides strategies for realizing and engineering topological light-matter interactions.
Advanced manipulation techniques have provided criminals with opportunities to make social panic or gain illicit profits through the generation of deceptive media, such as forged face images. In response, various deepfake detection methods have been proposed to assess image authenticity. Sequential deepfake detection, which is an extension of deepfake detection, aims to identify forged facial regions with the correct sequence for recovery. Nonetheless, due to the different combinations of spatial and sequential manipulations, forged face images exhibit substantial discrepancies that severely impact detection performance. Additionally, the recovery of forged images requires knowledge of the manipulation model to implement inverse transformations, which is difficult to ascertain as relevant techniques are often concealed by attackers. To address these issues, we propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images and achieve recovery without requiring knowledge of the corresponding manipulation method. Furthermore, existing evaluation metrics only consider detection accuracy at a single inferring step, without accounting for the matching degree with ground-truth under continuous multiple steps. To overcome this limitation, we propose a novel evaluation metric called Complete Sequence Matching (CSM), which considers the detection accuracy at multiple inferring steps, reflecting the ability to detect integrally forged sequences. Extensive experiments on several typical datasets demonstrate that MMNet achieves state-of-the-art detection performance and independent recovery performance.
The constant interplay and information exchange between cells and their micro-environment are essential to their survival and ability to execute biological functions. To date, a few leading technologies such as traction force microscopy, have been broadly used in measuring cellular forces. However, the considerable limitations, regarding the sensitivity and ambiguities in data interpretation, are hindering our thorough understanding of mechanobiology. Herein, we propose an innovative approach, namely quantum-enhanced diamond molecular tension microscopy (QDMTM), to precisely quantify the integrin-based cell adhesive forces. Specifically, we construct a force sensing platform by conjugating the magnetic nanotags labeled, force-responsive polymer to the surface of diamond membrane containing nitrogen vacancy (NV) centers. Thus, the coupled mechanical information can be quantified through optical readout of spin relaxation of NV centers modulated by those magnetic nanotags. To validate QDMTM, we have carefully performed corresponding measurements both in control and real cell samples. Particularly, we have obtained the quantitative cellular adhesion force mapping by correlating the measurement with established theoretical model. We anticipate that our method can be routinely used in studying important issues like cell-cell or cell-material interactions and mechanotransduction.
Mechanical modulation of the lattice parameter can modify the electronic structure and manipulate the magnetic coupling of a material without introducing impurities. Inspired by success in pressure-controlled magnetism, we investigate the effect of hydrostatic pressure on quantized Chern states in the antiferromagnetic topological insulator MnBi2Te4, using transport as a probe. We show that pressure can enhance the robustness of quantum anomalous Hall (QAH) phases that are otherwise delicate in 7SL MnBi2Te4 and in the spin-flop (SF) state of 8SL MnBi2Te4. We explain our findings using a coupled Dirac cone model of MnBi2Te4, which identifies stronger hybridization between van der Waals layers as the driver of topological states. We further demonstrate that moderate pressures readily available in laboratory systems can provide reversible control of magnetic and topological phases. Our results reveal a strong connection between the mechanical engineering of band topology and magnetism.
Medical image segmentation based on deep learning often fails when deployed on images from a different domain. The domain adaptation methods aim to solve domain-shift challenges, but still face some problems. The transfer learning methods require annotation on the target domain, and the generative unsupervised domain adaptation (UDA) models ignore domain-specific representations, whose generated quality highly restricts segmentation performance. In this study, we propose a novel Structure-Modal Constrained (SMC) UDA framework based on a discriminative paradigm and introduce edge structure as a bridge between domains. The proposed multi-modal learning backbone distills structure information from image texture to distinguish domain-invariant edge structure. With the structure-constrained self-learning and progressive ROI, our methods segment the kidney by locating the 3D spatial structure of the edge. We evaluated SMC-UDA on public renal segmentation datasets, adapting from the labeled source domain (CT) to the unlabeled target domain (CT/MRI). The experiments show that our proposed SMC-UDA has a strong generalization and outperforms generative UDA methods.
Theoretical predictions of NMR chemical shifts from first-principles can greatly facilitate experimental interpretation and structure identification. However, accurate prediction of chemical shifts using the best coupled cluster methods can be prohibitively expensive for systems larger than ten to twenty non-hydrogen atoms on today's computers. By contrast machine learning methods offer inexpensive alternatives but are hampered by generalization to molecules outside the original training set. Here we propose a novel machine learning feature representation informed by intermediate calculations of atomic chemical shielding tensors within a molecular environment using an inexpensive quantum mechanics method, and training it to predict NMR chemical shieldings of a high-level composite theory that is comparable to CCSD(T) in the complete basis set limit. The inexpensive shift machine learning (iShiftML) algorithm is trained through a new progressive active learning workflow that reduces the total number of expensive calculations required when constructing the dataset, while allowing the model to continuously improve on data it has never seen. Furthermore, we show that the error estimations from our model correlate quite well with actual errors to provide confidence values on new predictions. We illustrate the predictive capacity of iShiftML across gas phase experimental chemical shifts for small organic molecules and much larger and more complex natural products in which we can accurately differentiate between subtle diastereomers based on chemical shift assignments.
Federated Learning (FL) has emerged as a promising approach for collaborative model training without sharing private data. However, privacy concerns regarding information exchanged during FL have received significant research attention. Gradient Inversion Attacks (GIAs) have been proposed to reconstruct the private data retained by local clients from the exchanged gradients. While recovering private data, the data dimensions and the model complexity increase, which thwart data reconstruction by GIAs. Existing methods adopt prior knowledge about private data to overcome those challenges. In this paper, we first observe that GIAs with gradients from a single iteration fail to reconstruct private data due to insufficient dimensions of leaked gradients, complex model architectures, and invalid gradient information. We investigate a Temporal Gradient Inversion Attack with a Robust Optimization framework, called TGIAs-RO, which recovers private data without any prior knowledge by leveraging multiple temporal gradients. To eliminate the negative impacts of outliers, e.g., invalid gradients for collaborative optimization, robust statistics are proposed. Theoretical guarantees on the recovery performance and robustness of TGIAs-RO against invalid gradients are also provided. Extensive empirical results on MNIST, CIFAR10, ImageNet and Reuters 21578 datasets show that the proposed TGIAs-RO with 10 temporal gradients improves reconstruction performance compared to state-of-the-art methods, even for large batch sizes (up to 128), complex models like ResNet18, and large datasets like ImageNet (224*224 pixels). Furthermore, the proposed attack method inspires further exploration of privacy-preserving methods in the context of FL.
We show how to prepare macroscopic entanglement between an atomic ensemble and a large number of magnons in a ferrimagnetic YIG crystal. Specifically, we adopt an opto-magnomechanical configuration where the magnetostriction-induced magnomechanical displacement couples to an optical cavity via radiation pressure, and the latter further couples to an ensemble of two-level atoms that are placed inside the cavity. We show that by properly driving the cavity and magnon modes, optomechanical entanglement is created which is further distributed to the atomic and magnonic systems, yielding stationary entanglement between atoms and magnons. The atom-magnon entanglement is a result of the combined effect of opto- and magnomechanical cooling and optomechanical parametric down-conversion interactions. A competition mechanism between two mechanical cooling channels is revealed. We further show that genuine tripartite entanglement of three massive subsystems, i.e., atoms, magnons and phonons, can also be achieved in the same system. Our results indicate that the hybrid opto-magnomechanical system may become a promising system for preparing macroscopic quantum states involving magnons, photons, phonons and atoms.
We investigate the hydrostatic equilibrium of white dwarfs within the framework of Rastall-Rainbow gravity, aiming to explore the effects of this modified gravitational theory on their properties. By employing the Chandrasekhar equation of state in conjunction with the modified Tolman-Oppenheimer-Volkoff equation, we derive the mass-radius relations for white dwarfs. Our results show that the maximum mass of white dwarfs deviates significantly from the predictions of general relativity, potentially exceeding the Chandrasekhar limit. Furthermore, we discuss other properties of white dwarfs, such as the gravitational redshift, compactness and dynamical stability, shedding light on their behavior within the context of this modified gravitational framework.
Abundant high-rate (n, k) minimum storage regenerating (MSR) codes have been reported in the literature. However, most of them require contacting all the surviving nodes during a node repair process, resulting in a repair degree of d=n-1. In practical systems, it may not always be feasible to connect and download data from all surviving nodes, as some nodes may be unavailable. Therefore, there is a need for MSR code constructions with a repair degree of d k, and has a smaller sub-packetization level or finite field than some existing constructions. Additionally, in conjunction with a previous generic transformation to reduce the sub-packetization level, we obtain an MDS array code with a small sub-packetization level and $(1+ε)$-optimal repair bandwidth (i.e., $(1+ε)$ times the optimal repair bandwidth) for repair degree d=n-1. This code outperforms some existing ones in terms of either the sub-packetization level or the field size.
Stroke can lead to the impaired motor ability of the patient's lower limbs and hemiplegia. Accurate assessment of the lower limbs' motor ability is important for diagnosis and rehabilitation. To digitalize such assessment so that each test can be traced back any time and subjectivity can be avoided, we test how dual-modality smart shoes equipped with pressure-sensitive insoles and inertial measurement units can be used for this purpose. A 5m walking test protocol, including the left and right turns, is designed. Data are collected from 23 patients and 17 healthy subjects. For the lower limbs' motor ability, the tests are observed by two physicians and assessed using the five graded Medical Research Council scale for muscle examination. The average of two physicians' scores for the same patient is used as the ground truth. Using the feature set we developed, 100\% accuracy is achieved in classifying the patients and healthy subjects. For patients' muscle strength, a mean absolute error of 0.143 and a maximum error of 0.395 is achieved using our feature set and the regression method, closer to the ground truth than the scores from each physician (mean absolute error: 0.217, maximum error: 0.5). We thus validate the possibility of using such smart shoes to objectively and accurately evaluate the lower limbs' muscle strength of the stroke patients.
How humans understand and recognize the actions of others is a complex neuroscientific problem that involves a combination of cognitive mechanisms and neural networks. Research has shown that humans have brain areas that recognize actions that process top-down attentional information, such as the temporoparietal association area. Also, humans have brain regions dedicated to understanding the minds of others and analyzing their intentions, such as the medial prefrontal cortex of the temporal lobe. Skeleton-based action recognition creates mappings for the complex connections between the human skeleton movement patterns and behaviors. Although existing studies encoded meaningful node relationships and synthesized action representations for classification with good results, few of them considered incorporating a priori knowledge to aid potential representation learning for better performance. LA-GCN proposes a graph convolution network using large-scale language models (LLM) knowledge assistance. First, the LLM knowledge is mapped into a priori global relationship (GPR) topology and a priori category relationship (CPR) topology between nodes. The GPR guides the generation of new "bone" representations, aiming to emphasize essential node information from the data level. The CPR mapping simulates category prior knowledge in human brain regions, encoded by the PC-AC module and used to add additional supervision-forcing the model to learn class-distinguishable features. In addition, to improve information transfer efficiency in topology modeling, we propose multi-hop attention graph convolution. It aggregates each node's k-order neighbor simultaneously to speed up model convergence. LA-GCN reaches state-of-the-art on NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.
Non-classical quantum states are the pivotal features of a quantum system that differs from its classical counterpart. However, the generation and coherent control of quantum states in a macroscopic spin system remain an outstanding challenge. Here we experimentally demonstrate the quantum control of a single magnon in a macroscopic spin system (i.e., 1~mm-diameter yttrium-iron-garnet sphere) coupled to a superconducting qubit via a microwave cavity. By tuning the qubit frequency {\it in situ} via the Autler-Townes effect, we manipulate this single magnon to generate its non-classical quantum states, including the single-magnon state and the superposition state of a single magnon and vacuum. Moreover, we confirm the deterministic generation of these non-classical states by Wigner tomography. Our experiment offers the first reported deterministic generation of the non-classical quantum states in a macroscopic spin system and paves a way to explore its promising applications in quantum engineering.
A protocol for realizing discrete-variable quantum teleportation in an optomagnonic system is provided. Using optical pulses, an arbitrary photonic qubit state encoded in orthogonal polarizations is transferred onto the joint state of a pair of magnonic oscillators in two macroscopic yttrium-iron-garnet (YIG) spheres that are placed in an optical interferometer. We further show that optomagnonic entanglement swapping can be realized in an extended dual-interferometer configuration with a joint Bell-state detection. Consequently, magnon Bell states are prepared. We analyze the effect of the residual thermal occupation of the magnon modes on the fidelity in both the teleportation and entanglement swapping protocols.