arXiv subjects

Yin Li

Publications and source records attributed to Yin Li.

At least 55 records · Page 3Linked to original sources

arXiv2025

SAM2MOT: A Novel Paradigm of Multi-Object Tracking by Segmentation

Inspired by Segment Anything 2, which generalizes segmentation from images to videos, we propose SAM2MOT--a novel segmentation-driven paradigm for multi-object tracking that breaks away from the conventional detection-association framework. In contrast to previous approaches that treat segmentation as auxiliary information, SAM2MOT places it at the heart of the tracking process, systematically tackling challenges like false positives and occlusions. Its effectiveness has been thoroughly validated on major MOT benchmarks. Furthermore, SAM2MOT integrates pre-trained detector, pre-trained segmentor with tracking logic into a zero-shot MOT system that requires no fine-tuning. This significantly reduces dependence on labeled data and paves the way for transitioning MOT research from task-specific solutions to general-purpose systems. Experiments on DanceTrack, UAVDT, and BDD100K show state-of-the-art results. Notably, SAM2MOT outperforms existing methods on DanceTrack by +2.1 HOTA and +4.5 IDF1, highlighting its effectiveness in MOT. Code is available at https://github.com/TripleJoy/SAM2MOT.

cs.CV↗

arXiv2025

Persistence of unknottedness of clean Lagrangian intersections

Let $Q_0$ and $Q_1$ be two Lagrangian spheres in a $6$-dimensional symplectic manifold. Assume that $Q_0$ and $Q_1$ intersect cleanly along a circle that is unknotted in both $Q_0$ and $Q_1$. We prove that there is no nearby Hamiltonian isotopy of $Q_0$ and $Q_1$ to a pair of Lagrangian spheres meeting cleanly along a circle that is knotted in either component, answering a question of Smith. The proof is based on a classification of the spherical summands in the prime decomposition of an exact Lagrangian in the Stein neighborhood of the union $Q_0\cup Q_1$ and the deep result that lens space rational Dehn surgeries characterize the unknot.

math.SG↗

arXiv2025

TUS-REC2024: A Challenge to Reconstruct 3D Freehand Ultrasound Without External Tracker

Trackerless freehand ultrasound reconstruction aims to reconstruct 3D volumes from sequences of 2D ultrasound images without relying on external tracking systems. By eliminating the need for optical or electromagnetic trackers, this approach offers a low-cost, portable, and widely deployable alternative to more expensive volumetric ultrasound imaging systems, particularly valuable in resource-constrained clinical settings. However, predicting long-distance transformations and handling complex probe trajectories remain challenging. The TUS-REC2024 Challenge establishes the first benchmark for trackerless 3D freehand ultrasound reconstruction by providing a large publicly available dataset, along with a baseline model and a rigorous evaluation framework. By the submission deadline, the Challenge had attracted 43 registered teams, of which 6 teams submitted 21 valid dockerized solutions. The submitted methods span a wide range of approaches, including the state space model, the recurrent model, the registration-driven volume refinement, the attention mechanism, and the physics-informed model. This paper provides a comprehensive background introduction and literature review in the field, presents an overview of the challenge design and dataset, and offers a comparative analysis of submitted methods across multiple evaluation metrics. These analyses highlight both the progress and the current limitations of state-of-the-art approaches in this domain and provide insights for future research directions. All data and code are publicly available to facilitate ongoing development and reproducibility. As a live and evolving benchmark, it is designed to be continuously iterated and improved. The Challenge was held at MICCAI 2024 and is organised again at MICCAI 2025, reflecting its sustained commitment to advancing this field.

eess.IV↗

arXiv2025

Visual Bridge: Universal Visual Perception Representations Generating

Recent advances in diffusion models have achieved remarkable success in isolated computer vision tasks such as text-to-image generation, depth estimation, and optical flow. However, these models are often restricted by a ``single-task-single-model'' paradigm, severely limiting their generalizability and scalability in multi-task scenarios. Motivated by the cross-domain generalization ability of large language models, we propose a universal visual perception framework based on flow matching that can generate diverse visual representations across multiple tasks. Our approach formulates the process as a universal flow-matching problem from image patch tokens to task-specific representations rather than an independent generation or regression problem. By leveraging a strong self-supervised foundation model as the anchor and introducing a multi-scale, circular task embedding mechanism, our method learns a universal velocity field to bridge the gap between heterogeneous tasks, supporting efficient and flexible representation transfer. Extensive experiments on classification, detection, segmentation, depth estimation, and image-text retrieval demonstrate that our model achieves competitive performance in both zero-shot and fine-tuned settings, outperforming prior generalist and several specialist models. Ablation studies further validate the robustness, scalability, and generalization of our framework. Our work marks a significant step towards general-purpose visual perception, providing a solid foundation for future research in universal vision modeling.

cs.CV↗

arXiv2025

Mock Observations for the CSST Mission: Multi-Channel Imager--The Cluster Field

The Multi-Channel Imager (MCI), one of the instruments aboard the China Survey Space Telescope (CSST), is designed to simultaneously observe the sky in three filters, covering wavelengths from the near-ultraviolet (NUV) to the near-infrared (NIR). With its large field of view ($7.5^{\prime}\times7.5^{\prime}$), MCI is particularly well-suited for observing galaxy clusters, providing a powerful tool for investigating galaxy evolution, dark matter and dark energy through gravitational lensing. Here we present a comprehensive simulation framework of a strong lensing cluster as observed by MCI, aiming to fully exploit its capabilities in capturing lensing features. The framework simulates a strong lensing cluster from the CosmoDC2 catalog, calculating the gravitational potential and performing ray-tracing to derive the true positions, shapes and light distribution of galaxies within the cluster field. Additionally, the simulation incorporates intra-cluster light (ICL) and spectral energy distributions (SEDs), enabling further strong lensing analyses, such as ICL seperation from galaxy light and mass reconstruction combining strong and weak lensing measurements. This framework provides a critical benchmark for testing the MCI data pipeline and maximizing its potential in galaxy cluster research.

astro-ph.IM↗

arXiv2025

NF-SecRIS: RIS-Assisted Near-Field Physical Layer Security via Secure Location Modulation

The 6G wireless networks impose extremely high requirements on physical layer secure communication. However, the existing solutions usually can only achieve one-dimensional physical layer security (PLS) in the angle dimension, and cannot achieve PLS in the range dimension. In this paper, we propose the NF-SecRIS system, the first range-angle-dependent (2D) PLS near-field communication system based on ultra-large-scale reconfigurable intelligent surface (RIS). We propose the secure location modulation scheme to synthesize the near-field spatial-temporal coding pattern of RIS with extremely low complexity. It ensures that only legitimate user can receive the raw constellations, while potential eavesdroppers at other ranges or angles can only receive the obfuscated constellations. NF-SecRIS operates without requiring synchronization with either transmitter or receiver. We implement a prototype of NF-SecRIS and conduct comprehensive experiments with multiple modulation schemes. The results show that the bit error rate (BER) of legitimate user is below 10^{-4}, while eavesdroppers at other ranges or angles suffer from BER exceeding 40%. It validates the implementation of 2D PLS in near-field communications.

cs.ET↗

arXiv2025

TerraGen: A Unified Multi-Task Layout Generation Framework for Remote Sensing Data Augmentation

Remote sensing vision tasks require extensive labeled data across multiple, interconnected domains. However, current generative data augmentation frameworks are task-isolated, i.e., each vision task requires training an independent generative model, and ignores the modeling of geographical information and spatial constraints. To address these issues, we propose \textbf{TerraGen}, a unified layout-to-image generation framework that enables flexible, spatially controllable synthesis of remote sensing imagery for various high-level vision tasks, e.g., detection, segmentation, and extraction. Specifically, TerraGen introduces a geographic-spatial layout encoder that unifies bounding box and segmentation mask inputs, combined with a multi-scale injection scheme and mask-weighted loss to explicitly encode spatial constraints, from global structures to fine details. Also, we construct the first large-scale multi-task remote sensing layout generation dataset containing 45k images and establish a standardized evaluation protocol for this task. Experimental results show that our TerraGen can achieve the best generation image quality across diverse tasks. Additionally, TerraGen can be used as a universal data-augmentation generator, enhancing downstream task performance significantly and demonstrating robust cross-task generalisation in both full-data and few-shot scenarios.

cs.CV↗

arXiv2025

CURLING -- II. Improvement on the $H_{0}$ Inference from Pixelized Cluster Strong Lens Modeling

Strongly lensed supernovae (glSNe) provide a powerful, independent method to measure the Hubble constant, $H_{0}$, through time delays between their multiple images. The accuracy of this measurement depends critically on both the precision of time delay estimation and the robustness of lens modeling. In many current cluster-scale modeling algorithms, all multiple images used for modeling are simplified as point sources to reduce computational costs. In the first paper of the CURLING program, we demonstrated that such a point-like approximation can introduce significant uncertainties and biases in both magnification reconstruction and cosmological inference. In this study, we explore how such simplifications affect $H_0$ measurements from glSNe. We simulate a lensed supernova at $z=1.95$, lensed by a galaxy cluster at $z=0.336$, assuming time delays are measured from LSST-like light curves. The lens model is constructed using JWST-like imaging data, utilizing both Lenstool and a pixelated method developed in CURLING. Under a fiducial cosmology with $H_0=70\rm \ km \ s^{-1}\ Mpc^{-1}$, the Lenstool model yields $H_0=69.91^{+6.27}_{-5.50}\rm \ km\ s^{-1}\ Mpc^{-1}$, whereas the pixelated framework improves the precision by over an order of magnitude, $H_0=70.39^{+0.82}_{-0.60}\rm \ km \ s^{-1}\ Mpc^{-1}$. Our results indicate that in the next-generation observations (e.g., JWST), uncertainties from lens modeling dominate the error budget for $H_0$ inference, emphasizing the importance of incorporating the extended surface brightness of multiple images to fully leverage the potential of glSNe for cosmology.

astro-ph.CO↗

arXiv2025

Audio-Enhanced Vision-Language Modeling with Latent Space Broadening for High Quality Data Expansion

Transformer-based multimodal models are widely used in industrial-scale recommendation, search, and advertising systems for content understanding and relevance ranking. Enhancing labeled training data quality and cross-modal fusion significantly improves model performance, influencing key metrics such as quality view rates and ad revenue. High-quality annotations are crucial for advancing content modeling, yet traditional statistical-based active learning (AL) methods face limitations: they struggle to detect overconfident misclassifications and are less effective in distinguishing semantically similar items in deep neural networks. Additionally, audio information plays an increasing role, especially in short-video platforms, yet most pre-trained multimodal architectures primarily focus on text and images. While training from scratch across all three modalities is possible, it sacrifices the benefits of leveraging existing pre-trained visual-language (VL) and audio models. To address these challenges, we propose kNN-based Latent Space Broadening (LSB) to enhance AL efficiency and Vision-Language Modeling with Audio Enhancement (VLMAE), a mid-fusion approach integrating audio into VL models. This system deployed in production systems, leading to significant business gains.

cs.MM↗

arXiv2025

Variations of the vector magnetic structures in the solar polar regions observed by Hinode

Using the polar vector magnetic field data observed by Hinode from 2012 to 2021, we study the long-term variations of the magnetic flux, the flux proportion of different polarities, and the magnetic inclination with respect to the local normal in the solar polar regions above 70 degree latitude during solar cycle 24. In both polar regions after the polarity reversal, the unsigned magnetic fluxes of the dominant polarity increased to a peak of about 1.3$\times$10$^{22}$ Mx during the solar minimum, while those of the non-dominant polarity remained stable at approximately 0.5$\times$10$^{22}$ Mx. The proportions of the dominant polarity flux in the total flux in both polar regions increased to more than 70% at the solar minimum. These results reveal that the total magnetic flux and the number of open field lines in solar cycle 24 were larger than those in solar cycle 23, and also imply the existence of a local dynamo in polar regions. After the polarity reversal, the magnetic inclination of the dominant polarity fields decreased, indicating that the stronger the dominant polarity field, the more vertical the field lines. The inclination angle decreased with the increase of the threshold of radial magnetic flux density, revealing a fanning-out structure of the polar magnetic patches.

astro-ph.SR↗

arXiv2025

AI Pangaea: Unifying Intelligence Islands for Adapting Myriad Tasks

The pursuit of artificial general intelligence continuously demands generalization in one model across myriad tasks, even those not seen before. However, current AI models are isolated from each other for being limited to specific tasks, now first defined as Intelligence Islands. To unify Intelligence Islands into one, we propose Pangaea, the first AI supercontinent akin to the geological Pangaea. Pangaea encodes any data into a unified format and accumulates universal knowledge through pre-training on 296 datasets across diverse modalities. Eventually, it demonstrates remarkable generalization across 45 general tasks and 15 scientific tasks encompassing a wide range of scientific subjects. By investigating Pangaea deeper, the scaling effect of modality is revealed, quantifying the universal knowledge accumulation across modalities as the cumulative distribution function of a geometric distribution. On the whole, Pangaea shows strong potential to handle myriad tasks, indicating a new direction toward artificial general intelligence.

cs.AI↗

arXiv2025

Recovering Parametric Scenes from Very Few Time-of-Flight Pixels

We aim to recover the geometry of 3D parametric scenes using very few depth measurements from low-cost, commercially available time-of-flight sensors. These sensors offer very low spatial resolution (i.e., a single pixel), but image a wide field-of-view per pixel and capture detailed time-of-flight data in the form of time-resolved photon counts. This time-of-flight data encodes rich scene information and thus enables recovery of simple scenes from sparse measurements. We investigate the feasibility of using a distributed set of few measurements (e.g., as few as 15 pixels) to recover the geometry of simple parametric scenes with a strong prior, such as estimating the 6D pose of a known object. To achieve this, we design a method that utilizes both feed-forward prediction to infer scene parameters, and differentiable rendering within an analysis-by-synthesis framework to refine the scene parameter estimate. We develop hardware prototypes and demonstrate that our method effectively recovers object pose given an untextured 3D model in both simulations and controlled real-world captures, and show promising initial results for other parametric scenes. We additionally conduct experiments to explore the limits and capabilities of our imaging solution.

cs.CV↗

arXiv2025

CoPlay: Audio-agnostic Cognitive Scaling for Acoustic Sensing

Acoustic sensing manifests great potential in various applications that encompass health monitoring, gesture interface and imaging by leveraging the speakers and microphones on smart devices. However, in ongoing research and development in acoustic sensing, one problem is often overlooked: the same speaker, when used concurrently for sensing and other traditional applications (like playing music), could cause interference in both making it impractical to use in the real world. The strong ultrasonic sensing signals mixed with music would overload the speaker's mixer. To confront this issue of overloaded signals, current solutions are clipping or down-scaling, both of which affect the music playback quality and also sensing range and accuracy. To address this challenge, we propose CoPlay, a deep learning based optimization algorithm to cognitively adapt the sensing signal. It can 1) maximize the sensing signal magnitude within the available bandwidth left by the concurrent music to optimize sensing range and accuracy and 2) minimize any consequential frequency distortion that can affect music playback. In this work, we design a deep learning model and test it on common types of sensing signals (sine wave or Frequency Modulated Continuous Wave FMCW) as inputs with various agnostic concurrent music and speech. First, we evaluated the model performance to show the quality of the generated signals. Then we conducted field studies of downstream acoustic sensing tasks in the real world. A study with 12 users proved that respiration monitoring and gesture recognition using our adapted signal achieve similar accuracy as no-concurrent-music scenarios, while clipping or down-scaling manifests worse accuracy. A qualitative study also manifests that the music play quality is not degraded, unlike traditional clipping or down-scaling methods.

cs.SD↗

arXiv2025

Beyond-Voice: Towards Continuous 3D Hand Pose Tracking on Commercial Home Assistant Devices

The surging popularity of home assistants and their voice user interface (VUI) have made them an ideal central control hub for smart home devices. However, current form factors heavily rely on VUI, which poses accessibility and usability issues; some latest ones are equipped with additional cameras and displays, which are costly and raise privacy concerns. These concerns jointly motivate Beyond-Voice, a novel high-fidelity acoustic sensing system that allows commodity home assistant devices to track and reconstruct hand poses continuously. It transforms the home assistant into an active sonar system using its existing onboard microphones and speakers. We feed a high-resolution range profile to the deep learning model that can analyze the motions of multiple body parts and predict the 3D positions of 21 finger joints, bringing the granularity for acoustic hand tracking to the next level. It operates across different environments and users without the need for personalized training data. A user study with 11 participants in 3 different environments shows that Beyond-Voice can track joints with an average mean absolute error of 16.47mm without any training data provided by the testing subject.

cs.SD↗

arXiv2025

BodyWave: Egocentric Body Tracking using mmWave Radars on an MR Headset

Egocentric body tracking, also known as inside-out body tracking (IOBT), is an essential technology for applications like gesture control and codec avatar in mixed reality (MR), including augmented reality (AR) and virtual reality (VR). However, it is more challenging than exocentric body tracking due to the limited view angles of camera-based solutions, which provide only sparse and self-occluded input from head-mounted cameras, especially for lower-body parts. To address these challenges, we propose, BodyWave, an IOBT system based on millimeter-wave (mmWave) radar, which can detect non-line-of-sight. It offers low SWAP+C (size, weight, and power consumption), robustness to environmental and user factors, and enhanced privacy over camera-based solutions. Our prototype, modeled after the Meta Quest 3 form factor, places radars just 4cm away from the face, which significantly advances the practicality of radar-based IOBT. We tackle the sparsity issue of mmWave radar by processing the raw signal into high-resolution range profiles to predict fine-grained 3D coordinates of body keypoints. In a user study with 14 participants and around 500,000 frames of collected data, we achieved a mean per-joint position error (MPJPE) of 9.85 cm on unseen users, 4.94 cm with a few minutes of user calibration, and 3.86 cm in a fully-adapted user-dependent setting. This is comparable to state-of-the-art camera-based IOBT systems, introducing a robust and privacy-preserving alternative for MR applications.

eess.IV↗

arXiv2025

LETS Forecast: Learning Embedology for Time Series Forecasting

Real-world time series are often governed by complex nonlinear dynamics. Understanding these underlying dynamics is crucial for precise future prediction. While deep learning has achieved major success in time series forecasting, many existing approaches do not explicitly model the dynamics. To bridge this gap, we introduce DeepEDM, a framework that integrates nonlinear dynamical systems modeling with deep neural networks. Inspired by empirical dynamic modeling (EDM) and rooted in Takens' theorem, DeepEDM presents a novel deep model that learns a latent space from time-delayed embeddings, and employs kernel regression to approximate the underlying dynamics, while leveraging efficient implementation of softmax attention and allowing for accurate prediction of future time steps. To evaluate our method, we conduct comprehensive experiments on synthetic data of nonlinear dynamical systems as well as real-world time series across domains. Our results show that DeepEDM is robust to input noise, and outperforms state-of-the-art methods in forecasting accuracy. Our code is available at: https://abrarmajeedi.github.io/deep_edm.

cs.LG↗

arXiv2025

A dataset of primary nasopharyngeal carcinoma MRI with multi-modalities segmentation

Multi-modality magnetic resonance imaging(MRI) data facilitate the early diagnosis, tumor segmentation, and disease staging in the management of nasopharyngeal carcinoma (NPC). The lack of publicly available, comprehensive datasets limits advancements in diagnosis, treatment planning, and the development of machine learning algorithms for NPC. Addressing this critical need, we introduce the first comprehensive NPC MRI dataset, encompassing MR axial imaging of 277 primary NPC patients. This dataset includes T1-weighted, T2-weighted, and contrast-enhanced T1-weighted sequences, totaling 831 scans. In addition to the corresponding clinical data, manually annotated and labeled segmentations by experienced radiologists offer high-quality data resources from untreated primary NPC.

eess.IV↗

arXiv2025

Learning to Inference Adaptively for Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) have shown impressive capabilities in visual reasoning, yet come with substantial computational cost, limiting their deployment in resource-constrained settings. Despite recent effort on improving the efficiency of MLLMs, prior solutions fall short in responding to varying runtime conditions, in particular changing resource availability (e.g., contention due to the execution of other programs on the device). To bridge this gap, we introduce AdaLLaVA, an adaptive inference framework that learns to dynamically reconfigure operations in an MLLM during inference, accounting for the input data and a latency budget. We conduct extensive experiments across benchmarks involving question-answering, reasoning, and hallucination. Our results show that AdaLLaVA effectively adheres to input latency budget, achieving varying accuracy and latency tradeoffs at runtime. Further, we demonstrate that AdaLLaVA adapts to both input latency and content, can be integrated with token selection for enhanced efficiency, and generalizes across MLLMs. Our project webpage with code release is at https://zhuoyan-xu.github.io/ada-llava/.

cs.AI↗