arXiv Science⌕ Search

arXiv subjects

Siyuan Yu

Publications and source records attributed to Siyuan Yu.

At least 19 recordsLinked to original sources
arXiv2026

OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction

Fitting an underlying body model to 3D clothed human assets has been extensively studied, yet most approaches focus on either single-modal inputs such as point clouds or multi-view images alone, often requiring a known metric scale. This constraint is frequently impractical, especially for AI-generated assets where scale distortion is common. We propose OmniFit, a method that can seamlessly handle diverse multi-modal inputs, including full scans, partial depth observations, and image captures, while remaining scale-agnostic for both real and synthetic assets. Our key innovation is a simple yet effective conditional transformer decoder that directly maps surface points to dense body landmarks, which are then used for SMPL-X parameter fitting. In addition, an optional plug-and-play image adapter incorporates visual cues to compensate for missing geometric information. We further introduce a dedicated scale predictor that rescales subjects to canonical body proportions. OmniFit substantially outperforms state-of-the-art methods by 57.1 to 80.9 percent across daily and loose clothing scenarios. To the best of our knowledge, it is the first body fitting method to surpass multi-view optimization baselines and the first to achieve millimeter-level accuracy on the CAPE and 4D-DRESS benchmarks.

cs.CV↗
arXiv2026

ETCH-X: Robustify Expressive Body Fitting to Clothed Humans with Composable Datasets

Human body fitting, which aligns parametric body models such as SMPL to raw 3D point clouds of clothed humans, serves as a crucial first step for downstream tasks like animation and texturing. An effective fitting method should be both locally expressive-capturing fine details such as hands and facial features-and globally robust to handle real-world challenges, including clothing dynamics, pose variations, and noisy or partial inputs. Existing approaches typically excel in only one aspect, lacking an all-in-one solution. We upgrade ETCH to ETCH-X, which leverages a tightness-aware fitting paradigm to filter out clothing dynamics ("undress"), extends expressiveness with SMPL-X, and replaces explicit sparse markers (which are highly sensitive to partial data) with implicit dense correspondences ("dense fit") for more robust and fine-grained body fitting. Our disentangled "undress" and "dense fit" modular stages enable separate and scalable training on composable data sources, including diverse simulated garments (CLOTH3D), large-scale full-body motions (AMASS), and fine-grained hand gestures (InterHand2.6M), improving outfit generalization and pose robustness of both bodies and hands. Our approach achieves robust and expressive fitting across diverse clothing, poses, and levels of input completeness, delivering a substantial performance improvement over ETCH on both: 1) seen data, such as 4D-Dress (MPJPE-All, 33.0% ) and CAPE (V2V-Hands, 35.8% ), and 2) unseen data, such as BEDLAM2.0 (MPJPE-All, 80.8% ; V2V-All, 80.5% ). Code and models will be released at https://xiaobenli00.github.io/ETCH-X/.

cs.CV↗
arXiv2026

Toward scalable and bias-stable optical phased arrays on lithium tantalate

Ferroelectric materials are an ideal platform for high-speed reconfigurable photonic integrated circuits (PICs) for classical and quantum photonic computations, communications, and sensing. Most reconfigurable PIC devices achieve their functionalities via interference and are therefore highly sensitive to phase errors. Under static bias, carrier drift in ferroelectric waveguides induces continuous phase drift, creating a severe bottleneck for both PIC functionality and scalability. Here we propose achieving bias-stable and scalable ferroelectric PICs by exploiting the intrinsically low carrier drift of lithium tantalate (LT). Taking one of the PIC devices that is most sensitive to phase drift, the optical phased array (OPA), as an example, we designed and fabricated an integrated LT OPA that can keep the far-field main lobe 8 dB higher than side lobes for over 4 hours, representing at least a two-order-of-magnitude improvement over the state of the art. We demonstrated our device's capability in generating arbitrary spatiotemporal waveforms with a modulation frequency as low as 0.1 Hz, leading to practical applications in optical tweezers, trapped-ion quantum computers, adaptive optics for astronomy, AR, 3D printers, LiDAR, and free-space optical communications. Beyond OPA, our work establishes LT as a bias-stable, scalable, and high-speed PIC platform for large-scale classical and quantum photonic systems.

physics.optics↗
arXiv2025

MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices

Recently, video generation has witnessed rapid advancements, drawing increasing attention to image-to-video (I2V) synthesis on mobile devices. However, the substantial computational complexity and slow generation speed of diffusion models pose significant challenges for real-time, high-resolution video generation on resource-constrained mobile devices. In this work, we propose MobileI2V, a 270M lightweight diffusion model for real-time image-to-video generation on mobile devices. The core lies in: (1) We analyzed the performance of linear attention modules and softmax attention modules on mobile devices, and proposed a linear hybrid architecture denoiser that balances generation efficiency and quality. (2) We design a time-step distillation strategy that compresses the I2V sampling steps from more than 20 to only two without significant quality loss, resulting in a 10-fold increase in generation speed. (3) We apply mobile-specific attention optimizations that yield a 2-fold speed-up for attention operations during on-device inference. MobileI2V enables, for the first time, fast 720p image-to-video generation on mobile devices, with quality comparable to existing models. Under one-step conditions, the generation speed of each frame of 720p video is less than 100 ms. Our code is available at: https://github.com/hustvl/MobileI2V.

cs.CV↗
arXiv2025

Cyber Racing Coach: A Haptic Shared Control Framework for Teaching Advanced Driving Skills

This study introduces a haptic shared control framework designed to teach human drivers advanced driving skills. In this context, shared control refers to a driving mode where the human driver collaborates with an autonomous driving system to control the steering of a vehicle simultaneously. Advanced driving skills are those necessary to safely push the vehicle to its handling limits in high-performance driving such as racing and emergency obstacle avoidance. Previous research has demonstrated the performance and safety benefits of shared control schemes using both subjective and objective evaluations. However, these schemes have not been assessed for their impact on skill acquisition on complex and demanding tasks. Prior research on long-term skill acquisition either applies haptic shared control to simple tasks or employs other feedback methods like visual and auditory aids. To bridge this gap, this study creates a cyber racing coach framework based on the haptic shared control paradigm and evaluates its performance in helping human drivers acquire high-performance driving skills. The framework introduces (1) an autonomous driving system that is capable of cooperating with humans in a highly performant driving scenario; and (2) a haptic shared control mechanism along with a fading scheme to gradually reduce the steering assistance from autonomy based on the human driver's performance during training. Two benchmarks are considered: self-learning (no assistance) and full assistance during training. Results from a human subject study indicate that the proposed framework helps human drivers develop superior racing skills compared to the benchmarks, resulting in better performance and consistency.

cs.RO↗
arXiv2025

Spatial Envelope MPC: High Performance Driving without a Reference

This paper presents a novel envelope based model predictive control (MPC) framework designed to enable autonomous vehicles to handle high performance driving across a wide range of scenarios without a predefined reference. In high performance autonomous driving, safe operation at the vehicle's dynamic limits requires a real time planning and control framework capable of accounting for key vehicle dynamics and environmental constraints when following a predefined reference trajectory is suboptimal or even infeasible. State of the art planning and control frameworks, however, are predominantly reference based, which limits their performance in such situations. To address this gap, this work first introduces a computationally efficient vehicle dynamics model tailored for optimization based control and a continuously differentiable mathematical formulation that accurately captures the entire drivable envelope. This novel model and formulation allow for the direct integration of dynamic feasibility and safety constraints into a unified planning and control framework, thereby removing the necessity for predefined references. The challenge of envelope planning, which refers to maximally approximating the safe drivable area, is tackled by combining reinforcement learning with optimization techniques. The framework is validated through both simulations and real world experiments, demonstrating its high performance across a variety of tasks, including racing, emergency collision avoidance and off road navigation. These results highlight the framework's scalability and broad applicability across a diverse set of scenarios.

cs.RO↗
arXiv2025

Turbo-VAED: Fast and Stable Transfer of Video-VAEs to Mobile Devices

There is a growing demand for deploying large generative AI models on mobile devices. For recent popular video generative models, however, the Variational AutoEncoder (VAE) represents one of the major computational bottlenecks. Both large parameter sizes and mismatched kernels cause out-of-memory errors or extremely slow inference on mobile devices. To address this, we propose a low-cost solution that efficiently transfers widely used video VAEs to mobile devices. (1) We analyze redundancy in existing VAE architectures and get empirical design insights. By integrating 3D depthwise separable convolutions into our model, we significantly reduce the number of parameters. (2) We observe that the upsampling techniques in mainstream video VAEs are poorly suited to mobile hardware and form the main bottleneck. In response, we propose a decoupled 3D pixel shuffle scheme that slashes end-to-end delay. Building upon these, we develop a universal mobile-oriented VAE decoder, Turbo-VAED. (3) We propose an efficient VAE decoder training method. Since only the decoder is used during deployment, we distill it to Turbo-VAED instead of retraining the full VAE, enabling fast mobile adaptation with minimal performance loss. To our knowledge, our method enables real-time 720p video VAE decoding on mobile devices for the first time. This approach is widely applicable to most video VAEs. When integrated into four representative models, with training cost as low as $95, it accelerates original VAEs by up to 84.5x at 720p resolution on GPUs, uses as low as 17.5% of original parameter count, and retains 96.9% of the original reconstruction quality. Compared to mobile-optimized VAEs, Turbo-VAED achieves a 2.9x speedup in FPS and better reconstruction quality on the iPhone 16 Pro. The code and models will soon be available at https://github.com/hustvl/Turbo-VAED.

cs.CV↗
arXiv2025

Programmable electro-optic frequency comb empowers integrated parallel convolution processing

Integrated photonic convolution processors make optical neural networks (ONNs) a transformative solution for artificial intelligence applications such as machine vision. To enhance the parallelism, throughput, and energy efficiency of ONNs, wavelength multiplexing is widely applied. However, it often encounters the challenges of low compactness, limited scalability, and high weight reconstruction latency. Here, we proposed and demonstrated an integrated photonic processing unit with a parallel convolution computing speed of 1.62 trillion operations per second (TOPS) and a weight reconstruction speed exceeding 38 GHz. This processing unit simultaneously achieves, for the first time, multi-wavelength generation and weight mapping via a single programmable electro-optic (EO) frequency comb, featuring unprecedented compactness, device-footprint independent scalability, and near-unity optical power conversion efficiency (conversion efficiency from input optical power to output weighted comb lines). To demonstrate the reconfigurability and functionality of this processing unit, we implemented image edge detection and object classification based on EO combs obtained using the particle swarm algorithm and an EO comb neural network training framework, respectively. Our programmable EO comb-based processing framework establishes a new paradigm towards the development of low-latency monolithic photonic processors, promising real-time in-sensor learning for autonomous vehicles, intelligent robotics, and drones.

physics.optics↗
arXiv2025

MGPBD: A Multigrid Accelerated Global XPBD Solver

We introduce a novel Unsmoothed Aggregation (UA) Algebraic Multigrid (AMG) method combined with Preconditioned Conjugate Gradient (PCG) to overcome the limitations of Extended Position-Based Dynamics (XPBD) in high-resolution and high-stiffness simulations. While XPBD excels in simulating deformable objects due to its speed and simplicity, its nonlinear Gauss-Seidel (GS) solver often struggles with low-frequency errors, leading to instability and stalling issues, especially in high-resolution, high-stiffness simulations. Our multigrid approach addresses these issues efficiently by leveraging AMG. To reduce the computational overhead of traditional AMG, where prolongator construction can consume up to two-thirds of the runtime, we propose a lazy setup strategy that reuses prolongators across iterations based on matrix structure and physical significance. Furthermore, we introduce a simplified method for constructing near-kernel components by applying a few sweeps of iterative methods to the homogeneous equation, achieving convergence rates comparable to adaptive smoothed aggregation (adaptive-SA) at a lower computational cost. Experimental results demonstrate that our method significantly improves convergence rates and numerical stability, enabling efficient and stable high-resolution simulations of deformable objects.

cs.GR↗
arXiv2025

Single-mode InAs/GaAs quantum-dot DFB laser with oxidized aperture confined surface grating

InAs/GaAs quantum dot (QD) distributed feedback (DFB) lasers are promising candidates for next-generation photonic integrated circuits. We present a design that incorporates an oxidized aperture confined surface grating (OASG) structure, which reduces non-radiative recombination losses and surface optical losses sustained in device fabricated by conventionally fabrication methods including etching and regrowth. The OASG-DFB laser eliminates the need for ridge waveguide etching and avoids instability in sidewall grating coupling. Experimental results show stable single-mode operation, a maximum output power of 15.1 mW, a side-mode suppression ratio (SMSR) of 44 dB, and a narrow linewidth of 1.79 MHz. This approach simplifies fabrication, reduces costs, and enhances the scalability of GaAs-based QD DFB lasers for applications in optical communication and photonic integration.

physics.optics↗
arXiv2024

PsyDraw: A Multi-Agent Multimodal System for Mental Health Screening in Left-Behind Children

Left-behind children (LBCs), numbering over 66 million in China, face severe mental health challenges due to parental migration for work. Early screening and identification of at-risk LBCs is crucial, yet challenging due to the severe shortage of mental health professionals, especially in rural areas. While the House-Tree-Person (HTP) test shows higher child participation rates, its requirement for expert interpretation limits its application in resource-scarce regions. To address this challenge, we propose PsyDraw, a multi-agent system based on Multimodal Large Language Models that assists mental health professionals in analyzing HTP drawings. The system employs specialized agents for feature extraction and psychological interpretation, operating in two stages: comprehensive feature analysis and professional report generation. Evaluation of HTP drawings from 290 primary school students reveals that 71.03% of the analyzes achieved High Consistency with professional evaluations, 26.21% Moderate Consistency and only 2.41% Low Consistency. The system identified 31.03% of cases requiring professional attention, demonstrating its effectiveness as a preliminary screening tool. Currently deployed in pilot schools, \method shows promise in supporting mental health professionals, particularly in resource-limited areas, while maintaining high professional standards in psychological assessment.

cs.CL↗
arXiv2024

Broadband and Accurate Electric Tuning of On-Chip Efficient Nonlinear Parametric Conversion

On-chip nonlinear photonic conversion functions with wide and precise tunability as well as high conversion efficiency are highly desirable for a wide range of applications. Photonic crystal micro-ring resonators (PhCR) facilitate efficient nonlinear conversion and enable wavenumber-accurate selection of converted optical modes, but do not support post-fabrication reconfiguration of these operational modes. Coupled-ring resonators, on the other hand, allows post-fabrication reconfiguration but suffers from ambiguity in mode selectivity. We propose a novel segmented photonic crystal micro-ring resonator featuring half-circumference gratings that decouples the locking between the grating Bragg reflection peak and micro-ring resonance frequencies. By introducing complementary thermos-optical controllers that allow differential tuning between the grating reflection peak and the micro-ring resonance, the device supports electrically reconfigurable wavenumber-accurate optical mode selectivity, experimentally demonstrated as a voltage-tunable, power-efficient optical parametric oscillator. The device demonstrates electric tuning of signal and idler frequencies both in a per-FSR stepwise manner and in a gap-free continuous manner, achieving a broad optical frequency tuning range of > 5 THz and a conversion efficiency of >25%. The novel approach introduces unprecedented design flexibility as well as high and precise reconfigurability to integrated nonlinear photonics, providing a new pathway towards future high-performance on-chip nonlinear light sources.

physics.optics↗
arXiv2024

High power GaSb-based distributed feedback laser with laterally coupled dielectric gratings at 1.95{\mu}m

Traditional Distributed Feedback (DFB) or Distributed Bragg Reflector (DBR) lasers typically utilize buried gratings as frequency-selective optical feedback mechanisms. However, the fabrication of such gratings often necessitates regrowth processes, which can pose technical challenges for materials platforms such as GaAs and GaSb. Metal gratings were also used for GaSb lasers but they introduce additional absorption loss that limits device efficiency and output power. In this paper, we introduce a novel laterally coupled dielectric Bragg grating structure, which enables highly controllable, deterministic, and stable coupling between the grating and the optical mode. Our device demonstrates a continuous-wave output power of 47.02 mW at room temperature, exhibiting stable single-mode operation from 300-1000 mA and achieving a maximum side mode suppression ratio of 46.7 dB. These results underscore the innovative lateral coupled dielectric grating as a feasible and technologically superior approach for fabricating DFB and DBR lasers, which hold universal applicability across different material platforms and wavelength bands.

physics.optics↗
arXiv2024

Competitive Analysis of Online Path Selection: Impacts of Path Length, Topology, and System-Level Costs

Consider a communication network to which a sequence of self-interested users come and send requests for data transmission between nodes. This work studies the question of how to guide the path selection choices made by those online-arriving users and maximize the social welfare. Competitive analysis is the main technical tool. Specifically, the impacts of path length bounds and topology on the competitive ratio of the designed algorithm are analyzed theoretically and explored experimentally. We observe intricate and interesting relationships between the empirical performance and the studied network parameters, which shed some light on how to design the network. We also investigate the influence of system-level costs on the optimal algorithm design.

cs.DS↗
arXiv2024

Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework

Distributed stochastic gradient descent (SGD) has attracted considerable recent attention due to its potential for scaling computational resources, reducing training time, and helping protect user privacy in machine learning. However, the staggers and limited bandwidth may induce random computational/communication delays, thereby severely hindering the learning process. Therefore, how to accelerate asynchronous SGD by efficiently scheduling multiple workers is an important issue. In this paper, a unified framework is presented to analyze and optimize the convergence of asynchronous SGD based on stochastic delay differential equations (SDDEs) and the Poisson approximation of aggregated gradient arrivals. In particular, we present the run time and staleness of distributed SGD without a memorylessness assumption on the computation times. Given the learning rate, we reveal the relevant SDDE's damping coefficient and its delay statistics, as functions of the number of activated clients, staleness threshold, the eigenvalues of the Hessian matrix of the objective function, and the overall computational/communication delay. The formulated SDDE allows us to present both the distributed SGD's convergence condition and speed by calculating its characteristic roots, thereby optimizing the scheduling policies for asynchronous/event-triggered SGD. It is interestingly shown that increasing the number of activated workers does not necessarily accelerate distributed SGD due to staleness. Moreover, a small degree of staleness does not necessarily slow down the convergence, while a large degree of staleness will result in the divergence of distributed SGD. Numerical results demonstrate the potential of our SDDE framework, even in complex learning tasks with non-convex objective functions.

cs.LG↗
arXiv2024

High Spectral-Efficiency, Ultra-low MIMO SDM Transmission over a Field-Deployed Multi-Core OAM Fiber

Few-mode multi-core fiber (FM-MCF) based Space-Division Multiplexing (SDM) systems possess the potential to maximize the number of multiplexed spatial channels per fiber by harnessing both the space (fiber cores) and mode (optical mode per core) dimensions. However, to date, no SDM transmissions over field-deployed FM-MCFs in realistic outdoor settings have been reported, which contrasts with SDM schemes demonstrated using single-mode multi-core fibers (SM-MCFs) installed in practical fiber cable ducts. In this paper, we present the successful demonstration of bidirectional SDM transmission over a 5-km field-deployed seven ring-core fiber (7-RCF) with a cladding diameter of 178 ${\mu}$m, achieving a Spectral Efficiency (SE) of 2$\times$201.6 bit/s/Hz. This work establishes a new record for the highest SE attained in SDM demonstrations utilizing field-deployed fiber cables, achieving an approximate 10x increase compared to the SE of reported field-deployed optical fiber cable transmission systems. Notably, these results are realized through the utilization of small-scale modular 4$\times$4 multiple-input multiple-output (MIMO) processing with a time-domain equalization (TDE) tap number not exceeding 15, maintaining a complexity per unit capacity comparable to that of MIMO equalization in SDM demonstrations employing weakly coupled SM-MCF cables. These results underscore the significant potential for achieving heightened SE and expanding capacity per individual fiber using SDM techniques in practical applications.

cs.NI↗
arXiv2024

Scalable and Versatile Linear Computation with Minimalistic Photonic Matrix Processor

The advancement of artificial intelligence demands flexible multimodal data processing with high throughput and energy efficiency. Photonic integrated circuits (PIC) has demonstrated promising potentials in terms of low latency and low power consumption per operation for linear operations such as matrix-vector multiplication. However, the existing schemes face challenges in their scalability due to the use of photonic circuits that expand with the scale of the operants, despite efforts of exploiting the multiple optical parameter dimensions such as time, wavelength and spatial parallelism. They also lacked flexibility and efficiency in switching between different types of operations or tasks and adapting to multimodal data. In this article, we introduce an optical matrix processor (MP) with a minimalistic recursive structure for both multiplications and accumulations. The MP consists of an eletro-optic ring-modulator implemented as a thin-film lithium niobate PIC that allows flexible configurability and time-division multiplexed scheduling. The MP supports not only versatile linear operations including vector/matrix-vector multiplication and single/multi-kernel convolution but also ultrafast task switching and adaptability to data of different sizes, by simply adjusting the data baud rate relative to the ring delay without structural modifications. We demonstrate its capabilities in a optic-electronic convolutional neural network with a computing throughput up to 73.4 billion operations per second. The MP further supports high scalability through appropriate allocation of wavelength and space resources,extending computing parallelism to handle higher data volumes with higher energy efficiency. This novel scheme paves the way for a new class of photonic processorscapable of managing escalating data workloads with unprecedented flexibility, efficiency and scalability.

physics.optics↗