arXiv subjects

Hongyu Li

Publications and source records attributed to Hongyu Li.

At least 91 records · Page 5Linked to original sources

arXiv2025

LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding

Recent advancements in multimodal large language models (MLLMs) have shown promising results, yet existing approaches struggle to effectively handle both temporal and spatial localization simultaneously. This challenge stems from two key issues: first, incorporating spatial-temporal localization introduces a vast number of coordinate combinations, complicating the alignment of linguistic and visual coordinate representations; second, encoding fine-grained temporal and spatial information during video feature compression is inherently difficult. To address these issues, we propose LLaVA-ST, a MLLM for fine-grained spatial-temporal multimodal understanding. In LLaVA-ST, we propose Language-Aligned Positional Embedding, which embeds the textual coordinate special token into the visual space, simplifying the alignment of fine-grained spatial-temporal correspondences. Additionally, we design the Spatial-Temporal Packer, which decouples the feature compression of temporal and spatial resolutions into two distinct point-to-region attention processing streams. Furthermore, we propose ST-Align dataset with 4.3M training samples for fine-grained spatial-temporal multimodal understanding. With ST-align, we present a progressive training pipeline that aligns the visual and textual feature through sequential coarse-to-fine stages.Additionally, we introduce an ST-Align benchmark to evaluate spatial-temporal interleaved fine-grained understanding tasks, which include Spatial-Temporal Video Grounding (STVG) , Event Localization and Captioning (ELC) and Spatial Video Grounding (SVG). LLaVA-ST achieves outstanding performance on 11 benchmarks requiring fine-grained temporal, spatial, or spatial-temporal interleaving multimodal understanding. Our code, data and benchmark will be released at Our code, data and benchmark will be released at https://github.com/appletea233/LLaVA-ST .

cs.CV↗

arXiv2025

Channel Estimation for Beyond Diagonal RIS via Tensor Decomposition

This paper addresses the channel estimation problem for beyond diagonal reconfigurable intelligent surface (BD-RIS) from a tensor decomposition perspective. We first show that the received pilot signals can be arranged as a three-way tensor, allowing us to recast the cascaded channel estimation problem as a block Tucker decomposition problem that yields decoupled estimates for the involved channel matrices while offering a substantial performance gain over the conventional (matrix-based) least squares (LS) estimation method. More specifically, we develop two solutions to solve the problem. The first one is a closed-form solution that extracts the channel estimates via a block Tucker Kronecker factorization (BTKF), which boils down to solving a set of parallel rank-one matrix approximation problems. Exploiting such a low-rank property yields a noise rejection gain compared to the standard LS estimation scheme while allowing the two involved channels to be estimated separately. The second solution is based on a block Tucker alternating least squares (BTALS) algorithm that directly estimates the involved channel matrices using an iterative estimation procedure. We discuss the uniqueness and identifiability issues and their implications for training design. We also propose a tensor-based design of the BD-RIS training tensor for each algorithm that ensures unique decoupled channel estimates under trivial scaling ambiguities. Our numerical results shed light on the tradeoffs offered by BTKF and BTALS methods. Specifically, while the first enjoys fast and parallel extraction of the channel estimates in closed form, the second has a more flexible training design, allowing for a significantly reduced training overhead compared to the state-of-the-art LS method.

eess.SP↗

arXiv2025

KMT-2022-BLG-1818Lb,c: A Cold Super-Jupiter with a Saturn Sibling

We present the discovery and analysis of the sixth microlensing two-planet system, KMT-2022-BLG-1818Lb,c, detected by a follow-up program targeting high-magnification events. Both planets are subject to the well-known ''Close/Wide'' degeneracy, although for the first planet, which has a super-Jovian mass ratio of $q_2 \simeq 5\times 10^{-3}$ in both solutions, the Close topology, with a normalized separation of $s\simeq 0.70$, is clearly preferred by $Δχ^2=26$. However, contrary to all previous two-planet microlensing systems, the mass ratio for the second planet, $q_3$, is substantially (factor of $\sim 10$) different for the Close and Wide topologies of the first planet. While this degeneracy is resolved in the present case due to high-cadence follow-up observations, the appearance of this new degeneracy indicates the need for caution in the analysis of future two-planet systems. A Bayesian analysis suggests that the host is likely a K-dwarf star in the Galactic disk. The first planet is probably a super-Jupiter on a Jupiter-like orbit, while the second planet is a Saturn-class planet on either a Mercury-like or Saturn-like orbit.

astro-ph.EP↗

arXiv2025

Non-Reciprocal Beyond Diagonal RIS: Multiport Network Models and Performance Benefits in Full-Duplex Systems

Beyond diagonal reconfigurable intelligent surfaces (BD-RIS) is a new advance in RIS techniques that introduces reconfigurable inter-element connections to generate scattering matrices not limited to being diagonal. BD-RIS has been recently proposed and proven to have benefits in enhancing channel gain and enlarging coverage in wireless communications. Uniquely, BD-RIS enables reciprocal and non-reciprocal architectures characterized by symmetric and non-symmetric scattering matrices. However, the performance benefits and new use cases enabled by non-reciprocal BD-RIS for wireless systems remain unexplored. This work takes a first step toward closing this knowledge gap and studies the non-reciprocal BD-RIS in full-duplex systems and its performance benefits over reciprocal counterparts. We start by deriving a general RIS aided full-duplex system model using a multiport circuit theory, followed by a simplified channel model based on physically consistent assumptions. With the considered channel model, we investigate the effect of BD-RIS non-reciprocity and identify the theoretical conditions for reciprocal and non-reciprocal BD-RISs to simultaneously achieve the maximum received power of the signal of interest in the uplink and the downlink. Simulation results validate the theories and highlight the significant benefits offered by non-reciprocal BD-RIS in full-duplex systems. The significant gains are achieved because of the non-reciprocity principle which implies that if a wave hits the non-reciprocal BD-RIS from one direction, the surface behaves differently than if it hits from the opposite direction. This enables an uplink user and a downlink user at different locations to optimally communicate with the same full-duplex base station via a non-reciprocal BD-RIS, which would not be possible with reciprocal surfaces.

eess.SP↗

arXiv2025

ViTa-Zero: Zero-shot Visuotactile Object 6D Pose Estimation

Object 6D pose estimation is a critical challenge in robotics, particularly for manipulation tasks. While prior research combining visual and tactile (visuotactile) information has shown promise, these approaches often struggle with generalization due to the limited availability of visuotactile data. In this paper, we introduce ViTa-Zero, a zero-shot visuotactile pose estimation framework. Our key innovation lies in leveraging a visual model as its backbone and performing feasibility checking and test-time optimization based on physical constraints derived from tactile and proprioceptive observations. Specifically, we model the gripper-object interaction as a spring-mass system, where tactile sensors induce attractive forces, and proprioception generates repulsive forces. We validate our framework through experiments on a real-world robot setup, demonstrating its effectiveness across representative visual backbones and manipulation scenarios, including grasping, object picking, and bimanual handover. Compared to the visual models, our approach overcomes some drastic failure modes while tracking the in-hand object pose. In our experiments, our approach shows an average increase of 55% in AUC of ADD-S and 60% in ADD, along with an 80% lower position error compared to FoundationPose.

cs.RO↗

arXiv2025

Generalizability of local neural operator: example for elastodynamic problems

Local neural operator (LNO) conception has provided a feasible way for scientific computations. The LNO learns transient partial differential equations from random field samples, and then the pre-trained LNO solves practical problems on specific computational domains. For applications, we may ask: Are the training samples rich enough? To what extent can we trust the solutions obtained from pre-trained LNO models for unknown cases? The generalizability of LNO could answer these questions. Here, we propose to use two plain scalar features, the amplitude and wavenumber of the input functions, to indicate the richness of training samples and to evaluate the generalization error of pre-trained LNO. In elastodynamic practices, we find that isolated evolving wavenumber modes for Lamé-Navier equation caused the training dataset to lack mode diversity. By data supplementation and model fine-tuning targeting to the discovered lack modes, the pre-trained and fine-tuned LNO model solves Lamb problem correctly and efficiently. These results and the proposed generalization criteria provide a paradigm for LNO applications.

physics.comp-ph↗

arXiv2025

Virtual domain extension for imposing boundary conditions in flow simulation using pre-trained local neural operator

This paper builds up a virtual domain extension (VDE) framework for imposing boundary conditions (BCs) in flow simulation using pre-trained local neural operator (LNO). It creates extended virtual domains to the input function to compensate for the corrosion nature of computational domains during LNO inference, thus turns the implementation of BC into the determination of field values on the extended domain. Several strategies to calculate the field values are proposed and validated in solving numerical examples, including padding operation, direct imposition, pressure symmetry, and optimization by backpropagation, and compared with boundary imposition in traditional solvers. It is found that the large time interval of LNO induces a relatively wide near-boundary domain to be processed, thus imposing BC on only a few nodes near the boundary following the immersed boundary conception in traditional solvers can hardly achieve high accuracy. With appropriate values assigned on the extended virtual domains, VDE can accurately impose BCs and lead to reasonable flow field predictions. This work provides a guidance for imposing BCs reliably in LNO prediction, which could facilitate the reuse of pre-trained LNO in more applications.

physics.flu-dyn↗

arXiv2025

Beyond Diagonal Reconfigurable Intelligent Surfaces in Wideband OFDM Communications: Circuit-Based Modeling and Optimization

This work investigates the modeling and optimization of beyond diagonal reconfigurable intelligent surface (BD-RIS), which generalizes conventional RIS with diagonal phase shift matrices and provides additional flexibility for manipulating wireless channels, in wideband communication systems. Specifically, we start from the signal modeling of the BD-RIS-aided orthogonal frequency division multiplexing (OFDM) system, which bridges the time-domain and frequency-domain channels, and explicitly shows the frequency dependence of the BD-RIS response. We next characterize the frequency dependence of the BD-RIS response based on circuit models. Benefiting from the admittance parameter analysis, we model individually each tunable admittance component of BD-RIS and derive an approximated linear expression with respect to the frequency of the transmit signals. With the proposed signal model for the BD-RIS-aided OFDM system and the frequency-dependent BD-RIS model, we propose algorithms to optimize the BD-RIS and the power allocation at the transmitter to maximize the average rate for a BD-RIS-aided OFDM system. Finally, simulation results show that BD-RIS outperforms conventional RIS in the OFDM system. More importantly, the impact of wideband modeling of BD-RIS on the system performance becomes more significant as the circuit complexity of BD-RIS architectures increases.

eess.SP↗

arXiv2024

Unleashing the Temporal-Spatial Reasoning Capacity of GPT for Training-Free Audio and Language Referenced Video Object Segmentation

In this paper, we propose an Audio-Language-Referenced SAM 2 (AL-Ref-SAM 2) pipeline to explore the training-free paradigm for audio and language-referenced video object segmentation, namely AVS and RVOS tasks. The intuitive solution leverages GroundingDINO to identify the target object from a single frame and SAM 2 to segment the identified object throughout the video, which is less robust to spatiotemporal variations due to a lack of video context exploration. Thus, in our AL-Ref-SAM 2 pipeline, we propose a novel GPT-assisted Pivot Selection (GPT-PS) module to instruct GPT-4 to perform two-step temporal-spatial reasoning for sequentially selecting pivot frames and pivot boxes, thereby providing SAM 2 with a high-quality initial object prompt. Within GPT-PS, two task-specific Chain-of-Thought prompts are designed to unleash GPT's temporal-spatial reasoning capacity by guiding GPT to make selections based on a comprehensive understanding of video and reference information. Furthermore, we propose a Language-Binded Reference Unification (LBRU) module to convert audio signals into language-formatted references, thereby unifying the formats of AVS and RVOS tasks in the same pipeline. Extensive experiments on both tasks show that our training-free AL-Ref-SAM 2 pipeline achieves performances comparable to or even better than fully-supervised fine-tuning methods. The code is available at: https://github.com/appletea233/AL-Ref-SAM2.

cs.CV↗

arXiv2024

Fermiology with nodal structures in nonsymmorphic superconductor LaNiGa$_2$: A de Haas-van Alphen study

Topological metals possess various types of symmetry-protected degenerate band crossings. When a topological metal becomes superconducting, the low-energy electronic excitations stemming from the band crossings located close to the Fermi level may contribute to highly unusual pairing symmetry and superconducting states. In this work, we study the electronic band structure of the time-reversal symmetry breaking superconductor LaNiGa$_2$ by means of quantum oscillation measurements. A comprehensive investigation combining angle-resolved high-field de Haas-van Alphen (dHvA) spectroscopy and first-principles calculations reveals the fermiology of LaNiGa$_2$ and verifies its nonsymmorphic $Cmcm$ lattice symmetry, which promises nodal band crossings pinned at the Fermi level with fourfold degeneracies. Moreover, such nodal structures, proposed to play a crucial role giving rise to the interorbital triplet pairing, are indeed captured by our dHvA analysis. Our results identify LaNiGa$_2$ as a prototypical topological crystalline superconductor and highlight the putative contribution of low-energy nodal quasiparticles to unconventional superconducting pairing.

cond-mat.supr-con↗

arXiv2024

A Decoupled Channel Estimation Method for Beyond Diagonal RIS

Beyond diagonal reconfigurable intelligent surface (BD-RIS) is a new architecture for RIS where elements are interconnected to provide more wave manipulation flexibility than traditional single connected RIS, enhancing data rate and coverage. However, channel estimation for BD-RIS is challenging due to the more complex multiple-connection structure involving their scattering elements. To address this issue, this paper proposes a decoupled channel estimation method for BD-RIS that yields separate estimates of the involved channels to enhance the accuracy of the overall combined channel by capitalizing on its Kronecker structure. Starting from a least squares estimate of the combined channel and by properly reshaping the resulting filtered signal, the proposed algorithm resorts to a Khatri-Rao Factorization (KRF) method that teases out the individual channels based on simple rank-one matrix approximation steps. Numerical results show that the proposed decoupled channel estimation yields more accurate channel estimates than the classical least squares scheme.

eess.SP↗

arXiv2024

Abnormally enhanced Hall Lorenz number in the magnetic Weyl semimetal NdAlSi

In Landau's celebrated Fermi liquid theory, electrons in a metal obey the Wiedemann--Franz law at the lowest temperatures. This law states that electron heat and charge transport are linked by a constant $L_0$, i.e., the Sommerfeld value of the Lorenz number ($L$). Such relation can be violated at elevated temperatures where the abundant inelastic scattering leads to a reduction of the Lorenz number ($L < L_0$). Here, we report a rare case of remarkably enhanced Lorenz number ($L > L_0$) discovered in the magnetic topological semimetal NdAlSi. Measurements of the transverse electrical and thermal transport coefficients reveal that the Hall Lorenz number $L_{xy}$ in NdAlSi starts to deviate from the canonical value far above its magnetic ordering temperature. Moreover, $L_{xy}$ displays strong nonmonotonic temperature and field dependence, reaching its maximum value close to 2$L_0$ in an intermediate parameter range. Further analysis excludes charge-neutral excitations as the origin of enhanced $L_{xy}$. Alternatively, we attribute it to the Kondo-type elastic scattering off localized 4$f$ electrons, which creates a peculiar energy distribution of the quasiparticle relaxation time. Our results provide insights into the perplexing transport phenomena caused by the interplay between charge and spin degrees of freedom.

cond-mat.str-el↗

arXiv2024

Two-dimensional superconductivity and anomalous vortex dissipation in newly-discovered transition metal dichalcogenide-based superlattices

Properties of layered superconductors can vary drastically when thinned down from bulk to monolayer, owing to the reduced dimensionality and weakened interlayer coupling. In transition metal dichalcogenides (TMDs), the inherent symmetry breaking effect in atomically thin crystals prompts novel states of matter, such as Ising superconductivity with an extraordinary in-plane upper critical field. Here, we demonstrate that two-dimensional (2D) superconductivity resembling those in atomic layers but with more fascinating behaviours can be realized in the bulk crystals of two new TMD-based superconductors Ba0.75ClTaS2 and Ba0.75ClTaSe2. They comprise an alternating stack of H-type TMD layers and Ba-Cl layers. In both materials, intrinsic 2D superconductivity develops below a Berezinskii-Kosterlitz-Thouless transition. The upper critical field along ab plane exceeds the Pauli limit (Hp); in particular, Ba0.75ClTaSe2 exhibits an extremely high in plane Hc2 (14Hp) and a colossal superconducting anisotropy of 150. Moreover, the temperature-field phase diagram of Ba0.75ClTaSe2 under an in-plane magnetic field contains a large phase regime of vortex dissipation, which can be ascribed to the Josephson vortex motion, signifying an unprecedentedly strong fluctuation effect in TMD-based superconductors. Our results provide a new path towards the establishment of 2D superconductivity and novel exotic quantum phases in bulk crystals of TMD-based superconductors.

cond-mat.supr-con↗

arXiv2024

TabSec: A Collaborative Framework for Novel Insider Threat Detection

In the era of the Internet of Things (IoT) and data sharing, users frequently upload their personal information to enterprise databases to enjoy enhanced service experiences provided by various online services. However, the widespread presence of system vulnerabilities, remote network intrusions, and insider threats significantly increases the exposure of private enterprise data on the internet. If such data is stolen or leaked by attackers, it can result in severe asset losses and business operation disruptions. To address these challenges, this paper proposes a novel threat detection framework, TabITD. This framework integrates Intrusion Detection Systems (IDS) with User and Entity Behavior Analytics (UEBA) strategies to form a collaborative detection system that bridges the gaps in existing systems' capabilities. It effectively addresses the blurred boundaries between external and insider threats caused by the diversification of attack methods, thereby enhancing the model's learning ability and overall detection performance. Moreover, the proposed method leverages the TabNet architecture, which employs a sparse attention feature selection mechanism that allows TabNet to select the most relevant features at each decision step, thereby improving the detection of rare-class attacks. We evaluated our proposed solution on two different datasets, achieving average accuracies of 96.71% and 97.25%, respectively. The results demonstrate that this approach can effectively detect malicious behaviors such as masquerade attacks and external threats, significantly enhancing network security defenses and the efficiency of network attack detection.

cs.CR↗

arXiv2024

Beyond Diagonal RIS: Passive Maximum Ratio Transmission and Interference Nulling Enabler

Beyond diagonal reconfigurable intelligent surfaces (BD-RIS) generalizes and goes beyond conventional diagonal reconfigurable intelligent surfaces (D-RIS) by interconnecting elements to generate beyond diagonal scattering matrices, which significantly strengthen the wireless channels. In this work, we use BD-RIS for passive multiuser beamforming in multiuser multiple-input-single-output (MU-MISO) systems. Specifically, we design the scattering matrix of BD-RIS to either maximize the sum received signal power at the users following maximum ratio transmission (MRT), or to nullify the interference at the users following zero forcing (ZF). Furthermore, we investigate uniform/optimized power allocation and ZF precoding at the base station (BS). Numerical results show that BD-RIS improves the interference nulling capability and sum rate with fewer reflecting elements (REs) compared to D-RIS. In addition, at moderate to high signal to noise ratios (SNRs), passive interference nulling reduces the complexity at the BS by relaxing the need for precoding or water-filling power allocation design. Furthermore, the passive MRT with ZF precoding achieves a tight sum rate performance to the joint design considering MU-MISO scenarios with many REs while maintaining low computational complexity and simplifying the channel estimation.

eess.SP↗

arXiv2024

ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer

Obstacle detection and tracking represent a critical component in robot autonomous navigation. In this paper, we propose ODTFormer, a Transformer-based model to address both obstacle detection and tracking problems. For the detection task, our approach leverages deformable attention to construct a 3D cost volume, which is decoded progressively in the form of voxel occupancy grids. We further track the obstacles by matching the voxels between consecutive frames. The entire model can be optimized in an end-to-end manner. Through extensive experiments on DrivingStereo and KITTI benchmarks, our model achieves state-of-the-art performance in the obstacle detection task. We also report comparable accuracy to state-of-the-art obstacle tracking models while requiring only a fraction of their computation cost, typically ten-fold to twenty-fold less. The code and model weights will be publicly released.

cs.RO↗

arXiv2024

Dynamic Prompting of Frozen Text-to-Image Diffusion Models for Panoptic Narrative Grounding

Panoptic narrative grounding (PNG), whose core target is fine-grained image-text alignment, requires a panoptic segmentation of referred objects given a narrative caption. Previous discriminative methods achieve only weak or coarse-grained alignment by panoptic segmentation pretraining or CLIP model adaptation. Given the recent progress of text-to-image Diffusion models, several works have shown their capability to achieve fine-grained image-text alignment through cross-attention maps and improved general segmentation performance. However, the direct use of phrase features as static prompts to apply frozen Diffusion models to the PNG task still suffers from a large task gap and insufficient vision-language interaction, yielding inferior performance. Therefore, we propose an Extractive-Injective Phrase Adapter (EIPA) bypass within the Diffusion UNet to dynamically update phrase prompts with image features and inject the multimodal cues back, which leverages the fine-grained image-text alignment capability of Diffusion models more sufficiently. In addition, we also design a Multi-Level Mutual Aggregation (MLMA) module to reciprocally fuse multi-level image and phrase features for segmentation refinement. Extensive experiments on the PNG benchmark show that our method achieves new state-of-the-art performance.

cs.CV↗

arXiv2024

A Dual-Function Radar-Communication System Empowered by Beyond Diagonal Reconfigurable Intelligent Surface

This work focuses on the use of reconfigurable intelligent surface (RIS) in dual-function radar-communication (DFRC) systems to improve communication capacity and sensing precision, and enhance coverage for both functions. In contrast to most of the existing RIS aided DFRC works where the RIS is modeled as a diagonal phase shift matrix and can only reflect signals to half space, we propose a novel beyond diagonal RIS (BD-RIS) aided DFRC system. Specifically, the proposed BD-RIS supports the hybrid reflecting and transmitting mode, and is compatible with flexible architectures, enabling the system to realize full-space coverage and to achieve enhanced performance. To achieve the expected benefits, we jointly optimize the transmit waveform, the BD-RIS matrices, and sensing receive filters, by maximizing the minimum signal-to-clutter-plus-noise ratio for fair target detection, subject to the constraints of the communication quality of service, different BD-RIS architectures and power budget. To solve the non-convex and non-smooth max-min problem, a general solution based on the alternating direction method of multipliers is provided. Numerical simulations validate the efficacy of the proposed algorithm and show the superiority of the BD-RIS aided DFRC system in terms of both communication and sensing compared to conventional RIS aided DFRC.

eess.SP↗