arXiv Science⌕ Search

arXiv subjects

Hao Shi

Publications and source records attributed to Hao Shi.

At least 19 recordsLinked to original sources
arXiv2026

The Flexoelectric Twist: Persistence of Heavy Fermion Picture in Small-Angle Twisted Bilayer Graphene

Twisted bilayer graphene (TBG) near the first magic angle ($θ\approx1.05^{\circ}$) hosts flat bands that provide a fertile ground for exploring strongly correlated and topological phases. At smaller twist angles ($θ<1^{\circ}$), attice relaxation profoundly alters the moiré landscape. The extent to which moiré lattice reconstruction at different length scales affects the electronic structure through electro-mechanical coupling remains unresolved. Here, using scanning tunneling microscopy on small-angle TBG devices ($θ\approx0.3-0.7^{\circ}$), we observe a pronounced $dI/dV$ peak at the AA site of the moiré lattice on top of a spectrum that is asymmetric about the charge neutrality point, lattice reconstruction over moiré lattice length scales, as well as a dilation of the atomic lattice at the AA sites. We demonstrate at a phenomenological level that the inclusion of moiré lattice reconstruction supports compact localized heavy fermion $f-$ states localized at AA sites. The additional inclusion of atomic lattice dilation enerates a strong flexo-electric response that explicitly breaks particle-hole symmetry, pushing compact, localized $f$-states at the AA sites down in energy. Consequently, these heavy fermion states energetically coexist with itinerant $c$-electrons at the AB/BA sites. Our spatially resolved spectroscopic measurements quantitatively visualize this coexistence, confirming the persistence of the heavy fermion model in the small-angle regime $θ<1^{\circ}$. Our findings provide microscopic insights into how flexo-electricity affects the low-energy electronic structure of small-angle TBG. More broadly, our study establishes small-angle TBG as a new class of moiré materials to explore the interacting Kondo lattice problem with high tunability of the $f-$occupation and $c-$density through twist-angle and electrostatic back gates, respectively.

cond-mat.mes-hall↗
arXiv2026

VideoLVR: Latent Visual Reasoning over Grounded Video Evidence

Chain-of-thought reasoning has substantially advanced video understanding in multimodal large language models (MLLMs). However, language-based reasoning traces do not provide a reliable mechanism for models to revisit relevant visual evidence, leading to visual forgetting and impairing their ability to reason over video content. Existing approaches maintain visual grounding during reasoning in two main ways: they either refer to evidence using textual coordinates, which specify symbolic locations rather than the visual content itself, or reinsert cropped regions or frames into the multimodal context, preserving visual details at the cost of substantial context expansion. To address these limitations, we introduce VideoLVR, which constructs latent visual evidence from local regions at relevant video moments, thereby providing visual support for intermediate reasoning steps. To supervise latent visual evidence generation, we introduce a visual token prediction objective that formulates visual token reconstruction as prediction over a vocabulary derived from the input video, thereby unifying visual evidence reconstruction and text generation within the same token prediction framework. This shared prediction framework enables MLLMs to reason over interleaved text tokens and latent visual states while integrating evidence from different moments. We further extend VideoLVR to reinforcement learning by treating sampled visual tokens as actions alongside text tokens and jointly optimizing the policy using final-answer rewards. By efficiently revisiting task-relevant visual information during reasoning, VideoLVR improves video understanding without substantially expanding the multimodal context. Experiments on multiple video benchmarks demonstrate its effectiveness, especially on tasks requiring fine-grained visual evidence.

cs.CV↗
arXiv2026

OACap: Caption-Guided Active Observation for Detailed Video Captioning

Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global description. We propose Observation-Adaptive Video Captioner (OACap), a caption-centered agent that turns unsupported or missing information into targeted observation requests, uses spatiotemporal tools to acquire evidence, and iteratively verifies and revises the caption. OACap captions consistently improve performance across general video-understanding benchmarks using a fixed text-only reader. To better diagnose the information they preserve, we introduce Factual-Semantic Captioning Bench (FSCap-Bench). Its Core-Support Factuality Score (CSFS) combines claim precision with importance-aware fact coverage and responds as expected to four controlled perturbation types on 93.0-98.3% of valid caption pairs. Its Cross-Modal Semantic Round-Trip Score (CSRS) measures semantic preservation through a video-caption-video-caption round trip by matching atomic facts from source and recovered captions. Together, they connect downstream textual-memory utility with factual and semantic preservation.

cs.CV↗
arXiv2026

Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) have made substantial progress in egocentric video understanding, but their ability to reason cooperatively from multiple embodied viewpoints remains largely unexplored. We study this problem through multi-robot cooperative dynamic spatial reasoning, where a model must answer spatial, temporal, visibility, and coordination questions by integrating synchronized egocentric videos from a team of moving robots. To support this setting, we introduce CoopSR, the first benchmark for this task, together with EgoTeam, a multi-robot egocentric QA dataset. EgoTeam contains 114,227 QA pairs spanning 19 question types, four difficulty tiers, and three team sizes in Habitat and iGibson, along with a real-world test set of around 2,326 QAs collected using two quadruped robots. We further propose SG-CoR (Spectral and Geometry-Aware Cooperative Reasoner), an MLLM framework for fine-grained cooperative spatial reasoning. SG-CoR combines dynamics-aware multi-robot frame sampling, spectral- and geometric-guided view fusion, and geometry- and motion-informed prompt distillation, enabling the model to benefit from privileged robot-pose supervision during training while requiring only egocentric videos at test time. Across 22 MLLM baselines, SG-CoR consistently improves cooperative reasoning, outperforming the strongest fine-tuned baseline by +3.87% on Habitat and +7.12% on iGibson. It also shows stronger generalization to unseen team sizes and real-world robot tests. Code can be found at https://github.com/KPeng9510/seeing-together.git.

cs.CV↗
arXiv2026

RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design

Robotic manipulation policies have made rapid progress in recent years, yet most existing approaches give limited consideration to memory capabilities. Consequently, they struggle to solve tasks that require reasoning over historical observations and maintaining task-relevant information over time, which are common requirements in real-world manipulation scenarios. Although several memory-aware policies have been proposed, systematic evaluation of memory-dependent manipulation remains underexplored, and the relationship between architectural design choices and memory performance is still not well understood. To address this gap, we introduce RMBench, a simulation benchmark comprising 9 manipulation tasks that span multiple levels of memory complexity, enabling systematic evaluation of policy memory capabilities. We further propose Mem-0, a modular manipulation policy with explicit memory components designed to support controlled ablation studies. Through extensive simulation and real-world experiments, we identify memory-related limitations in existing policies and provide empirical insights into how architectural design choices influence memory performance. The website is available at https://rmbench.github.io/.

cs.RO↗
arXiv2026

ZoomCap: Evidence-Routed Visual Zooming for Recoverable Visual Information Transfer

Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss small objects, text, and spatial relations or promote uncertain observations into confident claims. We formulate fine-grained captioning as recoverable visual information transfer, in which a teacher acquires missing evidence under a finite observation budget and converts it into reusable supervision. ZoomCap uses a draft caption to route conditional knowledge retrieval, OCR-sensitive reading, coverage-gap recovery, and adaptive visual zooming, allocating deeper crops only to the most complex eligible region, deriving spatial relations from grounded geometry, and fusing non-duplicate evidence. We also introduce FactCycle, which scores caption-derived atomic claims on two blind anchors--source image (truthfulness) and reconstruction (realizability)--separating fabrication from generator loss claim by claim; human and multi-judge comparisons complement this fact-level view. Against Qwen3.5-397B-A17B captions, ZoomCap achieves an 81.9% human tie-adjusted preference score with a teacher over 10x smaller, and FactCycle counts 52.4 transferred claims per image vs. 31.5 for ScaleCap. The evidence also transfers: in continual pre-training, ZoomCap-450K improves the 11-benchmark average by 1.31-2.38 points over the strongest competing corpus and Ling-Tiny-VL-v3 gains 1.4 points on a production document/OCR monitor suite (1.5 on perception) at 60B tokens; in medical pre-training, 2.64M ZoomCap-captioned images improve five VQA benchmarks by 3.04 points (vs. 1.76 for same-image Gemini-3.1-Pro captions).

cs.CV↗
arXiv2026

Sharp Cramér-type moderate deviation bounds for the self-normalized maximum of partial sums

Let $X_1,X_2,\ldots$ be i.i.d. random variables with zero mean and variance $σ^2\in(0,\infty)$. In this paper, we establish a sharp Cramér-type moderate deviation for the self-normalized maximum of partial sums under a finite $p$-th absolute moment, $2<p\le3$. In particular, we show that $\mathbb{P}(M_n\ge xV_n)=2(1-Φ(x))\{1+O(1)β_{n,p}(1+x)^p\}$ for $0\le x\leβ_{n,p}^{-1/p}$, where $M_n=\max_{1\le k\le n}\sum_{j=1}^kX_j$, $V_n=(\sum_{j=1}^nX_j^2)^{1/2}$, $β_{n,p}=\mathbb{E}|X_1|^p/(n^{(p-2)/2}σ^p)$, and $Φ$ is the standard normal distribution function. This proves a conjecture of Shao2010. More precisely, we bound the relative error by the same truncated-moment quantity as for self-normalized sums. The proof compares the maximum with the absolute sum while preserving their common denominator, through a reflected-walk representation and Fourier estimates with distribution-adapted smoothing.

math.PR↗
arXiv2026

Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions

Optical flow estimation is a fundamental task in dynamic scene understanding and computer vision. While recent deep models achieve impressive accuracy on clean benchmarks, their robustness to common visual and temporal degradations remains insufficiently explored. To address this gap, we establish a real-video-based spatiotemporal corruption benchmark suite that complements rendered and mixed-source robustness benchmarks, systematically evaluating two complementary aspects of visual perception: domain robustness and motion-distribution robustness. We construct two targeted benchmarks: KITTI Flow Corruptions (KITTI-FC) for dynamic outdoor scenes and GoPro Flow Corruptions (GoPro-FC) for diverse first-person-view complex motions. Both benchmarks use captured source data and apply 24 controlled, predominantly algorithmic corruptions encompassing digital, illumination, weather, noise, blur, and video artifacts. To quantify robustness, we use two complementary axes: the GT-free output-stability error Rs and the GT-based accuracy degradation Rd, together with their clean-error-normalized variants. We systematically evaluate 33 model variants from 17 representative optical flow methods. Our results show that, under the evaluated corruption schedules, models exhibit greater vulnerability to local structural information loss (e.g., JPEG compression and spatter) than to global perceptual distortions (e.g., fog and saturation). We further find that in-domain fine-tuning does not guarantee relative robustness and that no single current method dominates across all tested source and motion settings. Our work provides a controlled diagnostic framework for assessing optical flow robustness and developing reliable perception systems in complex environments. Code and benchmark resources are publicly available at https://github.com/ZhonghuaYi/optical_flow_robustness_benchmark.

eess.IV↗
arXiv2026

A robust digital receiver architecture for continuous-variable quantum key distribution

Continuous-variable quantum key distribution utilizes advanced digital signal processing techniques to mitigate channel impairments, driving the progress of high-performance quantum secure communications. Specifically, digital heterodyne receivers are commonly implemented using down-conversion algorithms to detect both components simultaneously where the signal is modulated onto an intermediate frequency to avoid low-frequency noise and bypass stringent optical phase-locking requirements. However, a rigorous analysis of the quantum security implications of the digital down-conversion algorithm remains absent. In this paper, we investigate the algorithm process from the perspective of quantum operator evolution, revealing that the frequency folding effect inherent in down-conversion inevitably mixes the image-frequency information into the baseband signal. To address this issue, we propose a novel operator reconstruction detection scheme based on dual-path digital heterodyne receiver. By performing a non-equal weighted combination of the output operators along two optical paths in the digital domain, this scheme perfectly cancels out the image-frequency operator physically. The robust receiver architecture preserves the baseband noise-suppression advantages, inherently achieves perfect mode balancing for asymmetric optical detection losses, and fundamentally closes the image-frequency crosstalk. Practical experiments have validated the feasibility of the proposed architecture and its effective suppression of image-frequency noise, achieving a performance of 11.4 Mbps @ 42 km. This work lays a solid theoretical and algorithmic foundation for future high-performance, highly secure digital continuous-variable quantum key distribution systems.

quant-ph↗
arXiv2026

Can LLM Agents Automate Reinforcement Learning for Text-to-Speech?

Although reinforcement learning (RL) post-training repairs the localized segmental errors of zero-shot text-to-speech (TTS), arriving at a working recipe still relies on tedious manual tuning, and whether LLM agents can take over this research pipeline is unclear. We investigate this question with AgenticTTS-Forge, a collaborative workflow that structures human guidance and agentic execution around a shared workspace, applied to CosyVoice2-0.5B. To measure what the agent automates, we audit its trajectory stage by stage against the published recipe. To measure what it exploits, we score its policies with held-out observers hidden from the agent. Our results show that the agent recovers an underspecified recipe, improves it, and, when gains stall, surveys the literature unprompted and pivots from the LM carrier to the flow carrier, halving Bad cases. However, its autonomy exposes three traps across the data, proxy, and algorithm axes: the held-out set leaks through a channel the contract never reads, a self-shaped reward inflates the proxy where it is scored, and separately tuned policies do not compose additively. These findings show that the binding constraint is measurement rather than reasoning, and can inform the design of harnesses whose contracts read every channel the agent does.

cs.SD↗
arXiv2026

Beyond Acoustic Prefixes: Persistent Access to Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognition

Large language models (LLMs) provide strong linguistic priors for serialized output training (SOT), yet LLM-based multi-talker ASR degrades substantially as the number of overlapping talkers increases. Conventional systems expose acoustic evidence primarily through an initial projected mixture prefix, requiring the decoder to preserve and recover talker-relevant information indirectly throughout autoregressive generation. We first examine whether this limitation can be resolved by enriching the static prefix using discrete connectionist temporal classification (CTC) tokens, hybrid token--acoustic prompts, and continuous talker-specific representations. The results suggest that acoustic content alone does not fully address the conditioning bottleneck. We therefore extend onset-based serialization from the output target to the acoustic-conditioning pathway and introduce persistent decoder-side access to SOT-aligned serialized acoustic memory. Talker-specific representations are organized in utterance-onset order and retained as external acoustic memory, while the conventional mixture prefix provides a complementary global-conditioning path. LLM layers query this memory throughout generation through gated residual cross-attention. We further introduce a second adaptation stage that jointly applies low-rank updates to the acoustic-retrieval pathway and selected LLM self-attention projections. Experiments on LibriMix show consistent improvements over static-prefix prompting. These results indicate that effective LLM-based multi-talker ASR depends not only on providing richer acoustic representations, but also on maintaining persistent access to acoustic evidence structured according to the serialized output.

cs.SD↗
arXiv2026

What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling

World action models (WAMs) predict the future alongside actions during training. Due to the heavy computation cost of video denoising, whether the future must still be generated during inference is disputed: Explicit WAMs denoise it into clean frames along with every action chunk, whereas Latent WAMs discard it entirely for acceleration. We find that latent WAMs, despite matching explicit ones on in-distribution tasks, fail to retain the generalization benefits that originally motivated WAMs. To demonstrate this, we evaluate generalization along three axes: environmental perturbation, data efficiency, and task generalization. Controlled comparisons with a matched backbone, training data, and budget reveal consistent degradation across all three axes when the action expert no longer conditions on future representations. Further analysis shows that the gap arises almost entirely from the first denoising step: the benefit comes from preparing the future, not generating it. We therefore propose Simple-WAM, which simplifies future modeling into a single forward pass of fully noised video tokens and adapts the training-time noise schedule to this inference behavior. Across simulation and real-world tasks, Simple-WAM achieves the best of both worlds, leading explicit WAMs in generalization performance with efficiency comparable to Latent WAMs. Project Page: https://zrporz.github.io/Simple-WAM-Web/

cs.CV↗
arXiv2026

Ruby-ASR: Evidence-Preserving Supervision for Joint Orthographic and Lexical-Reading Recognition

Conventional Japanese automatic speech recognition (ASR) is supervised by an orthographic transcript, although the same written form can correspond to different lexical readings realized in speech. Such utterances receive an identical target, so their reading distinction is absent from the supervision interface and cannot be recovered reliably by post-hoc text-only grapheme-to-phoneme conversion. We present Ruby-ASR, which refines the conventional target into a span-bound orthographic--lexical-reading sequence. Unlike separate full-sentence orthographic and phonological outputs, the ruby representation locally binds each written span to its realized reading and permits deterministic recovery of both views. We instantiate the target under subtitle-style and verbatim-style transcription conventions using a Qwen3-ASR backbone; a mora-level CTC objective provides auxiliary monotonic reading supervision. The experimental results across five Japanese benchmarks show that refining the recognition target can improve lexical-reading recovery without sacrificing readable orthographic transcription. We release the checkpoints and inference code.

cs.CL↗
arXiv2026

Cube-Splat: High-Fidelity 360° Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization

Recent progress in 3D Gaussian Splatting (3DGS) has enabled dense visual SLAM with pinhole cameras, yet most pipelines are not designed for panoramic imagery. We present Cube-Splat, the first panoramic GS-SLAM framework that factorizes each 360° frame into a cubemap of four fixed-orientation virtual pinhole views sharing a single optical center. By designating the front face as the primary pose state, we accumulate gradients from all faces via an adjoint mapping, thereby enabling multi-face observations to coherently update a single state while strictly preserving cross-view geometric consistency. Concurrently, our mapping module densifies and optimizes anisotropic Gaussians using aggregated cubemap rays for high-fidelity, dense reconstruction. Furthermore, to rigorously evaluate panoramic SLAM under diverse and challenging conditions, we introduce SynPano, a highly scalable, photorealistic synthetic dataset featuring parameterized complex trajectories and multi-modal ground truth. Extensive evaluations on two public benchmarks (PALVIO and OmniBlender) and our SynPano dataset, collectively encompassing both indoor and outdoor scenes, demonstrate that Cube-Splat achieves state-of-the-art (SOTA) performance in tracking accuracy and reconstruction fidelity. Both the source code and the SynPano dataset are available at https://github.com/guoxf304/CubeSplat.

cs.CV↗
arXiv2026

Spheriverse: 3D Scene Understanding from Spherical Observations in the Wild

Spherical observations provide global visual context for 3D scene understanding. However, visual information is encoded in an angular domain, whereas the physical world is represented in Cartesian coordinates. This cross-space representation gap complicates geometric correspondence and semantic evidence aggregation. To delve into this challenge, we introduce Spheriverse, comprising 64,400 temporally aligned spherical image-LiDAR pairs organized into 644 sequences. The dataset spans diverse scenes, illumination, and weather conditions, with fine-grained semantic classes. We further establish benchmarks for semantic occupancy prediction, semantic mapping, and 3D object detection, evaluating 30+ methods through overall and scene-wise comparisons. For dense prediction, we propose SphereOcc, an occupancy framework that couples spherical geometry modeling with semantic evidence retrieval. Cartesian-Spherical Representation Remodeling (CSRR) incorporates spherical range-azimuth geometry into Cartesian voxel features through region-wise modulation. Spherical Evidence Re-querying (SER) then conditions queries on voxel content and range-height-azimuth geometry to adaptively retrieve relevant semantic evidence from source spherical image features. SphereOcc achieves 13.91% mIoU and 24.65% GeoIoU, yielding relative improvements of 13.9% and 9.3% over the respective best-performing methods, TPVFormer and SurroundOcc. It also ranks first in both metrics across all five scene categories, with consistent advantages across the evaluated spatial partitions and reduced fields of view. The established benchmark and source code will be available at https://feit-feiteng.github.io/Spheriverse.

cs.CV↗
arXiv2026

PanoGS-SLAM: Panoramic 3D Gaussian Splatting SLAM

Real-time dense SLAM is a core capability for robotics applications that require robust localization and high- quality mapping in dynamic or fast-changing environments. Recent 3D Gaussian Splatting (3DGS)-based SLAM methods have shown promising performance, but most are designed for narrow-FoV pinhole cameras, where limited angular coverage weakens pose observability and often leads to unstable photo- metric optimization under rapid motion and large viewpoint changes. We present PanoGS-SLAM, the first panoramic dense SLAM system built on 3D Gaussian Splatting. Our method per- forms differentiable rendering and pose optimization directly in the spherical domain, enabling omnidirectional photometric constraints for more stable tracking. To improve geometric consistency and robustness, we introduce (1) a sphere-consistent photometric loss that compensates for the area distortion of equirectangular projection, and (2) a depth-guided Gaussian initialization strategy that stabilizes incremental mapping in newly observed regions. Extensive experiments on both real and synthetic panoramic benchmarks (PALVIO and SynPano) show that PanoGS-SLAM consistently outperforms geometric and GS-based baselines in tracking accuracy and rendering quality, while achieving fast front-end convergence and real-time perfor- mance. In addition, controlled field-of-view experiments reveal a clear monotonic improvement in optimization conditioning and convergence stability as angular coverage increases, high- lighting the fundamental role of sensing geometry in shaping the optimization landscape of differentiable Gaussian-based SLAM. The source code will be made publicly available.

cs.CV↗
arXiv2026

Streaming Translation and Transcription Through Speech-to-Text Causal Alignment

Simultaneous machine translation (SiMT) has traditionally relied on offline machine translation models coupled with human-engineered heuristics or learned policies. We propose Hikari, a policy-free, end-to-end model for simultaneous speech-to-text translation and streaming transcription. We also introduce Decoder Time Dilation, a mechanism that counteracts the overrepresentation of WAIT tokens in training. We present a supervised fine-tuning strategy that trains the model to recover from delays, significantly improving the quality-latency trade-off. Despite its modest size, Hikari delivers competitive translation quality at consistently low latency, comparing favorably with published IWSLT 2026 submissions up to 38x larger and with proprietary API systems across en-ja, en-de, and en-ru. We release our model weights and code to facilitate further research.

cs.CL↗
arXiv2026

Topological Necessities: Mechanism-Invariant Strategic Subgoals for Cross-Embodiment Goal-Conditioned Control

Long-horizon goal-conditioned reinforcement learning delegates control to a high-level module that proposes subgoals, but existing subgoals are implicit byproducts of value functions or latent actions, tied to the executor that produced them. We study a different object: a route-conditioned order of unavoidable stages that every successful executor must traverse, recoverable from offline trajectories and belonging to none of them. Its defining properties are topological: an unskippable stage is a separating set that every admissible path must cross, and a loop in free space forces a route choice. We read the two by homology in dimensions 0 and 1 over a transport-weighted carrier built from successful trajectories, yielding an enumerable gate set with shell-level certificates; the certified gates are what we call topological necessities. Certified gates enter the decision loop as a recursive topological gate hierarchy. Under a fixed, isomorphic free space, the object survives executor replacement: gates frozen on PointMaze data transfer without retraining to Ant and Humanoid, attaining the highest Humanoid aggregate under a unified interface (96.1), with +36.0 over a map-privileged reference on the multi-route task (p=1.4e-5); the planner saturates PointMaze (100+/-0) and matches or exceeds the strongest baselines on AntMaze (giant +22.9) and Kitchen (+15.8/+12.6).

cs.LG↗