Zihan Zhou | arXiv Science

arXiv2026

ORMOT: A Dataset and Framework for Omnidirectional Referring Multi-Object Tracking

Omnidirectional cameras provide 360° spatial coverage, making them increasingly valuable in applications such as autonomous driving and video surveillance. While Multi-Object Tracking (MOT) and its language-guided extension, Referring Multi-Object Tracking (RMOT), have achieved notable progress, existing methods rely on conventional cameras with limited fields of view, causing critical contextual cues to be lost when targets move outside the frame. This fundamentally limits the model's ability to interpret long-horizon language descriptions involving sequential actions, spatial relations, and group behaviors. In this work, we propose Omnidirectional Referring Multi-Object Tracking (ORMOT), a novel task extending RMOT to omnidirectional imagery, where 360° coverage ensures complete scene context for accurate language-guided tracking. To advance this task, we construct ORSet, a dataset comprising 27 omnidirectional scenes, 848 language descriptions, and 3,401 annotated objects. Furthermore, we propose ORTrack, an LVLM-driven framework that enables zero-shot language-guided detection and robust cross-frame association in complex 360° environments. Experiments on ORSet demonstrate that ORTrack achieves state-of-the-art performance, providing a strong baseline for future research. The dataset and code will be open-sourced at https://github.com/chen-si-jia/ORMOT.

cs.CV↗

arXiv2026

SpatialCrafter: Single Image World Modeling with Generative 3D Proxies

Explorable image-to-scene generation is essential for applications in gaming, robotics, and virtual reality. Existing methods based on video diffusion model (VDM) commonly rely on incomplete conditioning signals such as sparse point clouds or 2D panoramas, leading to stochastic hallucinations, long-term drifts and suboptimal 3D consistency. We present SpatialCrafter, a novel two-stage framework that addresses these issues by introducing a global 3D proxy for high-fidelity image-to-scene generation. Specifically, we decompose the generation process into global proxy generation and appearance refinement. For proxy generation, we propose a Point-anchored Sparse Structure~(PaSS) Flow module that predicts a spatially aligned and geometrically consistent 3D proxy. For appearance refinement, we re-frame the VDM as a Generative Deferred Refiner which synthesizes high-frequency photorealistic details upon proxy-defined scene geometry. To better integrate the proxy with the pre-trained VDM, we introduce Parallel Geometry Injection and Proxy-Aware Corruption training strategies, which improve robustness to proxy artifacts without disrupting the pretrained generative manifold. Furthermore, as no suitable dataset exists for this explorable scene generation task, we construct a new large-scale dataset of 115K scenes. To the best of our knowledge, it is the first hybrid dataset for image-to-scene generation. Extensive experiments on both synthetic and real-world datasets show that SpatialCrafter outperforms state-of-the-art methods, mitigates long-term drift, and remains robust and consistent under rapid camera motion and extreme viewpoint changes. Our project page: \href{https://fangchuan.github.io/SpatialCrafter/}{fangchuan.github.io/SpatialCrafter/}

cs.CV↗