arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.22371

Gen4P: Video Generators are 4D Perceivers

Yitong Xia·Zeren Jiang·Chuanxia Zheng
View original source ↗Read source document ↗Export metadata ↓

Abstract

We introduce Gen4P, a unified framework for recovering 4D geometry and motion from monocular video. We show that a pretrained video generator can be repurposed into a feed-forward 4D perceiver without iterative denoising. Building on on-demand querying, Gen4P processes clean video latents in a single encoder pass and predicts the requested 4D attributes through a lightweight cross-attention decoder. The on-demand query formulation enables flexible spatial sampling, and our proposed continuous-time queries further provide temporal flexibility. To strengthen geometric knowledge, we propose a straightforward plug-in solution by incorporating a pretrained video-depth LoRA as a frozen geometry prior, improving overall performance without adding extra trainable parameters. Gen4P achieves state-of-the-art or competitive performance on tasks of 3D point tracking, video depth and camera pose estimation. Qualitative results on in-the-wild videos illustrate its capability of tracking complex motion while maintaining the delicate structure. Together, these results demonstrate the potential of video generative pretraining for unified 4D perception. See https://gen4p.github.io for more visual results.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yitong Xia, Zeren Jiang, Chuanxia Zheng. 2026-10-15. Gen4P: Video Generators are 4D Perceivers. https://arxiv.org/abs/2610.22371

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Text-to-Image Synthesis: A Decade Survey

Text-to-image synthesis (T2I) focuses on generating high-quality images from textual descriptions In this survey, we review 490 papers on T2I spanning the past decade, with representative advances covered through 2026. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation, and we additionally examine the shared components whose evolution has driven recent progress. Building on this foundation, we discuss the development of these models for T2I and organize cutting-edge research by problem rather than by chronology alone, covering controllability, personalized and identity-preserving generation, instruction-based image editing, preference alignment and post-training with human and AI feedback, efficiency, safety, and consistency in content and spatial relationships. We further review unified multimodal understanding-and-generation models, summarize the datasets and evaluation metrics commonly used in T2I research, and provide a comparative discussion of the three paradigms together with their respective trade-offs. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and forward-looking research opportunities in this field.

cs.CV↗
arXiv2026

Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking

Robust 3D state estimation in multi-object tracking is crucial for object localization in self-driving vehicles. Misalignment between the Kalman filter (KF) motion model and an object's motion can cause state drift, particularly in distant and occlusion cases. The static motion model assumption in the conventional KF is widely used in 3D MOT due to its low latency; however, its estimation accuracy degrades as the object's motion disaligns with the chosen model due to traffic. Although recent work introduces a multimodel KF that runs multiple motion models simultaneously to improve state robustness, it incurs significant computational overhead, so many studies still favor the constant-motion KF to maintain efficiency. This work addresses this trade-off by introducing a motion dynamics Kalman filter (MD-KF) that accounts for motion variation through temporal modeling of object motion and adaptively scales the contribution of the motion components in the motion model to correct state-prediction drift caused by this variation, while maintaining computational efficiency by retaining a single-model structure. Evaluations on KITTI and the Waymo Open Dataset, complemented by vehicle experiments, show that MD-KF achieves tracking performance comparable to multimodel KF with an average 1.72x speedup, while outperforming it in handling highly distant and prolonged occluded objects.

cs.CV↗
arXiv2026

RUSplatting: Robust 3D Gaussian Splatting for Sparse-View Underwater Scene Reconstruction

Reconstructing high-fidelity underwater scenes remains a challenging task due to light absorption, scattering, and limited visibility inherent in aquatic environments. This paper presents an enhanced Gaussian Splatting-based framework that improves both the visual quality and geometric accuracy of deep underwater rendering. We propose decoupled learning for RGB channels, guided by the physics of underwater attenuation, to enable more accurate colour restoration. To address sparse-view limitations and improve view consistency, we introduce a frame interpolation strategy with a novel adaptive weighting scheme. Additionally, we introduce a new loss function aimed at reducing noise while preserving edges, which is essential for deep-sea content. We also release a newly collected dataset, Submerged3D, captured specifically in deep-sea environments. Experimental results demonstrate that our framework consistently outperforms state-of-the-art methods with PSNR gains up to 1.90dB, delivering superior perceptual quality and robustness, and offering promising directions for marine robotics and underwater visual analytics. The code of RUSplatting is available at https://github.com/theflash987/RUSplatting and the dataset Submerged3D can be downloaded at https://zenodo.org/records/15482420.

cs.CV↗