arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2609.38615

Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation

Hongjia Zhai·Xiyu Zhang·Haoran Zhang·Zhichao Ye·Haomin Liu·Guofeng Zhang·Ian Reid·Xingxing Zuo
View original source ↗Read source document ↗Export metadata ↓

Abstract

Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Exocentric-to-egocentric video generation offers a scalable alternative by transforming abundant third-person manipulation videos into first-person observations. However, existing methods often struggle to faithfully preserve demonstrated hand-object interactions (HOI) across large viewpoint changes due to insufficient fine-grained interaction guidance and weak object-centric anchoring. We present Exo2EgoHOI, an HOI-aware video generative framework for interaction-preserving exocentric-to-egocentric translation. To preserve fine-grained HOI, we introduce a unified 4D HOI prior that combines scene geometry, articulated hand renderings, and dense hand-object relation fields, together with a dual-branch residual adapter for injecting structural and relational cues into the video generation backbone. To preserve object consistency, we introduce Decomposed Gated Cross-Attention, which separately encodes object and background references and adaptively integrates global semantic and local appearance features as object-centric anchors. Experiments on ARCTIC-HOI and Ego-Exo4D demonstrate substantial improvements in object consistency and HOI preservation while maintaining competitive visual fidelity. In particular, on ARCTIC-HOI, Exo2EgoHOI improves object mIoU by 32.3% and reduces MPJPE and PA-MPJPE by 34.7% and 50.0%, respectively, relative to the respective best baseline results. Project page: https://rcl-robotics.github.io/Exo2EgoHOI/.

Explore related subjects

cs.CVcs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hongjia Zhai, Xiyu Zhang, Haoran Zhang, Zhichao Ye, Haomin Liu, Guofeng Zhang, Ian Reid, Xingxing Zuo. 2026-09-29. Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation. https://arxiv.org/abs/2609.38615

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Disentangling Latent Embeddings with Sparse Linear Concept Cones (SLiCC)

Pretrained vision models encode semantic information in latent embedding spaces that are useful for downstream tasks. We hypothesize that this information can be disentangled to separate the content of complex scenes by decomposing each embedding into multiple concept-specific components. We propose Sparse Linear Concept Cones (SLiCC), a supervised dictionary learning approach that estimates a linear synthesis model consisting of sparse, non-negative combinations of groups of dictionary atoms, whose group-wise activity is guided by multi-label information. Each concept-specific component is a non-negative combination of atoms associated with a label and lies in a corresponding concept cone. We introduce an alternating optimization algorithm and an autoencoder formulation of SLiCC. We apply SLiCC to three types of CLIP and DINOv2 embeddings across five datasets. For CLIP, we additionally exploit image-text alignment to enable a self-supervised SLiCC. We also introduce a fully unsupervised autoencoder that enforces group sparsity based on the norm of neuron activations. Quantitative results show that supervised SLiCC improves concept-filtered image retrieval over unfiltered embeddings, including retrieval of finer-grained information not provided during training. Beyond retrieval of individual concepts, the disentangled components enable concept deletion and recombination across images. We further explore conditional image generation using Image-to-Prompt, with qualitative results illustrating retrieval and generation from manipulated concept representations.

cs.CV↗
arXiv2026

BitC-3DGS: High-Capacity 3D Gaussian Splatting Watermarking via Bit Compression

High-capacity watermarking is necessary for 3D Gaussian Splatting (3DGS) assets to embed rich information (\eg, ownership, provenance, and authentication codes), enabling reliable identification and provenance tracking in large-scale 3D asset pipelines. Existing bit-to-token watermarking methods based on a pre-trained text encoder are limited to 77-bit messages due to CLIP's fixed 77-token context length, as tokens beyond this limit are unsupported by learned positional embeddings. To address this limitation, we introduce BitC-3DGS, a bit-compression framework that encodes multiple message bits per token. It employs a bit-compressed tokenization scheme that encodes multiple bits within the same chunk into a single semantic token. To enable recovery of the compressed information, it further introduces a dual-branch architecture for joint chunk decompression and bit decoding, along with a hard-message sampling strategy to improve combinatorial coverage during decoder training. Extensive experiments on the Blender and LLFF datasets demonstrate the effectiveness of BitC-3DGS for high-capacity watermarking, achieving high message recovery accuracy and rendering fidelity. For example, it supports 128-bit message capacity with recovery accuracy comparable to that of 64-bit messages in recent state-of-the-art methods.

cs.CV↗
arXiv2026

CylindTrack: Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking in Mobile Robots

Maintaining consistent target identities and observation histories is essential for mobile robot perception. Panoramic Multi-Object Tracking (MOT) benefits from broad visual coverage but must address coordinate discontinuities at the equirectangular image seam and fluctuations in relative-depth estimates. Such fluctuations can undermine depth-aware association, highlighting the need for temporally consistent depth cues. We propose CylindTrack, a depth-aware cylindrical tracking-by-detection framework that jointly models panoramic topology and temporal depth consistency. Depth-Temporal Trajectory Modeling (DTM) maintains instance depth and its rate of change as filtered trajectory states. Spherical Spatio-Temporal Consistency Learning (SSTC) combines a Temporal Mixer with Spherical Geometry-aware Attention to refine depth-aware representations. A Topology-Aware Cylindrical Motion Model (TCMM) represents horizontal motion in an unwrapped angular state and incorporates periodic overlap, angular consistency, and filtered depth cues into association. Experiments on QuadTrack and JRDB demonstrate improved identity consistency and trajectory continuity, with HOTA scores of 33.67 and 31.12 and IDF1 scores of 40.45 and 34.33, respectively. The framework processes 28.56 and 21.34 frames per second, respectively, on an NVIDIA GeForce RTX 3090 GPU under the reported evaluation settings. These results support persistent panoramic multi-object tracking for mobile robot perception. The source code will be released at https://github.com/warriordby/CylindTrack.

cs.CV↗