arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.32332

4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents

Yanhao Jia·Yi Zhu·Yiran Zhong·Weigao Sun·Feida Zhu·Pengkun Jiao·Sihan Yang·Ming Ma·Steven Hoi·Erik Cambria
View original source ↗Read source document ↗Export metadata ↓

Abstract

Humans efficiently process multimodal streams spanning hours to years, building persistent, fine-grained spatiotemporal memories that support reasoning about and predicting long-horizon events. In contrast, existing benchmarks focus on short- and medium-context tasks, and most foundation models struggle to preserve fine-grained information over long horizons. To evaluate and improve long-horizon spatiotemporal reasoning, we introduce the LongVideo dataset and the 4DMemoryBench. LongVideo comprises about 900 hours of video and over 5,000 manually reviewed questions covering visual, textual, and spatiotemporal reasoning across diverse context lengths. We further develop the 4DMemory Harness, which integrates four complementary skills, 22+ specialized tool models, and a model-specific retrieval-augmented generation system. Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50\%. Compared to previous benchmark, the same models exhibit a performance drop of more than 10\% on 4DMemoryBench. We hope 4DMemory will facilitate the development of spatiotemporal agents capable of addressing complex, long-horizon tasks.

Explore related subjects

cs.CVcs.AIcs.CLcs.MM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yanhao Jia, Yi Zhu, Yiran Zhong, Weigao Sun, Feida Zhu, Pengkun Jiao, Sihan Yang, Ming Ma, Steven Hoi, Erik Cambria. 2026-10-23. 4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents. https://arxiv.org/abs/2610.32332

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Disentangling Latent Embeddings with Sparse Linear Concept Cones (SLiCC)

Pretrained vision models encode semantic information in latent embedding spaces that are useful for downstream tasks. We hypothesize that this information can be disentangled to separate the content of complex scenes by decomposing each embedding into multiple concept-specific components. We propose Sparse Linear Concept Cones (SLiCC), a supervised dictionary learning approach that estimates a linear synthesis model consisting of sparse, non-negative combinations of groups of dictionary atoms, whose group-wise activity is guided by multi-label information. Each concept-specific component is a non-negative combination of atoms associated with a label and lies in a corresponding concept cone. We introduce an alternating optimization algorithm and an autoencoder formulation of SLiCC. We apply SLiCC to three types of CLIP and DINOv2 embeddings across five datasets. For CLIP, we additionally exploit image-text alignment to enable a self-supervised SLiCC. We also introduce a fully unsupervised autoencoder that enforces group sparsity based on the norm of neuron activations. Quantitative results show that supervised SLiCC improves concept-filtered image retrieval over unfiltered embeddings, including retrieval of finer-grained information not provided during training. Beyond retrieval of individual concepts, the disentangled components enable concept deletion and recombination across images. We further explore conditional image generation using Image-to-Prompt, with qualitative results illustrating retrieval and generation from manipulated concept representations.

cs.CV↗
arXiv2026

BitC-3DGS: High-Capacity 3D Gaussian Splatting Watermarking via Bit Compression

High-capacity watermarking is necessary for 3D Gaussian Splatting (3DGS) assets to embed rich information (\eg, ownership, provenance, and authentication codes), enabling reliable identification and provenance tracking in large-scale 3D asset pipelines. Existing bit-to-token watermarking methods based on a pre-trained text encoder are limited to 77-bit messages due to CLIP's fixed 77-token context length, as tokens beyond this limit are unsupported by learned positional embeddings. To address this limitation, we introduce BitC-3DGS, a bit-compression framework that encodes multiple message bits per token. It employs a bit-compressed tokenization scheme that encodes multiple bits within the same chunk into a single semantic token. To enable recovery of the compressed information, it further introduces a dual-branch architecture for joint chunk decompression and bit decoding, along with a hard-message sampling strategy to improve combinatorial coverage during decoder training. Extensive experiments on the Blender and LLFF datasets demonstrate the effectiveness of BitC-3DGS for high-capacity watermarking, achieving high message recovery accuracy and rendering fidelity. For example, it supports 128-bit message capacity with recovery accuracy comparable to that of 64-bit messages in recent state-of-the-art methods.

cs.CV↗
arXiv2026

CylindTrack: Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking in Mobile Robots

Maintaining consistent target identities and observation histories is essential for mobile robot perception. Panoramic Multi-Object Tracking (MOT) benefits from broad visual coverage but must address coordinate discontinuities at the equirectangular image seam and fluctuations in relative-depth estimates. Such fluctuations can undermine depth-aware association, highlighting the need for temporally consistent depth cues. We propose CylindTrack, a depth-aware cylindrical tracking-by-detection framework that jointly models panoramic topology and temporal depth consistency. Depth-Temporal Trajectory Modeling (DTM) maintains instance depth and its rate of change as filtered trajectory states. Spherical Spatio-Temporal Consistency Learning (SSTC) combines a Temporal Mixer with Spherical Geometry-aware Attention to refine depth-aware representations. A Topology-Aware Cylindrical Motion Model (TCMM) represents horizontal motion in an unwrapped angular state and incorporates periodic overlap, angular consistency, and filtered depth cues into association. Experiments on QuadTrack and JRDB demonstrate improved identity consistency and trajectory continuity, with HOTA scores of 33.67 and 31.12 and IDF1 scores of 40.45 and 34.33, respectively. The framework processes 28.56 and 21.34 frames per second, respectively, on an NVIDIA GeForce RTX 3090 GPU under the reported evaluation settings. These results support persistent panoramic multi-object tracking for mobile robot perception. The source code will be released at https://github.com/warriordby/CylindTrack.

cs.CV↗