arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2506.23120

Enhancing Spatial Reasoning in Multimodal Large Language Models through Reasoning-based Segmentation

Zhenhua Ning·Zhuotao Tian·Shaoshuai Shi·Guangming Lu·Daojing He·Wenjie Pei·Li Jiang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Recent advances in point cloud perception have demonstrated remarkable progress in scene understanding through vision-language alignment leveraging large language models (LLMs). However, existing methods may still encounter challenges in handling complex instructions that require accurate spatial reasoning, even if the 3D point cloud data provides detailed spatial cues such as size and position for identifying the targets. To tackle this issue, we propose Relevant Reasoning Segmentation (R$^2$S), a reasoning-based segmentation framework. The framework emulates human cognitive processes by decomposing spatial reasoning into two sequential stages: first identifying relevant elements, then processing instructions guided by their associated visual priors. Furthermore, acknowledging the inadequacy of existing datasets in complex reasoning tasks, we introduce 3D ReasonSeg, a reasoning-based segmentation dataset comprising 25,185 training samples and 3,966 validation samples with precise annotations. Both quantitative and qualitative experiments demonstrate that the R$^2$S and 3D ReasonSeg effectively endow 3D point cloud perception with stronger spatial reasoning capabilities, and we hope that they can serve as a new baseline and benchmark for future work.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Zhenhua Ning, Zhuotao Tian, Shaoshuai Shi, Guangming Lu, Daojing He, Wenjie Pei, Li Jiang. 2025-06-29. Enhancing Spatial Reasoning in Multimodal Large Language Models through Reasoning-based Segmentation. https://arxiv.org/abs/2506.23120

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion

Estimating the 6D pose and 3D size of an object from visual data is a fundamental task in computer vision. Although single-view geometry is a deeply established domain, contemporary category-level methods frequently rely on rigid prerequisites such as precise object models, ground truth depth, or multi-modal LiDAR integration to achieve robust results. In this work, we introduce a unified generative framework that addresses both single-view category-level pose estimation and temporal sequence tracking using only RGB input. Our method leverages score-based diffusion models to generate a rich multi-hypothesis pose distribution, inherently capturing spatial and geometric uncertainties. While existing diffusion-based estimators typically rely on computationally expensive likelihood models to prune outliers, we propose an efficient alternative utilising Mean Shift to directly isolate the distribution's mode as the final pose estimate. Our approach establishes a new state-of-the-art baseline on the challenging REAL275 benchmark among two-stage, crop-based estimators. Furthermore, by decoupling object detection from pose estimation, our generative framework explicitly avoids the catastrophic domain overfitting inherent to end-to-end single-stage detectors, achieving highly robust zero-shot generalisation on the unseen Wild6D dataset. Finally, we demonstrate that the iterative nature of our score-based sampler enables a seamless transition to video sequences by preserving and propagating the multi-hypothesis distribution across time as a coherent temporal prior.

cs.CV↗
arXiv2026

A Unified Hierarchical Framework for Fine-grained Cross-view Geo-localization over Large-scale Scenarios

Cross-view geo-localization is a promising solution for large-scale localization problems, requiring the sequential execution of retrieval and metric localization tasks to achieve fine?grained predictions. However, existing methods typically focus on designing standalone models for these two tasks, resulting in inefficient collaboration and increased training overhead. In this paper, we propose UnifyGeo, a novel unified hierarchical geo-localization framework that integrates retrieval and metric localization tasks into a single network. Specifically, we first em?ploy a unified learning strategy to jointly learn multi-granularity representations, establishing task associations between retrieval and metric localization. Subsequently, we design a re-ranking mechanism guided by a dedicated loss function, which enhances geo-localization performance by improving both retrieval accuracy and metric localization references. Extensive experiments demonstrate that UnifyGeo significantly outperforms state-of-the?art methods in both task-isolated and task-associated settings. On the challenging VIGOR benchmark, UnifyGeo achieves 39.64% and 25.58% 1-meter-level localization recall under same-area and cross-area evaluations, respectively, demonstrating strong fine?grained localization capability in large-scale scenarios. Code will be available at https://github.com/chord-sz/UnifyGeo.

cs.CV↗
arXiv2026

CineScale: Tuning-Free High-Resolution Video Generation

Video diffusion models have achieved remarkable progress in recent years, yet generating high-resolution videos remain a fundamental challenge. Most video generators are trained at limited spatial resolutions due to the scarcity of high-resolution 4K video data and the prohibitive computational cost of large-scale training on such data. Most video diffusion models are trained on 720p videos and are therefore effectively limited to generating videos at similar resolutions during inference. To address this gap, we propose CineScale. CineScale, to the best of our knowledge, is the first tuning-free inference framework enabling pretrained video diffusion models to generate high-quality videos at resolutions far beyond those seen during training. Our key observation is that generation quality degrades at higher resolutions because positional encodings shift beyond their training distribution, producing blurred details and structurally incoherent videos. To address this gap, we introduce Adaptively Rectified RoPE. Our extensive experiments show that CineScale enables pretrained diffusion models, despite never being trained on high-resolution data, to generate high-fidelity 4K video without any fine-tuning, improving local detail and sharpness while preserving temporal coherence. This demonstrates that high-resolution generation capabilities can be unlocked purely at inference time.

cs.CV↗