arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2109.02281

Exploiting Spatial-Temporal Semantic Consistency for Video Scene Parsing

Xingjian He·Weining Wang·Zhiyong Xu·Hao Wang·Jie Jiang·Jing Liu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Compared with image scene parsing, video scene parsing introduces temporal information, which can effectively improve the consistency and accuracy of prediction. In this paper, we propose a Spatial-Temporal Semantic Consistency method to capture class-exclusive context information. Specifically, we design a spatial-temporal consistency loss to constrain the semantic consistency in spatial and temporal dimensions. In addition, we adopt an pseudo-labeling strategy to enrich the training dataset. We obtain the scores of 59.84% and 58.85% mIoU on development (test part 1) and testing set of VSPW, respectively. And our method wins the 1st place on VSPW challenge at ICCV2021.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xingjian He, Weining Wang, Zhiyong Xu, Hao Wang, Jie Jiang, Jing Liu. 2021-09-06. Exploiting Spatial-Temporal Semantic Consistency for Video Scene Parsing. https://arxiv.org/abs/2109.02281

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Dream4D: Lifting Camera-Controlled I2V towards Spatiotemporally Consistent 4D Generation

The synthesis of spatiotemporally coherent 4D content presents fundamental challenges in computer vision, requiring simultaneous modeling of high-fidelity spatial representations and physically plausible temporal dynamics. Current approaches often struggle to maintain view consistency while handling complex scene dynamics, particularly in large-scale environments with multiple interacting elements. This work introduces Dream4D, a novel framework that bridges this gap through a synergy of controllable video generation and neural 4D reconstruction. Our approach seamlessly combines a two-stage architecture: it first predicts optimal camera trajectories from a single image using few-shot learning, then generates geometrically consistent multi-view sequences via a specialized pose-conditioned diffusion process, which are finally converted into a persistent 4D representation. This framework is the first to leverage both rich temporal priors from video diffusion models and geometric awareness of the reconstruction models, which significantly facilitates 4D generation and shows higher quality (e.g., mPSNR, mSSIM) over existing methods.

cs.CV↗
arXiv2026

Adaptive Dual-Constrained Line Aggregation for Cross-Paradigm Line Segment Detection

Line segment detection has been studied for decades, yet existing methods are typically designed for different detection paradigms. Generic line segment detectors aim to recover all meaningful line segments in an image, whereas recent deep-learning-based approaches mainly target wireframe line segments that describe salient geometric structures. Because these paradigms follow different detection objectives, methods optimized for one often perform poorly on the other. In this work, we propose Adaptive Dual-Constrained Line Aggregation (ADLA), a line extraction framework designed to operate across different line segment detection paradigms. Starting from an edge strength map, ADLA progressively aggregates pixels into candidate line segments under two complementary geometric constraints: orientation coherence and bounded orthogonal distance to an adaptively estimated line model. During aggregation, the line centroid and orientation are dynamically updated using the accumulated supporting pixels, progressively improving the geometric consistency of the estimated line. Edge strength information is further incorporated into orientation estimation, seed selection, model refinement, and segment validation, reducing the need for extensive parameter tuning. Experiments on three publicly available datasets covering generic, wireframe, and Manhattan line segment detection demonstrate consistently strong performance across substantially different annotation settings. ADLA achieves (F^H) scores of 0.8665 on YorkUrban-LineSegment dataset, 0.8720 on ShanghaiTech dataset, and 0.7297 on YorkUrban dataset. These results demonstrate the effectiveness and flexibility of ADLA across different line segment detection paradigms. The source code for this work is publicly available at https://github.com/ChenguangTelecom/adla .

cs.CV↗
arXiv2026

Confidence-Calibrating Regularization for Robust Brain MRI Segmentation Under Domain Shift

The Segment Anything Model (SAM) exhibits strong zero-shot performance on natural images but suffers from domain shift and overconfidence when applied to medical volumes. We propose \textbf{CalSAM}, a lightweight adaptation framework that (i) reduces encoder sensitivity to domain shift via a \emph{Feature Fisher Information Penalty} (FIP) computed on 3D feature maps and (ii) penalizes overconfident voxel-wise errors through a \emph{Confidence Misalignment Penalty} (CMP). The combined loss, \(\mathcal{L}_{\mathrm{CalSAM}}\) fine-tunes only the mask decoder while keeping SAM's encoders frozen. On cross-center and scanner-shift evaluations, CalSAM substantially improves accuracy and calibration: e.g., on the BraTS scanner split (Siemens$\to$GE) CalSAM shows a $+7.4\%$ relative improvement in $\mathrm{DSC}$ (80.1\% vs.\ 74.6\%), a $-26.9\%$ reduction in $\mathrm{HD95}$ (4.6 mm vs.\ 6.3 mm), and a $-39.5\%$ reduction in $\mathrm{ECE}$ (5.2\% vs.\ 8.6\%). On ATLAS-C (motion corruptions), CalSAM achieves a $+5.3\%$ relative improvement in $\mathrm{DSC}$ (75.9\%) and a $-32.6\%$ reduction in $\mathrm{ECE}$ (5.8\%). Ablations show FIP and CMP contribute complementary gains ($p<0.01$), and the Fisher penalty incurs a modest $\sim$15\% training-time overhead. CalSAM therefore delivers improved domain generalization and better-calibrated uncertainty estimates for brain MRI segmentation, while retaining the computational benefits of freezing SAM's encoder.

cs.CV↗