arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.36832

Drop the Details, Get the Gist: Dynamic Conceptual Representation for Video Recognition

Ryoya Yoshiyama·Bowen Wang·Yuta Nakashima
View original source ↗Read source document ↗Export metadata ↓

Abstract

Concept Bottleneck Models (CBMs), which route predictions through a layer of human-interpretable concepts, have substantially improved the interpretability of visual classification by exposing an explicit, language-based rationale for each prediction. Most existing work focuses on static images, and temporal dynamics in videos have been underexplored in conceptual representation. We propose a framework that extends concept-based image recognition to procedural video understanding. It represents a multi-step video as a temporal sequence of segment-level concept score vectors and feeds this sequence into a Transformer encoder to predict the class label. We also explore a way to estimate the contribution of each concept for further interpretability. We evaluated three concept sets of different granularities on cooking and household-activity videos and found that coarser-grained concepts are sufficient for classifying these videos, which implies that fine-grained textual descriptions may be unnecessary.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ryoya Yoshiyama, Bowen Wang, Yuta Nakashima. 2026-10-27. Drop the Details, Get the Gist: Dynamic Conceptual Representation for Video Recognition. https://arxiv.org/abs/2610.36832

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectors

Achieving constant accuracy in object detection is challenging due to the inherent variability of object sizes. One effective approach to this problem involves optimizing input resolution, referred to as a multi-resolution strategy. Previous approaches to resolution optimization have often been based on pre-defined resolutions with manual selection. However, there is a lack of study on run-time resolution optimization for existing architectures. This paper introduces DyRA, a dynamic resolution adjustment network providing an image-specific scale factor for existing detectors. This network is co-trained with detectors utilizing specially designed loss functions, namely ParetoScaleLoss and BalanceLoss. ParetoScaleLoss determines an adaptive scale factor for robustness, while BalanceLoss optimizes overall scale factors according to the localization performance of the detector. The loss function is devised to minimize the accuracy drop across contrasting objectives of different-sized objects for scaling. Our proposed network can improve accuracy across various models, including RetinaNet, Faster-RCNN, FCOS, DINO, and H-Deformable-DETR. The code is available at https://github.com/DaEunFullGrace/DyRA.git.

cs.CV↗
arXiv2026

Smart Surveillance: Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking

Event-triggered recording reduces the footage a surveillance camera must store, but the trigger decides how much idle video is kept, how many events are missed, and how much computation is spent. We compare six motion triggers (frame differencing, dense optical flow, MOG2, KNN, CNT, GSOC), a CNN detector, strided detection, and three motion-gated hybrids under one recording policy, on a benchmark of seven static-camera scenes (5,800 frames, 27 annotated events) with thresholds transferred to unseen scenes by leave-one-video-out. Detector-based triggers stored 19-28% of idle frames, against 36-89% for motion triggers. Under simulated camera jitter, frame differencing, optical flow, MOG2 and KNN stored 97-100% of idle frames, while detector triggers stayed below 29%. Running the detector on every 8th frame kept 99.7% of event frames at 9.8 ms of CPU per frame, cheaper and more robust than gating it with background subtraction. On VisDrone, raising YOLO11n's input from 640 to 1280 px raised AP from 15.5 to 26.8, more than a larger model, and INT8 quantization cut latency by 17% for 0.3 AP. In tracking, longer track memory raised IDF1 from 70.5 to 76.7, but every tracker recovered only 39-44% of overlapped people, because detection rather than association failed.

cs.CV↗
arXiv2026

An Embarrassingly Simple Data Selection Strategy for Million-Scale Multi-Modal Instruction-Tuning

Recent research demonstrates that pretrained large language models (LLMs) require minimal but carefully curated data for instruction-tuning. Building on LLMs, multi-modal models further contend with far larger data volumes, heterogeneous sources, and cross-modal alignment, amplifying both the necessity and the complexity of data selection. However, existing methods rely on deeply extracted embeddings, gradients or narrow quality scores that are time-consuming to compute and limited in expressiveness, frequently failing to surpass a simple random-sampling baseline. To overcome these limitations, we step back from complex embedding spaces and target the fundamental objective of instruction tuning: how and what knowledge should be elicited from the pretrained model and exposed to users. We leverage interaction \underline{S}tyles as a proxy for structural diversity, and decompose data quality into capability-specific \underline{S}cores, yielding an explicit yet \underline{R}ich (SSR) profile for each sample. We further cast the selection as submodular coverage maximization over the capability--style grid, providing an approximation guarantee. SSR consistently delivers superior performance across two distinct architecture stacks, LLaVA-1.5 (Vicuna + CLIP) and LLaVA-OneVision (Qwen2 + SigLIP), two large-scale data pools, six data budget settings, and 14 multi-modal benchmarks. To our knowledge, SSR is the first data selection method validated at million-scale, achieving 99.1% of full-data performance using only 30% of a 2.6M pool. Furthermore, the SSR profile models and selected subsets transfer training-free across data pools and model architectures with capability extensibility.

cs.CV↗