arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2211.13726

Lightweight Event-based Optical Flow Estimation via Iterative Deblurring

Yilun Wu·Federico Paredes-Vallés·Guido C. H. E. de Croon
View original source ↗Read source document ↗Export metadata ↓

Abstract

Inspired by frame-based methods, state-of-the-art event-based optical flow networks rely on the explicit construction of correlation volumes, which are expensive to compute and store, rendering them unsuitable for robotic applications with limited compute and energy budget. Moreover, correlation volumes scale poorly with resolution, prohibiting them from estimating high-resolution flow. We observe that the spatiotemporally continuous traces of events provide a natural search direction for seeking pixel correspondences, obviating the need to rely on gradients of explicit correlation volumes as such search directions. We introduce IDNet (Iterative Deblurring Network), a lightweight yet high-performing event-based optical flow network directly estimating flow from event traces without using correlation volumes. We further propose two iterative update schemes: "ID" which iterates over the same batch of events, and "TID" which iterates over time with streaming events in an online fashion. Our top-performing ID model sets a new state of the art on DSEC benchmark. Meanwhile, the base ID model is competitive with prior arts while using 80% fewer parameters, consuming 20x less memory footprint and running 40% faster on the NVidia Jetson Xavier NX. Furthermore, the TID model is even more efficient offering an additional 5x faster inference speed and 8 ms ultra-low latency at the cost of only a 9% performance drop, making it the only model among current literature capable of real-time operation while maintaining decent performance.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yilun Wu, Federico Paredes-Vallés, Guido C. H. E. de Croon. 2024-05-05. Lightweight Event-based Optical Flow Estimation via Iterative Deblurring. https://doi.org/10.1109/icra57147.2024.10610353

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Q-SiT: Teaching LMMs for Image Quality Scoring and Interpreting

Image quality scoring and interpreting are two fundamental components of Image Quality Assessment (IQA). The former quantifies image quality, while the latter enables descriptive question answering about image quality. Traditionally, these two tasks have been addressed independently. However, image-quality-specific psychophysical studies suggest that these two tasks are conceptually interconnected: interpreting explicitly represents perceived quality attributes whereas scoring summarizes such evidence into an overall quality judgment. Thus, unifying these capabilities within a single model is both intuitive and logically coherent. In this paper, we propose Q-SiT (Quality Scoring and Interpreting joint Teaching), a unified framework that enables large multimodal models (LMMs) to learn both image quality scoring and interpreting simultaneously. We achieve this by transforming conventional IQA datasets into learnable question-answering datasets and incorporating human-annotated quality interpreting data for training. Furthermore, we introduce an efficient scoring \& interpreting balance strategy, which first determines the optimal data mix ratio on lightweight LMMs and then maps this ratio to primary LMMs for fine-tuning adjustment. This strategy not only mitigates task interference and enhances cross-task knowledge transfer but also significantly reduces computational costs compared to direct optimization on full-scale LMMs. With this joint learning framework and corresponding training strategy, we develop Q-SiT, the first model capable of simultaneously performing image quality scoring and interpreting tasks, along with its lightweight variant, Q-SiT-mini. Experimental results demonstrate that Q-SiT achieves strong performance in both tasks with superior generalization IQA abilities, while Q-SiT-mini significantly reduces computational overhead while maintaining competitive performance.

cs.CV↗
arXiv2026

Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment

The rapid digitization of histopathology slides has opened new opportunities for computational tools in clinical and research workflows. Content-based slide retrieval can help pathologists identify morphologically and semantically related precedent cases, supporting expert diagnosis and example-based education. Effective retrieval of whole-slide images (WSIs), however, remains challenging because gigapixel slides contain abundant irrelevant content, focal diagnostic patterns and slide-level semantic information that must be represented at a practicable search cost. Here we present PathSearch, a retrieval framework that combines fine-grained attentive mosaics with slide-level embeddings aligned through vision-language contrastive learning. Trained on 6,926 slide-report pairs, PathSearch captures both fine-grained morphological cues and high-level semantic patterns to enable accurate and flexible retrieval. The framework supports two key functionalities: (1) mosaic-based image-to-image (I2I) retrieval, ensuring accurate and efficient slide search; and (2) multimodal retrieval, where text queries can directly retrieve relevant slides. PathSearch was evaluated on eight tasks comprising 5,021 evaluation slides, spanning malignancy assessment on frozen and hematoxylin and eosin (H\&E)-stained slides, lymph-node metastasis detection, tumor subtyping, mixed-gallery rare-cancer retrieval, and hepatocellular carcinoma (HCC) risk stratification. Internal and external experimental results demonstrate that PathSearch consistently outperforms the strongest existing methods without compromising multimodal accuracy. A multi-center reader study further demonstrated increases in task-level mean diagnostic accuracy, confidence, and inter-observer agreement with PathSearch's support. Together, these results support the effectiveness of PathSearch across diverse retrieval tasks and evaluation settings.

cs.CV↗
arXiv2026

Understanding the Effects of Distractors on Reasoning Vision-Language Models

How does irrelevant information (i.e., distractors) affect test-time scaling in vision-language models (VLMs)? Prior work on text-only language models has shown that textual distractors can intensify inverse scaling, causing models to reason longer but less effective reasoning traces. In this work, we investigate whether similar phenomena arise in multimodal settings. We introduce Idis (Images with distractors), a visual question-answering dataset that systematically varies distractors along semantic and numerical dimensions. Our analyses reveal that visual distractors affect reasoning VLMs in a fundamentally different way from textual distractors: although inverse scaling still emerges, visual distractors reduce accuracy without increasing reasoning length. We further show that attribute counts extracted from reasoning traces provide key insights into how distractors interact with reasoning length and accuracy. As a sanity check, we propose a simple prompting strategy that mitigates distractor-driven predictions in reasoning vision-language models.

cs.CV↗