arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2603.02270

From Visual to Multimodal: Systematic Ablation of Encoders and Fusion Strategies in Animal Identification

Vasiliy Kudryavtsev·Kirill Borodin·German Berezin·Kirill Bubenchikov·Grach Mkrtchian·Alexander Ryzhkov
View original source ↗Read source document ↗Export metadata ↓

Abstract

Automated animal identification is a practical task for reuniting lost pets with their owners, yet current systems often struggle due to limited dataset scale and reliance on unimodal visual cues. This study introduces a multimodal verification framework that enhances visual features with semantic identity priors derived from synthetic textual descriptions. We constructed a massive training corpus of 1.9 million photographs covering 695,091~unique animals to support this investigation. Through systematic ablation studies, we identified SigLIP2-Giant and E5-Small-v2 as the optimal vision and text backbones. We further evaluated fusion strategies ranging from simple concatenation to adaptive gating to determine the best method for integrating these modalities. Our proposed approach utilizes a gated fusion mechanism and achieved a Top-1 accuracy of 84.28\% and an Equal Error Rate of 0.0422 on a comprehensive test protocol. These results represent an 11\% improvement over leading unimodal baselines and demonstrate that integrating synthesized semantic descriptions significantly refines decision boundaries in large-scale pet re-identification.\\\\ \textbf{Conference update.} This work was accepted to the FGVC13 Workshop at CVPR 2026. We thank Vadim Ezhov and Nikolai Shcheglov for their help with the poster presentation and demonstration at the conference.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Vasiliy Kudryavtsev, Kirill Borodin, German Berezin, Kirill Bubenchikov, Grach Mkrtchian, Alexander Ryzhkov. 2026-08-10. From Visual to Multimodal: Systematic Ablation of Encoders and Fusion Strategies in Animal Identification. https://doi.org/10.3390/jimaging12010030

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

We present Lunima-OmniLV (abbreviated as OmniLV), a universal multimodal multi-task framework for low-level vision that addresses over 100 sub-tasks across four major categories: image restoration, image enhancement, weak-semantic dense prediction, and stylization. OmniLV leverages both textual and visual prompts to offer flexible and user-friendly interactions. Built on Diffusion Transformer (DiT)-based generative priors, our framework supports arbitrary resolutions -- achieving optimal performance at 1K resolution -- while preserving fine-grained details and high fidelity. Through extensive experiments, we demonstrate that separately encoding text and visual instructions, combined with co-training using shallow feature control, is essential to mitigate task ambiguity and enhance multi-task generalization. Our findings also reveal that integrating high-level generative tasks into low-level vision models can compromise detail-sensitive restoration. These insights pave the way for more robust and generalizable low-level vision systems.

cs.CV↗
arXiv2026

Dream4D: Lifting Camera-Controlled I2V towards Spatiotemporally Consistent 4D Generation

The synthesis of spatiotemporally coherent 4D content presents fundamental challenges in computer vision, requiring simultaneous modeling of high-fidelity spatial representations and physically plausible temporal dynamics. Current approaches often struggle to maintain view consistency while handling complex scene dynamics, particularly in large-scale environments with multiple interacting elements. This work introduces Dream4D, a novel framework that bridges this gap through a synergy of controllable video generation and neural 4D reconstruction. Our approach seamlessly combines a two-stage architecture: it first predicts optimal camera trajectories from a single image using few-shot learning, then generates geometrically consistent multi-view sequences via a specialized pose-conditioned diffusion process, which are finally converted into a persistent 4D representation. This framework is the first to leverage both rich temporal priors from video diffusion models and geometric awareness of the reconstruction models, which significantly facilitates 4D generation and shows higher quality (e.g., mPSNR, mSSIM) over existing methods.

cs.CV↗
arXiv2026

Adaptive Dual-Constrained Line Aggregation for Cross-Paradigm Line Segment Detection

Line segment detection has been studied for decades, yet existing methods are typically designed for different detection paradigms. Generic line segment detectors aim to recover all meaningful line segments in an image, whereas recent deep-learning-based approaches mainly target wireframe line segments that describe salient geometric structures. Because these paradigms follow different detection objectives, methods optimized for one often perform poorly on the other. In this work, we propose Adaptive Dual-Constrained Line Aggregation (ADLA), a line extraction framework designed to operate across different line segment detection paradigms. Starting from an edge strength map, ADLA progressively aggregates pixels into candidate line segments under two complementary geometric constraints: orientation coherence and bounded orthogonal distance to an adaptively estimated line model. During aggregation, the line centroid and orientation are dynamically updated using the accumulated supporting pixels, progressively improving the geometric consistency of the estimated line. Edge strength information is further incorporated into orientation estimation, seed selection, model refinement, and segment validation, reducing the need for extensive parameter tuning. Experiments on three publicly available datasets covering generic, wireframe, and Manhattan line segment detection demonstrate consistently strong performance across substantially different annotation settings. ADLA achieves (F^H) scores of 0.8665 on YorkUrban-LineSegment dataset, 0.8720 on ShanghaiTech dataset, and 0.7297 on YorkUrban dataset. These results demonstrate the effectiveness and flexibility of ADLA across different line segment detection paradigms. The source code for this work is publicly available at https://github.com/ChenguangTelecom/adla .

cs.CV↗