arXiv Science⌕ Search

arXiv · 2611.09642

NAVE: A Normalized, Adaptive Conformer for Antarctic Whale Vocalization Detection

Abstract

Antarctic whale vocalization detection requires temporally localizing call boundaries in long underwater recordings and classifying them into predefined call types. Training reliable machine learning models for this task is difficult due to the variability and sparsity of whale calls in passive acoustic monitoring datasets. These difficulties are further amplified by time-varying noise that can mask whale calls and by site-specific ambient sound characteristics to which a trained model may not generalize. In this work, we present NAVE, a Conformer-based detector that improves whale-call detection via three components. First, its input representation combines per-bin complex mean subtraction, which suppresses constant site-specific ambient sound characteristics, with an additional per-channel energy normalization (PCEN) channel that adapts to temporal variations in the noise floor. Second, a frequency-dynamic stem processes these noise-aware representations with convolutions that adapt across the characteristic frequency bands of the target call types. Third, we use a Conformer backbone with a convolution kernel matched to the duration of the target calls that adds an inductive bias and improves temporal localization. We evaluate NAVE on BioDCASE 2026 Task 2 using leave-one-site-out cross-validation across the validation sites, which approximates deployment on unseen recording sites. A single NAVE model with a tuned per-class threshold exceeds the stronger WhaleVAD-BPN baseline, using no boundary-proposal network and no search over post-processing parameters. A controlled ablation study shows that the normalized front-end and a duration-matched convolutional kernel each contribute substantially to recovering the D downsweep, the hardest class for previous detectors, with the wide kernel giving the largest aggregate gain.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Matthias Nagl, Paul Primus, Gerhard Widmer. 2026-11-06. NAVE: A Normalized, Adaptive Conformer for Antarctic Whale Vocalization Detection. https://doi.org/10.5281/zenodo.22954971

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

REAL-2MIX: Evaluating Identity Faithfulness in Audio-Visual Target Speaker Extraction

Audio-visual target speaker extraction should recover the person indicated by the video, rather than repeatedly return the acoustically dominant voice. We introduce REAL-2MIX, a Mandarin benchmark of jointly recorded two-speaker mixtures with synchronized multi-view video and scene-matched solo references. Its 77 scenes contain 7,598 clips; filtering yields 6,038 evaluable mixtures, each queried for both speakers, giving 12,076 paired target queries. We also curate VOXBLINK2-AVSE, comprising 250,828 synchronized audio--lip pairs. A seven-system TF-GridNet study examines visual representations, training corpora, and conditioning strategies. We evaluate transcription accuracy, speaker identity, residual overlap, and perceptual quality. The AVSR early-fusion system achieves 22.99% CER, and 93.38% of its outputs are more similar to the target speaker than to the interferer. Yet only 42.49% satisfy both this identity criterion and the overlap detector's single-speaker criterion; both target queries satisfy these criteria for just 20.77% of mixtures. A separate Real-AVSE challenge study examines development-remix adaptation. The benchmark reveals that favorable content and identity scores need not coincide with low detected overlap, motivating paired-target evaluation.

eess.AS↗

AccentCanvas: Activity-Guided Bidirectional Accent Conversion

Accent conversion for dubbing must change pronunciation while preserving content, voice, and source timing. Normalization alone cannot produce requested non-native accents, and matching total duration does not ensure local speech/pause alignment. We present AccentCanvas, a shared discrete diffusion model for normalization and L1-to-L2 conversion, trained on cross-paired shared-prompt recordings and synthetic pairs. Frame-level speech activity and repetition-preserving units guide local timing through synthesis. Across both tasks, AccentCanvas achieves the highest accent conversion performance (Accent Arrival) and mean listener-rated accent similarity among compared systems, with over 73% lower speech-boundary error than the best baseline for each task. These gains accompany higher L1-to-L2 WER than TokAN and Vevo-Style, highlighting a tradeoff between requested-accent resemblance and ASR-based content recognition. https://smokedindia.github.io/accentcanvas/

eess.AS↗

GTCRN-SS: Lightweight Causal Speech Separation with Residual Allocation and Reconstruction Refinement

Low-complexity causal speech separation must preserve speaker information and allocate residual mixture energy without relying on future feature frames. We present GTCRN-SS, a two-stage time-frequency separator that adapts the compact GTCRN enhancement backbone to fixed-output speech separation. The model combines internal residual allocation, reconstruction-conditioned refinement, and frame-aligned recurrent-state transfer. Its complex-mask estimator assigns non-target content to an unreturned residual branch, while the complementary speech estimate conditions a second stage whose final masks remain anchored to the original mixture. A lightweight low-rank adapter transfers causal recurrent evidence between recursive executions without adding another recurrent memory network. The neural graph excludes future feature frames and incurs only the fixed analysis look-ahead of the centered short-time Fourier transform. On noisy Libri2Mix, GTCRN-SS achieves 10.1 dB SDRi and 9.5 dB SI-SNRi with fewer than one million parameters and a low computational footprint. Independently trained ablations confirm the benefits of recurrent-state transfer and reconstruction conditioning, and corpus-level diagnostics show that the internal branch primarily captures residual rather than target-speech content. These results establish GTCRN-SS as an effective quality-efficiency operating point for causal monaural speech separation.

eess.AS↗