arXiv Science⌕ Search

arXiv · 2611.11130

Virtual-Flow: Virtual Microphone-based Speech Enhancement via Semi-supervised Flow Matching

Abstract

Neural network-based virtual microphone estimation (neural-VME) synthesizes microphone signals at unobserved spatial locations from a limited set of real microphone (RM) physical recordings. This enables high-resolution arrays that improve spatial processing for downstream tasks such as speech enhancement. However, existing methods rely on supervised training with ground-truth signals captured at virtual locations, requiring datasets that scale with the number of virtual microphones (VM) and leaving open the question of how to design optimal large-array configurations. We propose Virtual-Flow, a flow-matching framework that implicitly generates VM signals conditioned on real recordings, without requiring ground-truth VM signals. We introduce a semi-supervised training strategy based on pseudo-VM targets constructed from time-delayed superpositions of real recordings. Virtual-Flow shows improved magnitude estimation in high-frequency bands, yielding richer spatial cues for beamforming. Experiments demonstrate that the proposed Virtual-Flow, combined with reflow distillation, outperforms existing neural-VME baselines in VM-based beamforming and achieves competitive end-to-end speech enhancement performance with low computational cost.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Dongheon Lee, Ashutosh Pandey, Sanjeel Parekh, Zhaoheng Ni, Daniel Wong, Jacob Donley, Buye Xu, Juan Azcarreta. 2026-11-08. Virtual-Flow: Virtual Microphone-based Speech Enhancement via Semi-supervised Flow Matching. https://arxiv.science/abs/2611.11130

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Score-Informed Transformer for Refining MIDI Velocity in Automatic Piano Transcription

MIDI velocity is crucial for capturing expressive dynamics in human performances. In practical scenarios, a music score with inaccurate velocities may be available alongside the performance audio (e.g., music education and free online archives), enabling the task of score-informed MIDI velocity estimation. In this work, we propose a modular, lightweight score-informed Transformer correction module that refines the velocity estimates of Automatic Music Transcription (AMT) systems. Leveraging high-quality expressive datasets for rigorous evaluation, we focus on piano transcription, integrating our module into multiple existing systems. Trained exclusively on the MAESTRO dataset, our method consistently reduces velocity errors and improves cross-dataset generalization to SMD and MAPS datasets. Under this training protocol, integrating our score-informed module with HPT (named Score-HPT) establishes a new state-of-the-art, outperforming existing score-informed methods and piano transcription models while adding only 1 M parameters. Furthermore, our framework remains robust to temporal misalignments caused by imperfect audio-score synchronization, a common real-world challenge.

eess.AS↗

Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening

Cough acoustics are promising for non-invasive tuberculosis (TB) screening, yet whether machine learning (ML) models capture disease-related acoustics or artifacts of data collection remains unresolved. We evaluated the cross-dataset generalizability of classical ML and deep learning (DL) cough-based TB classifiers across three independent datasets. Despite moderate within-dataset performance (ROC-AUC up to $0.76 \pm 0.06$), both pipelines fail to generalize, with external performance frequently below 0.6, indicating a possible limitation of the data. We further observed that audio representations are organized by recording device and dataset rather than TB status, predicted TB probability tracks country-level prevalence in CODA, and device mismatch degrades transfer while device-diverse training improves it. Additionally, a clinical-variable baseline generalizes more consistently (ROC-AUC $0.65-0.71$), indicating acquisition-specific variability in acoustics is a stronger driver of poor generalizability. High within-dataset performance is not enough. External validation is essential before cough-based TB models are clinically ready.

eess.AS↗

Personalized Causal Low-Bitrate Coded Speech Restoration with Speaker Embeddings

Legacy low-bitrate speech codecs such as AMR-NB remain widely used in telephony, where they restrict speech bandwidth and degrade listening quality. Coded speech restoration (CSR) aims to restore the speech bandwidth while improving the overall quality of coded speech, but typically without adapting to different speakers. In this paper, we systematically investigate personalization for restoration of speech coded at low bitrates, using a causal, low-complexity Universal Bandwidth Extension Generative Adversarial Network (UBGAN) model as the basis. We propose two personalization methods based on speaker embeddings, which incur little additional computational cost at inference: (i) conditioning a single CSR model on speaker embeddings, and (ii) cluster-based model selection, where multiple personalized models are trained on clusters of acoustically similar speakers. Experiments are performed on AMR-NB coded speech at 4.75 kbps. Experimental results with disjoint seen- and unseen-speaker test sets show that cluster-based model selection does not consistently outperform the non-personalized UBGAN baseline, whereas embedding conditioning significantly improves ViSQOL and WB-PESQ scores for both seen and unseen speakers.

eess.AS↗