arXiv Science⌕ Search

arXiv subjects

Tatiana Timofeeva

Publications and source records attributed to Tatiana Timofeeva.

3 recordsLinked to original sources

Nemotron 3 Diarization: Advancing the Accuracy-Efficiency Frontier in Streaming Speaker Diarization

Streaming Sortformer introduced an arrival-order speaker cache to maintain speaker identities across chunks in end-to-end streaming speaker diarization, with a single model supporting configurable latency profiles spanning offline operation to ultra-low-latency streaming. Nemotron 3 Diarization addresses the original model's four-speaker limitation by extending support to eight speakers while improving both accuracy and computational efficiency. The previous FastConformer-Transformer stack is streamlined into a pure Transformer encoder. A learned upsampling layer then maps the encoder's coarse 80-ms representations to 10-ms resolution, enabling fine-grained predictions. Training leverages large-scale simulated mixtures and adds activity and phantom-speaker losses to the standard Sortformer objectives. Despite doubling speaker capacity over its predecessor, the new architecture reduces training memory by a factor of 2.8, makes training $2.2\times$ faster, and delivers up to $6.4\times$ higher inference throughput. Nemotron 3 Diarization establishes a new state of the art in DER on DIHARD III and CALLHOME. When used to condition DiCoW, it also achieves state-of-the-art tcpWER on the single-microphone conditions of AMI and NOTSOFAR-1. Training code is open source, and model weights are released under the permissive OpenMDW-1.1 license.

eess.AS↗

STCON System for the CHiME-8 Challenge

This paper describes the STCON system for the CHiME-8 Challenge Task 1 (DASR) aimed at distant automatic speech transcription and diarization with multiple recording devices. Our main attention was paid to carefully trained and tuned diarization pipeline and speaker counting. This allowed to significantly reduce diarization error rate (DER) and obtain more reliable segments for speech separation and recognition. To improve source separation, we designed a Guided Target speaker Extraction (G-TSE) model and used it in conjunction with the traditional Guided Source Separation (GSS) method. To train various parts of our pipeline, we investigated several data augmentation and generation techniques, which helped us to improve the overall system quality.

eess.AS↗

Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario

Speaker diarization for real-life scenarios is an extremely challenging problem. Widely used clustering-based diarization approaches perform rather poorly in such conditions, mainly due to the limited ability to handle overlapping speech. We propose a novel Target-Speaker Voice Activity Detection (TS-VAD) approach, which directly predicts an activity of each speaker on each time frame. TS-VAD model takes conventional speech features (e.g., MFCC) along with i-vectors for each speaker as inputs. A set of binary classification output layers produces activities of each speaker. I-vectors can be estimated iteratively, starting with a strong clustering-based diarization. We also extend the TS-VAD approach to the multi-microphone case using a simple attention mechanism on top of hidden representations extracted from the single-channel TS-VAD model. Moreover, post-processing strategies for the predicted speaker activity probabilities are investigated. Experiments on the CHiME-6 unsegmented data show that TS-VAD achieves state-of-the-art results outperforming the baseline x-vector-based system by more than 30% Diarization Error Rate (DER) abs.

eess.AS↗