arXiv subjects

Shinji Watanabe

Publications and source records attributed to Shinji Watanabe.

At least 451 records · Page 25Linked to original sources

arXiv2020

End-to-End Automatic Speech Recognition Integrated With CTC-Based Voice Activity Detection

This paper integrates a voice activity detection (VAD) function with end-to-end automatic speech recognition toward an online speech interface and transcribing very long audio recordings. We focus on connectionist temporal classification (CTC) and its extension of CTC/attention architectures. As opposed to an attention-based architecture, input-synchronous label prediction can be performed based on a greedy search with the CTC (pre-)softmax output. This prediction includes consecutive long blank labels, which can be regarded as a non-speech region. We use the labels as a cue for detecting speech segments with simple thresholding. The threshold value is directly related to the length of a non-speech region, which is more intuitive and easier to control than conventional VAD hyperparameters. Experimental results on unsegmented data show that the proposed method outperformed the baseline methods using the conventional energy-based and neural-network-based VAD methods and achieved an RTF less than 0.2. The proposed method is publicly available.

eess.AS↗

arXiv2020

Speaker Diarization with Region Proposal Network

Speaker diarization is an important pre-processing step for many speech applications, and it aims to solve the "who spoke when" problem. Although the standard diarization systems can achieve satisfactory results in various scenarios, they are composed of several independently-optimized modules and cannot deal with the overlapped speech. In this paper, we propose a novel speaker diarization method: Region Proposal Network based Speaker Diarization (RPNSD). In this method, a neural network generates overlapped speech segment proposals, and compute their speaker embeddings at the same time. Compared with standard diarization systems, RPNSD has a shorter pipeline and can handle the overlapped speech. Experimental results on three diarization datasets reveal that RPNSD achieves remarkable improvements over the state-of-the-art x-vector baseline.

eess.AS↗

arXiv2020

End-to-End Multi-speaker Speech Recognition with Transformer

Recently, fully recurrent neural network (RNN) based end-to-end models have been proven to be effective for multi-speaker speech recognition in both the single-channel and multi-channel scenarios. In this work, we explore the use of Transformer models for these tasks by focusing on two aspects. First, we replace the RNN-based encoder-decoder in the speech recognition model with a Transformer architecture. Second, in order to use the Transformer in the masking network of the neural beamformer in the multi-channel case, we modify the self-attention component to be restricted to a segment rather than the whole sequence in order to reduce computation. Besides the model architecture improvements, we also incorporate an external dereverberation preprocessing, the weighted prediction error (WPE), enabling our model to handle reverberated signals. Experiments on the spatialized wsj1-2mix corpus show that the Transformer-based models achieve 40.9% and 25.6% relative WER reduction, down to 12.1% and 6.4% WER, under the anechoic condition in single-channel and multi-channel tasks, respectively, while in the reverberant case, our methods achieve 41.5% and 13.8% relative WER reduction, down to 16.5% and 15.2% WER.

eess.AS↗

arXiv2019

High-speed XYZ-nanopositioner for scanning ion conductance microscopy

We describe a tip-scan-type high-speed XYZ-nanopositioner designed for scanning ion conductance microscopy (SICM), with a special care being devoted to the way of nanopipette holding. The nanopipette probe is mounted in the center of a hollow piezoactuator, both ends of which are attached to identical diaphragm flexures, for Z-positioning. This design minimizes the generation of undesirable mechanical vibrations. Mechanical amplification is used to increase the XY-travel range of the nanopositioner. The first resonance frequencies of the nanopositioner are measured as $\sim$100 kHz and $\sim$2.3 kHz for the Z- and XY-displacements, respectively. The travel ranges are $\sim$6 $μ$m and $\sim$34 $μ$m for Z and XY, respectively. When this nanopositioner is used for hopping mode imaging of SICM with a $\sim$10-nm radius tip, the vertical tip velocity can be increased to 400 nm/ms; hence, the one-pixel acquisition time can be minimized to $\sim$1 ms.

physics.ins-det↗

arXiv2019

Development of High-Speed Ion Conductance Microscopy

Scanning ion conductance microscopy (SICM) can image the surface topography of specimens in ionic solutions without mechanical probe--sample contact. This unique capability is advantageous for imaging fragile biological samples but its highest possible imaging rate is far lower than the level desired in biological studies. Here, we present the development of high-speed SICM. The fast imaging capability is attained by a fast Z-scanner with active vibration control and pipette probes with enhanced ion conductance. By the former, the delay of probe Z-positioning is minimized to sub-10 us, while its maximum stroke is secured at 6 um. The enhanced ion conductance lowers a noise floor in ion current detection, increasing the detection bandwidth up to 100 kHz. Thus, temporal resolution 100-fold higher than that of conventional systems is achieved, together with spatial resolution around 20 nm.

physics.app-ph↗

arXiv2019

Non-Divergent Grüneisen Parameter in Quantum Critical Quasicrystal Yb$_{15}$Al$_{34}$Au$_{51}$: Reflection of Robustness of Quantum Criticality under Pressure

The mechanism of not diverging Grüneisen parameter in the quantum critical heavy-fermion quasicrystal (QC) Yb$_{15}$Al$_{34}$Au$_{51}$ is analyzed. We construct the formalism for calculating the specific heat $C_V(T)$, the thermal-expansion coefficient $α(T)$, and the Grüneisen parameter $Γ(T)$ near the quantum critical point of the Yb valence transition. By applying the framework to the QC, we calculate $C_V(T)$, $α(T)$, and $Γ(T)$, which explains the measurements. Not diverging $Γ(T)$ is attributed to the robustness of the quantum criticality in the QC under pressure. The difference in $Γ(T)$ at the lowest temperature between the QC and approximant crystal is shown to reflect the difference in the volume derivative of characteristic energy scales of the critical Yb-valence fluctuation and the Kondo temperature. Possible implications of our theory to future experiments are also discussed.

cond-mat.str-el↗

arXiv2019

Multilingual End-to-End Speech Translation

In this paper, we propose a simple yet effective framework for multilingual end-to-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-to-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-to-end ST problem. We show the effectiveness of multilingual end-to-end ST in two scenarios: one-to-many and many-to-many translations with publicly available data. We experimentally confirm that multilingual end-to-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic.

cs.CL↗

arXiv2019

Charge Transfer Effect under Odd-Parity Crystalline Electric Field

Charge-transfer effect under odd-parity crystalline electric field (CEF) is analyzed theoretically. In quantum-critical metal $β$-YbAlB$_4$, seven-fold configuration of B atoms surrounding Yb atom breaks local inversion symmetry at the Yb site, giving rise to the odd-parity CEF. Analysis of the CEF on the basis of hybridization picture shows that admixture of 4f and 5d wave functions at Yb with pure imaginary coefficient occurs, which makes magnetic-toroidal (MT) and electric-dipole (ED) degrees of freedom active. By constructing the minimal model for periodic crystal $β$-YbAlB$_4$, we show that the MT as well as ED fluctuation is divergently enhanced at the quantum critical point of valence transition simultaneously with critical valence fluctuations.

cond-mat.str-el↗

arXiv2019

Towards Online End-to-end Transformer Automatic Speech Recognition

The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks in end-to-end (E2E) automatic speech recognition (ASR) systems. However, Transformer has a drawback in that the entire input sequence is required to compute self-attention. We have proposed a block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. In this paper, we extend it towards an entire online E2E ASR system by introducing an online decoding process inspired by monotonic chunkwise attention (MoChA) into the Transformer decoder. Our novel MoChA training and inference algorithms exploit the unique properties of Transformer, whose attentions are not always monotonic or peaky, and have multiple heads and residual connections of the decoder layers. Evaluations of the Wall Street Journal (WSJ) and AISHELL-1 show that our proposed online Transformer decoder outperforms conventional chunkwise approaches.

eess.AS↗

arXiv2019

Grüneisen Parameter and Thermal Expansion near Magnetic Quantum Critical Points in Itinerant Electron Systems

Complete expressions of the thermal-expansion coefficient $α$ and the Grüneisen parameter $Γ$ are derived on the basis of the self-consistent renormalization (SCR) theory. By considering zero-point as well as thermal spin fluctuation under the stationary condition, the specific heat for each class of the magnetic quantum critical point (QCP) specified by the dynamical exponent $z=3$ (FM) and $z=2$ (AFM) and the spatial dimension ($d=3$ and $2$) is shown to be expressed as $C_{V}=C_a-C_b$, where $C_a$ is dominant at low temperatures, reproducing the past SCR criticality endorsed by the renormalization group theory. Starting from the explicit form of the entropy and using the Maxwell relation, $α=α_a+α_b$ (with $α_a$ and $α_b$ being related to $C_a$ and $C_b$, respectively) is derived, which is proven to be equivalent to $α$ derived from the free energy. The temperature-dependent coefficient found to exist in $α_b$, which is dominant at low temperatures, contributes to the crossover from the quantum-critical regime to the Curie-Weiss regime and even affects the quantum criticality at 2d AFM QCP. Based on these correctly calculated $C_{V}$ and $α$, Grüneisen parameter $Γ=Γ_a+Γ_b$ is derived, where $Γ_a$ and $Γ_b$ contain $α_a$ and $α_b$, respectively. The inverse susceptibility coupled to the volume $V$ in $Γ_b$ gives rise to divergence of $Γ$ at the QCP for each class even though characteristic energy scale of spin fluctuation $T_0$ is finite at the QCP, which gives a finite contribution in $Γ_a=-\frac{V}{T_0}\left(\frac{\partial T_0}{\partial V}\right)_{T=0}$. General properties of $α$ and $Γ$ including their signs as well as the relation to $T_0$ and the Kondo temperature in temperature-pressure phase diagrams of Ce- and Yb-based heavy electron systems are discussed.

cond-mat.str-el↗

arXiv2019

A practical two-stage training strategy for multi-stream end-to-end speech recognition

The multi-stream paradigm of audio processing, in which several sources are simultaneously considered, has been an active research area for information fusion. Our previous study offered a promising direction within end-to-end automatic speech recognition, where parallel encoders aim to capture diverse information followed by a stream-level fusion based on attention mechanisms to combine the different views. However, with an increasing number of streams resulting in an increasing number of encoders, the previous approach could require substantial memory and massive amounts of parallel data for joint training. In this work, we propose a practical two-stage training scheme. Stage-1 is to train a Universal Feature Extractor (UFE), where encoder outputs are produced from a single-stream model trained with all data. Stage-2 formulates a multi-stream scheme intending to solely train the attention fusion module using the UFE features and pretrained components from Stage-1. Experiments have been conducted on two datasets, DIRHA and AMI, as a multi-stream scenario. Compared with our previous method, this strategy achieves relative word error rate reductions of 8.2--32.4%, while consistently outperforming several conventional combination methods.

cs.CL↗

arXiv2019

Quantum Valence Criticality in Heavy Fermions on Periodic and Aperiodic Crystals

Progress of theories and experiments of the quantum valence criticality is overviewed focusing on recent discoveries of direct evidence of quantum critical point of valence transitions in periodic crystal $α$-YbAl$_{1-x}$Fe$_x$B$_4$ at $x=0.014$ and quasicrystal Yb$_{15}$(Al$_{1-x} $Ga$_{x}$)$_{34}$(Au$_{1-y}$Cu$_{y}$)$_{51}$ at $x=y=0$. The common quantum criticality as well as $T/B$ scaling behavior in the magnetization observed in the periodic crystal $β$-YbAlB$_4$ and quasicrystal Yb$_{15}$Al$_{34}$Au$_{51}$ is shown to be explained from the theory of critical Yb-valence fluctuations starting from the Hamiltonians for describing the low-energy electronic states in both systems.

cond-mat.str-el↗

arXiv2019

Grüneisen Parameter and Thermal Expansion by the Self-Consistent Renormalization Theory of Spin Fluctuations

The thermal expansion coefficient $α$ and the Grüneisen parameter $Γ$ near the magnetic quantum critical point (QCP) are derived on the basis of the self-consistent renormalization (SCR) theory of spin fluctuation. From the SCR entropy, the specific heat $C_{V}$, $α$, and $Γ$ are shown to be expressed in a simple form as $C_{V}=C_{\rm a}-C_{\rm b}$, $α=α_{\rm a}+α_{\rm b}$, and $Γ=Γ_{\rm a}+Γ_{\rm b}$, respectively, where $C_{\rm i}$, $α_{\rm i}$, and $Γ_{\rm i}$ $({\rm i}={\rm a}, {\rm b})$ are related with each other. As the temperature $T$ decreases, $C_{\rm a}$, $α_{\rm b}$, and $Γ_{\rm b}$ become dominant in $C_{V}$, $α$, and $Γ$, respectively. The inverse susceptibility of spin fluctuation coupled to the volume $V$ in $Γ_{\rm b}$ is found to give rise to the divergence of $Γ$ at the QCP for each class of ferromagnetism and antiferromagnetism (AFM) in spatial dimensions $d=3$ and $2$. This $V$-dependent inverse susceptibility in $α_{b}$ and $Γ_{\rm b}$ contributes to the $T$ dependences of $α$ and $Γ$, and even affects their criticality in the case of the AFM QCP in $d=2$. $Γ_{\rm a}$ is expressed as $Γ_{\rm a}(T=0)=-\frac{V}{T_0}\left(\frac{\partial T_0}{\partial V}\right)_{T=0}$ with $T_0$ being the characteristic temperature of spin fluctuation, which has an enhanced value in heavy electron systems.

cond-mat.str-el↗

arXiv2019

Multi-Stream End-to-End Speech Recognition

Attention-based methods and Connectionist Temporal Classification (CTC) network have been promising research directions for end-to-end (E2E) Automatic Speech Recognition (ASR). The joint CTC/Attention model has achieved great success by utilizing both architectures during multi-task training and joint decoding. In this work, we present a multi-stream framework based on joint CTC/Attention E2E ASR with parallel streams represented by separate encoders aiming to capture diverse information. On top of the regular attention networks, the Hierarchical Attention Network (HAN) is introduced to steer the decoder toward the most informative encoders. A separate CTC network is assigned to each stream to force monotonic alignments. Two representative framework have been proposed and discussed, which are Multi-Encoder Multi-Resolution (MEM-Res) framework and Multi-Encoder Multi-Array (MEM-Array) framework, respectively. In MEM-Res framework, two heterogeneous encoders with different architectures, temporal resolutions and separate CTC networks work in parallel to extract complimentary information from same acoustics. Experiments are conducted on Wall Street Journal (WSJ) and CHiME-4, resulting in relative Word Error Rate (WER) reduction of 18.0-32.1% and the best WER of 3.6% in the WSJ eval92 test set. The MEM-Array framework aims at improving the far-field ASR robustness using multiple microphone arrays which are activated by separate encoders. Compared with the best single-array results, the proposed framework has achieved relative WER reduction of 3.7% and 9.7% in AMI and DIRHA multi-array corpora, respectively, which also outperforms conventional fusion strategies.

eess.AS↗

arXiv2019

MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition

Recently, the end-to-end approach has proven its efficacy in monaural multi-speaker speech recognition. However, high word error rates (WERs) still prevent these systems from being used in practical applications. On the other hand, the spatial information in multi-channel signals has proven helpful in far-field speech recognition tasks. In this work, we propose a novel neural sequence-to-sequence (seq2seq) architecture, MIMO-Speech, which extends the original seq2seq to deal with multi-channel input and multi-channel output so that it can fully model multi-channel multi-speaker speech separation and recognition. MIMO-Speech is a fully neural end-to-end framework, which is optimized only via an ASR criterion. It is comprised of: 1) a monaural masking network, 2) a multi-source neural beamformer, and 3) a multi-output speech recognition model. With this processing, the input overlapped speech is directly mapped to text sequences. We further adopted a curriculum learning strategy, making the best use of the training set to improve the performance. The experiments on the spatialized wsj1-2mix corpus show that our model can achieve more than 60% WER reduction compared to the single-channel system with high quality enhanced signals (SI-SDR = 23.1 dB) obtained by the above separation function.

eess.AS↗

arXiv2019

Transformer ASR with Contextual Block Processing

The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks (RNNs) in end-to-end (E2E) automatic speech recognition (ASR) systems. However, the Transformer has a drawback in that the entire input sequence is required to compute self-attention. In this paper, we propose a new block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. We introduce a novel mask technique to implement the context inheritance to train the model efficiently. Evaluations of the Wall Street Journal (WSJ), Librispeech, VoxForge Italian, and AISHELL-1 Mandarin speech recognition datasets show that our proposed contextual block processing method outperforms naive block processing consistently. Furthermore, the attention weight tendency of each layer is analyzed to clarify how the added contextual inheritance mechanism models the global information.

eess.AS↗

arXiv2019

Espresso: A Fast End-to-end Neural Speech Recognition Toolkit

We present Espresso, an open-source, modular, extensible end-to-end neural automatic speech recognition (ASR) toolkit based on the deep learning library PyTorch and the popular neural machine translation toolkit fairseq. Espresso supports distributed training across GPUs and computing nodes, and features various decoding approaches commonly employed in ASR, including look-ahead word-based language model fusion, for which a fast, parallelized decoder is implemented. Espresso achieves state-of-the-art ASR performance on the WSJ, LibriSpeech, and Switchboard data sets among other end-to-end systems without data augmentation, and is 4--11x faster for decoding than similar systems (e.g. ESPnet).

cs.CL↗

arXiv2019

A Comparative Study on Transformer vs RNN in Speech Applications

Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS). This paper focuses on an emergent sequence-to-sequence model called Transformer, which achieves state-of-the-art performance in neural machine translation and other natural language processing applications. We undertook intensive studies in which we experimentally compared and analyzed Transformer and conventional recurrent neural networks (RNN) in a total of 15 ASR, one multilingual ASR, one ST, and two TTS benchmarks. Our experiments revealed various training tips and significant performance benefits obtained with Transformer for each task including the surprising superiority of Transformer in 13/15 ASR benchmarks in comparison with RNN. We are preparing to release Kaldi-style reproducible recipes using open source and publicly available datasets for all the ASR, ST, and TTS tasks for the community to succeed our exciting outcomes.

cs.CL↗