Acoustic Model Adaptation Methods Based on Pronunciation Variability Analysis for Enhancing the Recognition of Voice of Non-Native Speaker and Apparatus Thereof

• US 20090119105A1
• Filed: 03/30/2007
• Published: 05/07/2009
• Est. Priority Date: 03/31/2006
• Status: Active Grant

• Current Assignee: Gwangju Institute of Science and Technology
• Original Assignee: Gwangju Institute of Science and Technology
• Inventors: Kim, Hong Kook, Yoon, Jae Sam, Oh, Yoo Rhee
• Granted Patent: US 8,515,753 B2
• Time in Patent Office: Days
• US Class Current: 704/244
• CPC Class Codes: G10L 15/07 to the speaker

Abstract

The example embodiment of the present invention provides an acoustic model adaptation method for enhancing recognition performance for a non-native speaker'"'"'s speech. In order to adapt acoustic models, first, pronunciation variations are examined by analyzing a non-native speaker'"'"'s speech. Thereafter, based on variation pronunciation of a non-native speaker'"'"'s speech, acoustic models are adapted in a state-tying step during a training process of acoustic models. When the present invention for adapting acoustic models and a conventional acoustic model adaptation scheme are combined, more-enhanced recognition performance can be obtained. The example embodiment of the present invention enhances recognition performance for a non-native speaker'"'"'s speech while reducing the degradation of recognition performance for a native speaker'"'"'s speech.

Claims

1. An acoustic model adaptation method comprising the steps of:

a pronunciation variation analysis step for examining a variation pronunciation of a non-native speaker through pronunciation analysis of a non-native speaker'"'"'s speech; and

a step for adapting an acoustic model by using the analyzed variation pronunciation so that the acoustic model may be adapted for the non-native speaker'"'"'s speech.

2. The acoustic model adaptation method as claimed in claim 1, wherein the pronunciation variation analysis step comprises the steps of:

creating a speech recognition system which has been trained by a native speaker'"'"'s speech;

making the speech recognition system recognize a non-native speaker'"'"'s speech, and then creating a monophone confusion matrix; and

obtaining the variation pronunciation by analyzing the monophone confusion matrix.

3. The acoustic model adaptation method as claimed in claim 2, wherein the step for creating the speech recognition system which has been trained by the native speaker'"'"'s speech comprises the steps of:

creating monophone-based acoustic models by using training data by a native speaker;

expanding the monophone-based acoustic models to triphone-based acoustic models and then training the acoustic models by using the training data by the native speaker;

reducing a number of triphone-based acoustic models by using a state-tying scheme; and

increasing a mixture density of the state-tied triphone-based acoustic models.

4. The acoustic model adaptation method as claimed in claim 3, wherein the step of expanding the monophone-based acoustic models to the triphone-based acoustic models and then training the acoustic models by using the training data by the native speaker comprises the steps of:

locating all triphone-based acoustic models having a central phone (b) of a triphone (a-b+c) in a parent node of a decision tree;

locating each of the triphone-based acoustic models, which has been located in the parent node, in a corresponding terminal node through a decision questionnaire; and

tying the triphone-based acoustic models located in terminal nodes as one representative acoustic model.

5. The acoustic model adaptation method as claimed in claim 2, wherein, in the monophone confusion matrix, a row array includes pronunciations which must be recognized, and a column array includes pronunciations recognized from a non-native speaker'"'"'s speech.

6. The acoustic model adaptation method as claimed in claim 2, wherein the obtaining the variation pronunciation by analyzing the recognition result is achieved by taking an element having a largest value among elements of the confusion matrix.

7. The acoustic model adaptation method as claimed in claim 1, wherein the step for adapting an acoustic model by using the analyzed variation pronunciation so that the acoustic model may be adapted for the non-native speaker'"'"'s speech comprises the steps of:

creating monophone-based acoustic models by using training data by a native speaker, and then creating triphone-based acoustic models;

state-tying the created triphone-based acoustic models according to whether there is pronunciation variation by the non-native speaker; and

increasing a mixture density of the state-tied triphone-based acoustic models.

8. The acoustic model adaptation method as claimed in claim 7, wherein, when there is no pronunciation variation by the non-native speaker, a state-tying process, which has been used in the step of creating the speech recognition system having been trained by the native speaker'"'"'s speech, is used for state-tying the created triphone-based acoustic models.

9. The acoustic model adaptation method as claimed in claim 7, wherein, when there is pronunciation variation by the non-native speaker, the state-tying the created triphone-based acoustic models comprises the steps of:

locating all triphone-based acoustic models each of which has a variation pronunciation (b′

) by a non-native speaker as a central phone (b′

) thereof, as well as all triphone-based acoustic

models each of which has a monophone to be state-tied as a central phone (b) thereof, in a parent node; and

disposing each of the triphone-based acoustic models, which have been located in the parent node, in a corresponding terminal node, and tying the triphone-based acoustic models as one representative acoustic model.

10. An apparatus for recognizing a non-native speaker'"'"'s speech by using acoustic models which have been created according to the acoustic model adaptation method claimed in claim 1.