Method and device for generating vocabulary entry from acoustic data

• US 8,751,230 B2
• Filed: 06/17/2009
• Issued: 06/10/2014
• Est. Priority Date: 06/27/2008
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Saffer, Zsolt
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US12/997,898
• Publication Number:
  US 20110093259A1
• Time in Patent Office: 1,819 Days
• Field of Search: 704/231, 704/251, 704/235, 704/257
• US Class Current: 704/243
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

A method and a device (1) for automatically generating vocabulary entry from input acoustic data (3), comprising a vocabulary entry type-specific acoustic phonetic transcription module (2; T) and a classifier module (6; 6′) for the classification of vocabulary entry types on the basis of the phonetic structure, wherein the classification of vocabulary entries is carried out in accordance with a number of predetermined types; and vocabulary entry type-specific phoneme-to-grapheme conversion means (28), to derive the respective vocabulary entries comprising a pair of a phonetic transcription and its grapheme form.

Claims

1. A method for automatically generating a vocabulary entry from input acoustic data, said method comprising:

performing, by a computer processor, a vocabulary entry type-specific acoustic phonetic transcription of the input acoustic data and a classification of vocabulary entry types based on phonetic structure, wherein the classification of vocabulary entries is carried out in accordance with a number of predetermined types;

performing, by the computer processor, a vocabulary entry type-specific phoneme-to-grapheme conversion, to derive the respective vocabulary entries comprising a pair of the phonetic transcription of the input acoustic data and its grapheme form; and

providing a select one of the vocabulary entries to a database for use in a speech processing application;

wherein the classification of vocabulary entries is carried out together with the vocabulary entry type-specific acoustic phonetic transcription in a combined step.

2. The method according to claim 1, wherein syllable language modeling is applied during vocabulary entry type-specific acoustic phonetic transcription.

3. The method according to claim 1, wherein the vocabulary entry types are hypothesized during acoustic phonetic transcription, with separate acoustic phonetic transcription being applied for each hypothesized vocabulary entry type.

4. The method according to claim 1, wherein for automatic vocabulary entry type classification a neural network based language identification is applied.

5. The method according to claim 4, wherein at first, the neural network based classification is carried out, whereafter the acoustic phonetic transcription is performed on the basis of the vocabulary entry type information.

6. The method according to claim 1, wherein the vocabulary entries are classified according to four types, namely vocabulary entries fitting to a given language morphology;

vocabulary entries fitting to morphology of at least one other supported language;

vocabulary entries which do not fit to any supported language morphology but are normally pronounced; and

vocabulary entries which do not fit to any supported language morphology and are pronounced by spelling.

7. The method according to claim 6, wherein a standard acoustic model is applied together with syllable language models with respect to vocabulary entries which fit to one of the supplied language morphologies, whereas a high resolution acoustic model is applied with respect to normal abbreviation type and a spelling acoustic model is applied with respect to spelled abbreviations.

13. A non-transitory computer readable medium that includes a computer program that comprises sections of software code for performing the method according to claim 1 when the computer program is executed on the computer.

8. A device for automatically generating a vocabulary entry from input acoustic data and comprising:

a vocabulary entry type classifier that determines a vocabulary entry type based on a phonetic structure of the input acoustic data, and

an acoustic phonetic transcriber that provides a phonetic sequence based on the input acoustic data and the vocabulary entry type, a vocabulary entry compositor that generates the vocabulary entry,

wherein the vocabulary entry type classifier is arranged to classify the input acoustic data in accordance with a number of predetermined vocabulary entry types, and

wherein the vocabulary entry compositor includes a statistical phoneme-to-grapheme converter that generates a grapheme based on the vocabulary entry type;

wherein the vocabulary entry type classifier includes gates for respective vocabulary entry types, the gates being arranged to output a respective vocabulary entry type information together with the phonetic transcription.

9. The device according to claim 8, wherein the acoustic phonetic transcriber includes at least one syllable language model.

10. The device according to claim 8, wherein the acoustic phonetic transcriber includes standard, high resolution and spelling acoustic model means.

11. The device according to claim 8, wherein the vocabulary entry type classifier includes a neural network that is arranged to output probability information for each predetermined vocabulary entry type together with the input acoustic data.

12. The device according to claim 8, wherein the acoustic phonetic transcriber includes an acoustic model selector associated with a plurality of acoustic models and a syllable language model selector associated with a plurality of syllable language models, both selectors being controlled at least in part by the vocabulary entry type.

14. A non-transitory computer readable medium that includes a program that, when executed by a processor, causes the processor to:

determine a vocabulary entry type based on a phonetic structure of an input acoustic data,

provide a phonetic sequence based on the input acoustic data and the vocabulary entry type, and

generate a vocabulary entry based on a statistically determined grapheme based on the phonetic sequence and the vocabulary entry type,

wherein determining the vocabulary entry type is determined from among a plurality of predetermined vocabulary entry types, and

the plurality of predetermined vocabulary entry types includes a natural language entry type and an abbreviation entry type.

15. The medium of claim 14, wherein providing the phonetic sequence includes use of a standard acoustic model and a high resolution model, depending upon the determined vocabulary input type.

16. The medium of claim 14, wherein determining the vocabulary entry type includes hypothesizing each of the predetermined vocabulary entry types and a corresponding confidence level based on the phonetic sequence using each predetermined vocabulary entry type.

17. The medium of claim 14, wherein the determining of the vocabulary entry type includes providing a form of the input acoustic data to a neural network.

18. The medium of claim 14, wherein the providing of the phonetic sequence includes applying one or more acoustic models based on the vocabulary entry type.

19. A non-transitory computer readable medium that includes a program that, when executed by a processor, causes the processor to:

determine a vocabulary entry type based on a phonetic structure of an input acoustic data,

provide a phonetic sequence based on the input acoustic data and the vocabulary entry type, and

generate a vocabulary entry based on a statistically determined grapheme based on the phonetic sequence and the vocabulary entry type,

wherein determining the vocabulary entry type is determined from among a plurality of predetermined vocabulary entry types, and

the providing of the phonetic sequence includes applying one of a plurality of acoustic models depending upon the vocabulary entry type.

20. The medium of claim 19, wherein the plurality of acoustic models include two or more of:

a natural acoustic model, a high-resolution acoustic model, and a spelling acoustic model.