Language independent speech recognition

• US 6,085,160 A
• Filed: 07/10/1998
• Issued: 07/04/2000
• Est. Priority Date: 07/10/1998
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Lernout & Hauspie Speech Products NV (Intel Corporation)
• Inventors: Van Compernolle, Dirk, D'hoore, Bart
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/113,589
• Time in Patent Office: 725 Days
• Field of Search: 704/251, 704/254, 704/255, 704/243, 704/256, 704/2, 704/277
• US Class Current: 704/256.2
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

A speech recognition system uses language independent acoustic models derived from speech data from multiple languages to represent speech units which are concatenated into words. In addition, the input speech signal which is compared to the language independent acoustic models may be vector quantized according to a codebook which is derived from speech data from multiple languages.

Claims

1. A language independent speech recognition system comprising:

a. a speech pre-processor which receives input speech and produces a speech-related signal representative of the input speech;

b. a database of acoustic hidden Markov models which represent subword units in each of a plurality of languages, wherein any subword unit that is common to two or more of the plurality of languages is represented by a single common acoustic hidden Markov model;

c. a language model which characterizes a vocabulary of recognizable words and a set of grammar rules; and

d. a speech recognizer which compares the speech-related signal to the acoustic hidden Markov models and the language model, and recognizes the input speech as a specific word sequence of at least one word.

2. A system according to claim 1, wherein the speech pre-processor comprises a feature extractor which extracts relevant speech parameters to produce the speech-related signal.

3. A system according to claim 2, wherein the feature extractor includes a codebook created using speech data from the plurality of languages, and wherein the feature extractor uses vector quantization such that the speech-related signal is a sequence of feature vectors.

4. A system according to claim 1, wherein the acoustic hidden Markov models are created using speech data from the plurality of languages.

5. A system according to claim 1, wherein the subword units are at least one of phonemes, parts of phonemes, and sequences of phonemes.

6. A system according to claim 1, wherein the vocabulary of recognizable words contains words in the plurality of languages.

7. A system according to claim 1, wherein the vocabulary of recognizable words contains foreign-loan words.

8. A system according to claim 1, wherein the vocabulary of recognizable words contains proper names in the plurality of languages.

9. A system according to claim 1, wherein the words in the vocabulary of recognizable words are described by a voice print comprised of a user-trained sequence of acoustic hidden Markov models from the database.

10. A system according to claim 9, further comprising:

e. a speaker identifier which uses the voice prints to determine the identity of the speaker of the speech input.

11. A system according to claim 1, wherein the vocabulary of recognizable words contains words in a language not present in the plurality of languages.

12. A system according to claim 1, wherein the speech recognizer compares the relevant speech parameters to acoustic models which represent subword units in a first language in the plurality of languages, and recognizes the speech input as a specific word sequence of at least one word in a second language in the plurality of languages so that input speech from a non-native speaker may be recognized.

13. A system according to claim 1, wherein the speech recognizer further provides an indication of how closely the input speech matches the words in the vocabulary of recognizable words, and if a specific pronunciation problem is present, assists a speaker of the input speech in correcting a pronunciation problem.

14. A method of language independent speech recognition comprising:

a. receiving input speech with a speech pre-processor and producing a speech-related signal representative of the input speech;

b. representing in a database of acoustic hidden Markov models, subword units in each of a plurality of languages, wherein any subword unit that is common to two or more of the plurality of languages is represented by a single common acoustic hidden Markov model;

c. characterizing in a language model a vocabulary of recognizable words and a set of grammar rules; and

d. comparing in a speech recognizer the speech-related signal to the acoustic hidden Markov models and the language model, and recognizing the input speech as a specific word sequence of at least one word.

15. A method according to claim 14, wherein the step of receiving uses a speech pre-processor which further comprises a feature extractor for extracting relevant speech parameters and producing the speech-related signal.

16. A method according to claim 15, wherein the feature extractor includes a codebook created using speech data from the plurality of languages, and wherein the feature extractor uses vector quantization such that the speech-related signal is a sequence of feature vectors.

17. A method according to claim 14, wherein the acoustic hidden Markov models are created using speech data from the plurality of languages.

18. A method according to claim 14, wherein the subword units are at least one of phonemes, parts of phonemes, and sequences of phonemes.

19. A method according to claim 14, wherein the vocabulary of recognizable words contains words in the plurality of languages.

20. A method according to claim 14, wherein the vocabulary of recognizable words contains foreign-loan words.

21. A method according to claim 14, wherein the vocabulary of recognizable words contains proper names in the plurality of languages.

22. A method according to claim 14, wherein the words in the vocabulary of recognizable words are described by a voice print comprised of a user-trained sequence of acoustic hidden Markov models from the database.

23. A method according to claim 22, further comprising

e. determining the identity of the speaker of the speech input with a speaker identifier which uses the voice prints.

24. A method according to claim 14, wherein the vocabulary of recognizable words contains words in a language not present in the plurality of languages.

25. A method according to claim 14, wherein the speech recognizer compares the relevant speech parameters to acoustic hidden Markov models which represent subword units in a first language in the plurality of languages, and recognizes the speech input as a specific word sequence of at least one word in a second language in the plurality of languages so that input speech from a non-native speaker may be recognized.

26. A method according to claim 14, wherein subpart (d) of the method further includes providing an indication of how closely the input speech matches the words in the vocabulary of recognizable words, and if a specific pronunciation problem is present, assists a speaker of the input speech in correcting a pronunciation problem.