Methods and apparatus for use in speech recognition systems for identifying unknown words and for adding previously unknown words to vocabularies and grammars of speech recognition systems

• US 9,754,586 B2
• Filed: 06/05/2008
• Issued: 09/05/2017
• Est. Priority Date: 11/30/2005
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Deligne, Sabine, Gopinath, Ramesh A., Kanevsky, Dimitri, Viswanathan, Mahesh
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US12/133,762
• Publication Number:
  US 20080270136A1
• Time in Patent Office: 3,379 Days
• Field of Search: 704 1- 10, 704231-257
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

The present invention concerns methods and apparatus for identifying and assigning meaning to words not recognized by a vocabulary or grammar of a speech recognition system. In an embodiment of the invention, the word may be in an acoustic vocabulary of the speech recognition system, but may be unrecognized by an embedded grammar of a language model of the speech recognition system. In another embodiment of the invention, the word may not be recognized by any vocabulary associated with the speech recognition system. In embodiments of the invention, at least one hypothesis is generated for an utterance not recognized by the speech recognition system. If the at least one hypothesis meets at least one predetermined criterion, a sword or more corresponding to the at least one hypothesis is added to the vocabulary of the speech recognition system. In other embodiments of the invention, before adding the word to the vocabulary of the speech recognition system, the at least one hypothesis may be presented to the user of the speech recognition system to determine if that is what the used intended when the user spoke.

Claims

1. A speech recognition system configured for use with a portable electronic device including telephonic capabilities, the portable electronic device comprising a user interface configured to receive speech from a speaker as input, the speech recognition system comprising:

at least one processor programmed to;

determine that the received speech includes an item in an acoustic vocabulary of the speech recognition system, wherein the item is not associated with a valid command construct of an open embedded grammar of the speech recognition system, wherein the open embedded grammar includes a plurality of valid command constructs, each of which includes at least one command word for performing an action on the portable electronic device in combination with one or more words in the acoustic vocabulary of the speech recognition system;

determine a first confidence score for the item, wherein the first confidence score is determined based, at least in part, on an acoustic score for the item, wherein the acoustic score for the item is determined based, at least in part, on at least a portion of the received speech, wherein the first confidence score is determined based, at least in part, on a language model score for the item, wherein the language model score for the item is determined based, at least in part, on at least a portion of the received speech, and wherein the first confidence score is determined based, at least in part, on a sum of the acoustic score for the item and the language model score for the item;

determine a second confidence score for at least one word in a valid command construct of the open embedded grammar, wherein the second confidence score is determined based, at least in part, on an acoustic score for the at least one word in the valid command construct, wherein the acoustic score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, wherein the second confidence score is determined based, at least in part, on a language model score for the at least one word in the valid command construct, wherein the language model score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, and wherein and the second confidence score is determined based, at least in part, on a sum of the acoustic score for the at least one word in the valid command construct and the language model score for the at least one word in the valid command construct; and

add a new command construct to the open embedded grammar based, at least in part, on a comparison of the first confidence score and the second confidence score, wherein the new command construct includes the item.

2. The speech recognition system of claim 1, wherein the portable electronic device is a cellular telephone.

3. The speech recognition system of claim 1, wherein the at least one processor is further programmed to:

prior to adding the new command construct, request confirmation from the speaker that the new command construct corresponds to what the speaker intended, and wherein adding the new command construct is further based, at least in part, on receiving a confirmation from the speaker.

4. The speech recognition system of claim 3, wherein the confirmation comprises confirmation of a spelling of the item and/or confirmation of a pronunciation of the item.

5. The speech recognition system of claim 1, wherein the at least one processor is further programmed to:

instruct the speaker to illustrate the new command construct by inputting keystrokes, and wherein adding the new command construct is further based, at least in part, on the inputted keystrokes.

6. The speech recognition system of claim 1, wherein the at least one processor is further programmed to:

detect biometric information identifying that the received speech is associated with the new command construct, and wherein adding the new command construct is further based, at least in part, on the detected biometric information.

7. The speech recognition system of claim 6, wherein the biometric information comprises speech biometric information and/or data derived from video information.

8. The speech recognition system of claim 1, wherein determining that received speech includes an item in an acoustic vocabulary of the speech recognition system comprises determining that the acoustic score for the item is greater than a threshold value.

15. The method of claim 6, wherein the biometric information comprises speech biometric information and/or data derived from video information.

9. A method for use with a speech recognition system configured for use with a portable electronic device including telephonic capabilities, the portable electronic device comprising a user interface configured to receive speech from a speaker as input, the method comprising:

determining that the received speech includes an item in an acoustic vocabulary of the speech recognition system, wherein the item is not associated with a valid command construct of an open embedded grammar of the speech recognition system, wherein the open embedded grammar includes a plurality of valid command constructs, each of which includes at least one command word for performing an action on the portable electronic device in combination with one or more words in the acoustic vocabulary of the speech recognition system;

determining a first confidence score for the item, wherein the first confidence score is determined based, at least in part, on an acoustic score for the item, wherein the acoustic score for the item is determined based, at least in part, on at least a portion of the received speech, wherein the first confidence score is determined based, at least in part, on a language model score for the item, wherein the language model score for the item is determined based, at least in part, on at least a portion of the received speech, and wherein the first confidence score is determined based, at least in part, on a sum of the acoustic score for the item and the language model score for the item;

determining a second confidence score for at least one word in a valid command construct of the open embedded grammar, wherein the second confidence score is determined based, at least in part, on an acoustic score for the at least one word in the valid command construct, wherein the acoustic score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, wherein the second confidence score is determined based, at least in part, on a language model score for the at least one word in the valid command construct, wherein the language model score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, and wherein the second confidence score is determined based, at least in part, on a sum of the acoustic score for the at least one word in the valid command construct and the language model score for the at least one word in the valid command construct; and

adding a new command construct to the open embedded grammar based, at least in part, on a comparison of the first confidence score and the second confidence score, wherein the new command construct includes the item.

10. The method of claim 9, wherein the portable electronic device is a cellular telephone.

11. The method of claim 9, further comprising:

prior to adding the new command construct, request confirmation from the speaker that the new command construct corresponds to what the speaker intended, and wherein adding the new command construct is further based, at least in part, on receiving a confirmation from the speaker.

12. The method of claim 11, wherein the confirmation comprises confirmation of a spelling of the item and/or confirmation of a pronunciation of the item.

13. The method of claim 9, further comprising:

instructing the speaker to illustrate the new command construct by inputting keystrokes, and wherein adding the new command construct is further based, at least in part, on the inputted keystrokes.

14. The method of claim 9, further comprising:

detecting biometric information identifying that the received speech is associated with the new command construct, and wherein adding the new command construct is further based, at least in part, on the detected biometric information.

16. The method of claim 9, wherein determining that received speech includes an item in an acoustic vocabulary of the speech recognition system comprises determining that the acoustic score for the item is greater than a threshold value.