Natural language speech recognition using slot semantic confidence scores related to their word recognition confidence scores

• US 6,567,778 B1
• Filed: 06/23/1999
• Issued: 05/20/2003
• Est. Priority Date: 12/21/1995
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Jackson, Eric G., Chao Chang, Eric I
• Primary Examiner(s): SMITS, TALIVALDIS IVARS
• Application Number:
  US09/339,470
• Time in Patent Office: 1,427 Days
• Field of Search: 704/257, 704/275
• US Class Current: 704/257
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A stream of input speech is coupled as an input to a speech recognizer. The speech can be provided to the speech recognizer directly from a user or first stored and provided from a memory circuit. Each input word is recognized by the speech recognizer and a word confidence score is associated with each corresponding recognized word. The recognized words and their associated word confidence scores are provided to a natural language interpreter which parses the stream of recognized words into predetermined edges. From the edges, the natural language interpreter forms semantic slots which represent a semantic meaning. A slot confidence score related to the word or phone confidence scores for each of the words in the slot is determined for each slot. Based upon the slot confidence score, an ancillary application program determines whether to accept the words used to fill each slot. If the slot is rejected, the application program can request the user to repeat the information necessary to fill that slot only, rather than requiring the user to repeat the entire stream of input speech.

Claims

1. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. parsing the recognized words and deriving at least one unit of information regarding a predetermined subject; and

c. forming a semantic confidence score for each unit of information, wherein the semantic confidence score is related to the word confidence scores for each of the words that unit of information is derived from.

2. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. parsing the recognized words and then forming at least one unit of information, wherein the recognized words in each unit of information comprise information regarding a predetermined subject; and

c. forming a semantic confidence score for each unit of information, wherein the semantic confidence score is related to the word confidence scores for each of the words in that unit of information.

3. The method according to claim 2 wherein each unit of information comprises a slot.

4. The method according to claim 3 wherein each slot is filled with a slot value.

5. The method according to claim 3 further comprising the step of varying the recognized words in each unit of information to determine a crucial word for filling a slot.

6. The method according to claim 3 wherein each slot is filled with the words for that unit of information.

7. The method according to claim 2 further comprising the step of varying the recognized words in each unit of information to determine a crucial word for filling a unit of information.

8. The method according to claim 2 further comprising the step of determining whether any semantic confidence score fails to exceed a predetermined threshold.

9. The method according to claim 8 wherein each unit of information can have a unique predetermined threshold.

10. The method according to claim 8 further comprising the step of querying a user regarding information from a failed unit of information.

11. The method according to claim 2 wherein the step of parsing is performed in accordance with a grammar having a predetermined set of grammar rules.

12. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. parsing the recognized words and then forming at least one unit of information, wherein the recognized words in each unit of information comprise information regarding a predetermined subject; and

c. forming a semantic confidence score for each unit of information, wherein the semantic confidence score is related to a confidence associated with the phones for each of the words in that unit of information.

13. The method according to claim 12 wherein each unit of information comprises a slot.

14. The method according to claim 13 wherein each slot is filled with a slot value.

15. The method according to claim 13 wherein each slot is filled with the words for that unit of information.

16. The method according to claim 12 further comprising the step of determining whether any confidence score fails to exceed a predetermined threshold.

17. The method according to claim 16 wherein each unit of information can have a unique predetermined threshold.

18. The method according to claim 16 further comprising the step of querying a user regarding information from a failed unit of information.

19. The method according to claim 12 wherein the step of parsing is performed in accordance with a grammar having a predetermined set of grammar rules.

20. A method of interpreting natural language comprising the steps of:

a. providing a speech recognizer to receive an input stream of speech from a user and to provide a stream of recognized words therefrom, wherein each recognized word has a word confidence score;

b. providing a natural language interpreter to receive the stream of recognized words, wherein each word is associated with its word confidence score;

c. using the natural language interpreter to parse the stream of recognized words into a plurality of units of information in accordance with predetermined grammar rules; and

d. forming a semantic confidence score for each unit of information, wherein the semantic confidence score is related to the word confidence scores for each of the words in that unit of information.

21. The method according to claim 20 wherein each unit of information comprises a slot.

22. The method according to claim 21 wherein each slot is filled with a slot value.

23. The method according to claim 21 wherein each slot is filled with the words for that unit of information.

24. The method according to claim 20 further comprising the step of determining whether any semantic confidence score fails to exceed a predetermined threshold.

25. The method according to claim 24 wherein each unit of information can have a unique predetermined threshold.

26. The method according to claim 24 further comprising the step of querying a user regarding information from a failed unit of information.

27. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a stream of input speech from a user;

b. storing the input speech in a memory circuit forming a stored speech;

c. providing the stored speech to a speech recognition system for generating a stream of recognized words;

d. forming a word confidence score for each of the recognized words;

e. providing the recognized words and their associated word confidence score to a natural language interpreter;

f. parsing the recognized words into a plurality of units of information, wherein the recognized words in each unit of information comprise information regarding a predetermined subject;

g. forming a semantic confidence score for each unit of information, wherein the semantic confidence score is related to the word confidence scores for each of the words in that unit of information;

h. determining whether any semantic confidence score fails to exceed a predetermined threshold; and

i. querying the user regarding information from a failed unit of information.

28. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a stream of input speech from a user;

b. providing the stored speech to a speech recognition system for generating a stream of recognized words;

c. forming a word confidence score for each of the recognized words;

d. providing the recognized words and their associated word confidence score to a natural language interpreter;

e. parsing a value for the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

f. forming a semantic confidence score for each slot, wherein the semantic confidence score is related to the word confidence scores for each of the words in that slot;

g. determining whether any semantic confidence score fails to exceed a predetermined threshold; and

h. querying the user regarding information from a failed slot.

29. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. means for parsing a value for the recognized words into at least one slot, wherein the values in each slot comprise information regarding a predetermined subject; and

c. means for forming a semantic confidence score for each slot, wherein the semantic confidence score is related to the word confidence scores for each of the words in that slot.

30. The apparatus according to claim 29 further comprising means for determining whether any semantic confidence score fails to exceed a predetermined threshold.

31. The method according to claim 30 wherein each slot can have a unique predetermined threshold.

32. The apparatus according to claim 30 further comprising means for querying a user regarding information from a failed slot.

33. The apparatus according to claim 29 wherein the means for parsing works in accordance with a grammar having a predetermined set of grammar rules.

34. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. means for parsing a value for the recognized words into at least one slot, wherein the values in each slot comprise information regarding a predetermined subject; and

c. means for forming a semantic confidence score for each slot, wherein the semantic confidence score is related to confidence score associated with the phones for each of the words in that slot.

35. The apparatus according to claim 34 further comprising means for determining whether any semantic confidence score fails to exceed a predetermined threshold.

36. The method according to claim 35 wherein each slot can have a unique predetermined threshold.

37. The apparatus according to claim 35 further comprising means for querying a user regarding information from a failed slot.

38. The apparatus according to claim 34 wherein the means for parsing works in accordance with a grammar having a predetermined set of grammar rules.

39. An apparatus for interpreting natural language comprising:

a. a speech recognizer for receiving an input stream of speech from a user and for providing a stream of recognized words therefrom, wherein each recognized word has a word confidence score;

b. a natural language interpreter for receiving the stream of recognized words, wherein each word is associated with its word confidence score;

c. the natural language interpreter further comprising means for parsing values for the stream of recognized words into a plurality of slots in accordance with predetermined grammar rules; and

d. means for forming a semantic confidence score for each slot, wherein the semantic confidence score is related to the word confidence scores for each of the words in that slot.

40. The apparatus according to claim 39 further comprising means for determining whether any semantic confidence score fails to exceed a predetermined threshold.

41. The apparatus according to claim 40 further comprising means for querying a user regarding information from a failed slot.

42. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a stream of input speech from a user;

b. means for storing the input speech in a memory circuit forming a stored speech;

c. means for providing the stored speech to a speech recognition system for generating a stream of recognized words;

d. means for forming a word confidence score for each of the recognized words;

e. means for providing the recognized words and their associated word confidence score to a natural language interpreter;

f. means for parsing values for the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

g. means for forming a semantic confidence score for each slot, wherein the semantic confidence score is related to the word confidence scores for each of the words in that slot;

h. means for determining whether any semantic confidence score fails to exceed a predetermined threshold; and

i. means for querying the user regarding information from a failed slot.

43. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a stream of input speech from a user;

b. means for providing the stored speech to a speech recognition system for generating a stream of recognized words;

c. means for forming a word confidence score for each of the recognized words;

d. means for providing the recognized words and their associated word confidence score to a natural language interpreter;

e. means for parsing values for the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

f. means for forming a semantic confidence score for each slot, wherein the semantic confidence score is related to the word confidence scores for each of the words in that slot;

g. means for determining whether any semantic confidence score fails to exceed a predetermined threshold; and

h. means for querying the user regarding information from a failed slot.

44. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. parsing the recognized words and then forming at least one slot, wherein the recognized words in each slot comprise information regarding a predetermined subject; and

c. forming a slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot.

45. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. parsing the recognized words and then forming at least one slot, wherein the recognized words in each slot comprise information regarding a predetermined subject; and

c. forming a slot confidence score for each slot, wherein the slot confidence score is related to a confidence associated with the phones for each of the words in that slot.

46. A method of interpreting natural language comprising the steps of:

a. providing a speech recognizer to receive an input stream of speech from a user and to provide a stream of recognized words therefrom, wherein each recognized word has a word confidence score;

b. providing a natural language interpreter to receive the stream of recognized words, wherein each word is associated with its word confidence score;

c. using the natural language interpreter to parse the stream of recognized words into a plurality of slots in accordance with predetermined grammar rules; and

d. forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot.

47. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a stream of input speech from a user;

b. storing the input speech in a memory circuit forming a stored speech;

c. providing the stored speech to a speech recognition system for generating a stream of recognized words;

d. forming a word confidence score for each of the recognized words;

e. providing the recognized words and their associated word confidence score to a natural language interpreter;

f. parsing the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

g. forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot;

h. determining whether any slot confidence score fails to exceed a predetermined threshold; and

i. querying the user regarding information from a failed slot.

48. A method of interpreting natural language received as speech comprising the steps of:

a. receiving a stream of input speech from a user;

b. providing the stored speech to a speech recognition system for generating a stream of recognized words;

c. forming a word confidence score for each of the recognized words;

d. providing the recognized words and their associated word confidence score to a natural language interpreter;

e. parsing the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

f. forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot;

g. determining whether any slot confidence score fails to exceed a predetermined threshold; and

h. querying the user regarding information from a failed slot.

49. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. means for parsing the recognized words into at least one slot, wherein the recognized words in each slot comprise information regarding a predetermined subject; and

c. means for forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot.

50. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a plurality of recognized words, wherein each recognized word has a word confidence score;

b. means for parsing the recognized words into at least one slot, wherein the recognized words in each slot comprise information regarding a predetermined subject; and

c. means for forming an slot confidence score for each slot, wherein the slot confidence score is related to confidence score associated with the phones for each of the words in that slot.

51. An apparatus for interpreting natural language comprising:

a. a speech recognizer for receiving an input stream of speech from a user and for providing a stream of recognized words therefrom, wherein each recognized word has a word confidence score;

b. a natural language interpreter for receiving the stream of recognized words, wherein each word is associated with its word confidence score;

c. the natural language interpreter further comprising means for parsing the stream of recognized words into a plurality of slots in accordance with predetermined grammar rules; and

d. means for forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot.

52. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a stream of input speech from a user;

b. means for storing the input speech in a memory circuit forming a stored speech;

c. means for providing the stored speech to a speech recognition system for generating a stream of recognized words;

d. means for forming a word confidence score for each of the recognized words;

e. means for providing the recognized words and their associated word confidence score to a natural language interpreter;

f. means for parsing the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

g. means for forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot;

h. means for determining whether any slot confidence score fails to exceed a predetermined threshold; and

i. means for querying the user regarding information from a failed slot.

53. An apparatus for interpreting natural language received as speech comprising the steps of:

a. means for receiving a stream of input speech from a user;

b. means for providing the stored speech to a speech recognition system for generating a stream of recognized words;

c. means for forming a word confidence score for each of the recognized words;

d. means for providing the recognized words and their associated word confidence score to a natural language interpreter;

e. means for parsing the recognized words into a plurality of slots, wherein the recognized words in each slot comprise information regarding a predetermined subject;

f. means for forming an slot confidence score for each slot, wherein the slot confidence score is related to the word confidence scores for each of the words in that slot;

g. means for determining whether any slot confidence score fails to exceed a predetermined threshold; and

h. means for querying the user regarding information from a failed slot.