Query expansion and weighting based on results of automatic speech recognition

• US 6,856,957 B1
• Filed: 02/07/2001
• Issued: 02/15/2005
• Est. Priority Date: 02/07/2001
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Dumoulin, Benoit
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/779,023
• Time in Patent Office: 1,469 Days
• Field of Search: 704/231, 704/251, 704/257, 704/275, 704/270.1, 704/272, 704/256, 379/88.01, 379/88.02, 379/88.03
• US Class Current: 704/257
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A technique for identifying one or more items from amongst a plurality of items in response to a spoken utterance is used to improve call routing and information retrieval systems which employ automatic speech recognition (ASR). An automatic speech recognizer is used to recognize the utterance, including generating a plurality of hypotheses for the utterance. A query element is then generated for use in identifying one or more items from amongst the plurality of items. The query element includes a set of values representing two or more of the hypotheses, each value corresponding to one of the words in the hypotheses. Each value in the query element is then weighted based on hypothesis confidence, word confidence, or both, as determined by the ASR process. The query element is then applied to the plurality of items to identify one or more items which satisfy the query.

Claims

1. A method of identifying one or more items from amongst a plurality of items in response to a spoken utterance, the method comprising:

using an automatic speech recognizer to recognize the utterance, including generating a plurality of hypotheses for the utterance; and

generating a query element based on the utterance, for use in identifying one or more items from amongst the plurality of items, such that the query element includes values representing two or more hypotheses of the plurality of hypotheses.

2. A method as recited in claim 1, wherein the query element includes values representing a best hypothesis and a hypothesis other than the best hypothesis from the plurality of hypotheses.

3. A method as recited in claim 1, wherein the query element includes values representing all of the plurality of hypotheses.

4. A method as recited in claim 1, wherein the query element is a vector.

5. A method as recited in claim 1, wherein each of the hypotheses includes one or more words, wherein the query element includes a set of values, each value corresponding to one of said words, and wherein the method further comprises weighting each of the values in the query element based on a confidence measure of the hypothesis that includes the word corresponding to said value.

6. A method as recited in claim 5, wherein the confidence measure of each hypothesis is based on a rank of said hypothesis among the plurality of hypotheses.

7. A method as recited in claim 5, wherein the method further comprises weighting each of the values in the query element based on a confidence measure of the word corresponding to said value.

8. A method as recited in claim 1, further comprising applying the query element to the plurality of items to identify one or more items from amongst the plurality of items.

9. A method as recited in claim 8, wherein each of the items is a destination in a call routing system.

10. A method as recited in claim 9, wherein each of the items is a dataset in an information retrieval system.

11. A method as recited in claim 1, wherein the plurality of items are items of text data.

12. A method as recited in claim 1, wherein the plurality of items are items of audio data.

13. A method of identifying one or more items from amongst a plurality of items in response to a spoken utterance, the method comprising:

using an automatic speech recognizer to recognize the utterance, including generating a plurality of hypotheses for the utterance, wherein each of the hypotheses includes one or more words;

generating a query element for use in identifying one or more items from amongst the plurality of items, wherein the query element includes a set of values representing all of the plurality of hypotheses, each value corresponding to one of said words; and

weighting each of the values in the query element based on a confidence measure of the hypothesis that includes the word corresponding to said value, wherein the confidence measure of each hypothesis is based on a rank of the hypothesis among the plurality of hypotheses.

14. A method as recited in claim 13, further comprising weighting each of the values in the query element based on a confidence measure of the word corresponding to said value.

15. A method as recited in claim 13, further comprising applying the query element to the plurality of items to identify one or more items from amongst the plurality of items.

16. A method as recited in claim 15, wherein each of the items is a destination in a call routing system.

17. A method as recited in claim 15, wherein each of the items is a dataset in a database in an information retrieval system.

18. A method as recited in claim 13, wherein the plurality of items are items of text data.

19. A method as recited in claim 13, wherein the plurality of items are items of audio data.

20. An apparatus for identifying one or more items from amongst a plurality of items in response to a spoken utterance, the apparatus comprising:

means for using an automatic speech recognizer to recognize the utterance, including generating a plurality of hypotheses for the utterance; and

means for generating a set of values representing a query, for use in identifying one or more items from amongst the plurality of items, the set of values including values representing a best hypothesis and a hypothesis other than the best hypothesis from the plurality of hypotheses.

21. An apparatus as recited in claim 20, wherein the set of values includes values representing all of the plurality of hypotheses.

22. An apparatus as recited in claim 20, wherein each of the hypotheses includes one or more words, wherein each value of the set of values corresponds to one of said words, and wherein the apparatus further comprises means for weighting each of the values based on a confidence measure of the hypothesis that includes the word corresponding to said value.

23. An apparatus as recited in claim 22, wherein the confidence measure of each hypothesis is based on a rank of the hypothesis among the plurality of hypotheses.

24. An apparatus as recited in claim 22, wherein the apparatus further comprises means for weighting each of the values in the set of values based on a confidence measure of the word corresponding to said value.

25. An apparatus as recited in claim 20, further comprising means for applying the set of values to the plurality of items to identify one or more items from amongst the plurality of items.

26. An apparatus as recited in claim 25, wherein the apparatus is part of a call routing system, such that each of the plurality of items is a call destination.

27. An apparatus as recited in claim 25, wherein the apparatus is part of an information retrieval system, such that each of the plurality of items is a dataset in a database of the information retrieval system.

28. An apparatus as recited in claim 20, wherein the plurality of items are items of text data.

29. An apparatus as recited in claim 20, wherein the plurality of items are items of audio data.

30. An information retrieval system comprising:

a database;

an information retrieval engine to identify and retrieve one or more items from the database which satisfy a text-based query; and

an automatic speech recognizer to generate the query in response to an utterance of a user, the automatic speech recognizer configured to;

generate a plurality of hypotheses for the utterance; and

generate a query element representing the query, the query element including values representing two or more hypotheses of the plurality of hypotheses.

31. An information retrieval system as recited in claim 30, wherein the query element includes values representing all of the plurality of hypotheses.

32. An information retrieval system as recited in claim 31, wherein each of the hypotheses includes one or more words, wherein each value in the query element corresponds to one of said words, and wherein the method further comprises weighting each of the values in the query element based on a confidence measure of the hypothesis that includes the corresponding word.

33. An information retrieval system as recited in claim 32, wherein the confidence measure of each hypothesis is based on a rank of the hypothesis among the plurality of hypotheses.

34. An information retrieval system as recited in claim 32, wherein the automatic speech recognizer is further configured to weight each of the values in the query element based on a confidence measure of the word corresponding to said value.

35. An information retrieval system as recited in claim 30, wherein the information retrieval engine uses the query to retrieve text data satisfying the query from the database.

36. An information retrieval system as recited in claim 30, wherein the information retrieval engine uses the query to retrieve audio data satisfying the query from the database.

37. A call routing system comprising:

a database;

a call routing engine to identify and provide a caller with access to a call destination which satisfies a text-based query; and

an automatic speech recognizer to generate the query in response to an utterance of the caller, the automatic speech recognizer configured to;

generate a plurality of hypotheses for the utterance; and

generate a query element representing the query, the query element including values representing two or more hypotheses of the plurality of hypotheses.

38. A call routing system as recited in claim 37, wherein the query element includes values representing all of the plurality of hypotheses.

39. A call routing system as recited in claim 38, wherein each of the hypotheses includes one or more words, wherein each value in the query element corresponds to one of said words, and wherein the method further comprises weighting each of the values in the query element based on a confidence measure of the hypothesis that includes the corresponding word.

40. A call routing system as recited in claim 39, wherein the confidence measure of each hypothesis is based on a rank of the hypothesis among the plurality of hypotheses.

41. A call routing system as recited in claim 39, wherein the automatic speech recognizer is further configured to weight each of the values in the query element based on a confidence measure of the word corresponding to said value.

42. A call routing system as recited in claim 37, wherein the information retrieval engine uses the query to retrieve text data satisfying the query from the database.

43. A call routing system as recited in claim 37, wherein the information retrieval engine uses the query to retrieve audio data satisfying the query from the database.