Method and apparatus for processing spoken search queries

• US 8,239,366 B2
• Filed: 09/08/2010
• Issued: 08/07/2012
• Est. Priority Date: 09/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Sejnoha, Vladimir, Ganong, William F. III, Vozila, Paul J., Bodenstab, Nathan M., Tam, Yik-Cheung
• Primary Examiner(s): NGUYEN, PHONG H
• Application Number:
  US12/877,549
• Publication Number:
  US 20120059810A1
• Time in Patent Office: 699 Days
• Field of Search: 707/706, 707/707, 707/759, 707/760, 707/767, 707/810
• US Class Current: 707/707
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems

Abstract

Some embodiments relate to a method of performing a search for content on the Internet, in which a user may speak a search query and speech recognition may be performed on the spoken query to generate a text search query to be provided to a plurality of search engines. This enables a user to speak the search query rather than having to type it, and also allows the user to provide the search query only once, rather than having to provide it separately to multiple different search engines.

Claims

1. A method, implemented at least in part on a microprocessor, of performing a search for content on the Internet, the method comprising:

receiving voice input provided from a user; and

generating at least one text search query for a plurality of Internet-accessible search engines that search for content on the Internet, wherein the at least one text search query is generated, at least in part, by performing speech recognition on the voice input;

wherein the at least one text search query comprises at least two text search queries;

wherein the act of generating further comprises;

generating a first of the at least two text search queries, at least in part by performing speech recognition on the voice input using a first language model associated with a first of the plurality of search engines; and

generating a second of the at least two text search queries, at least in part by performing speech recognition on the voice input using a second language model, different from the first language model, associated with a second of the plurality of search engines;

wherein the first language model is one that was trained on content indexed by the first of the plurality of search engines;

wherein the first of the plurality of search engines is a site-specific search engine; and

wherein the second of the plurality of search engines is a general search engine.

2. The method of claim 1, further comprising:

issuing the first of the at least two text search queries to the first of the plurality of search engines; and

issuing the second of the at least two text search queries to the second of the plurality of search engines.

3. The method of claim 1, wherein the first of the at least two text search queries is different from the second of the at least two text search queries.

4. The method of claim 1, wherein the act of generating further comprises:

selecting one of a plurality of available language models to be used in performing automatic speech recognition on the voice input; and

performing automatic speech recognition on the voice input using the selected one of the plurality of language models.

5. The method of claim 4, wherein the act of selecting further comprises:

selecting the one of a plurality of available language models to be used in performing automatic speech recognition on the voice input based, at least in part, on the content of the voice input.

6. The method of claim 1, further comprising:

periodically updating the first language model and/or the second language model based on frequently searched terms.

7. The method of claim 1, further comprising:

updating the first language model and/or the second language model based on the user'"'"'s previous selection of search results.

8. The method of claim 1, wherein the plurality of Internet-accessible search engines each returns results selected from the group consisting of web pages hosted on one or more web sites or domains, content hosted on one or more web sites or domains, and hyperlinks to web pages and/or content hosted on one or more web sites or domains.

9. The method of claim 1, wherein the second language model is one that was trained on content indexed by the second of the plurality of search engines.

10. The method of claim 1, wherein the first language model is one that was trained on content related to the content indexed by the first of the plurality of search engines in addition to the content indexed by the first of the plurality of search engines.

11. The method of claim 1, wherein the first language model is one that was trained on content in addition to the content indexed by the first of the plurality of search engines.

12. The method of claim 1, wherein generating the first of the at least two search queries further comprises performing speech recognition on the voice input using a language model different from the first language model.

13. The method of claim 12, wherein the language model different from the first language model is a general language model.

14. The method of claim 13, wherein the language model different from the first language model is the second language model.

15. The method of claim 1, wherein generating the first of the at least two search queries further comprises:

performing speech recognition on the voice input using a different language model, different from the first language model, to generate a first recognition result for the voice input; and

determining a topic of the voice input by analyzing the first recognition result;

determining that the first language model is appropriate for the topic;

wherein performing the speech recognition on the voice input using the first language model generates a second recognition result for the voice input; and

wherein generating the first of the at least two search queries further comprises using the second recognition result to form the first of the at least two search queries.

16. The method of claim 15, wherein the different language model is a general language model.

17. The method of claim 15, wherein the different language model is the second language model.

18. The method of claim 1, wherein generating the second of the at least two search queries further comprises performing speech recognition on the voice input using a language model different from the second language model.

19. The method of claim 18, wherein the language model different from the second language model is a topic-specific language model.

20. The method of claim 18, wherein the language model different from the second language model is the first language model.

21. The method of claim 1, wherein generating the second of the at least two search queries further comprises:

performing speech recognition on the voice input using the second language model to generate a first recognition result for the voice input;

determining a topic of the voice input by analyzing the first recognition result;

determining that a topic-specific language model is appropriate for the topic;

performing speech recognition on the voice input using the topic-specific language model to generate a second recognition result for the voice input; and

wherein generating the second of the at least two search queries further comprises using the second recognition result to form the second of the at least two search queries.

22. The method of claim 21, wherein the topic-specific language model is the first language model.

23. The method of claim 21, wherein the second language model is a general language model.

24. The method of claim 21, wherein determining that the topic-specific language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the topic-specific language model.

25. The method of claim 21, wherein determining that the first language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the first language model.

26. The method of claim 1,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein generating the first of the at least two text search queries comprises using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

27. The method of claim 26, wherein the selected language model is a general language model.

28. The method of claim 26, wherein the selected language model is the first language model.

29. The method of claim 26, wherein the selected language model is the second language model.

30. The method of claim 1,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein generating the second of the at least two text search queries comprises using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

31. The method of claim 30, wherein the selected language model is a topic-specific language model.

32. The method of claim 30, wherein the selected language model is the first language model.

33. The method of claim 30, wherein the second language model is a general language model.

34. At least one tangible computer-readable medium encoded with instructions that, when executed by at least one hardware computer processor, perform a method of performing a search for content on the Internet, the method comprising:

receiving voice input provided from a user; and

generating at least one text search query for a plurality of Internet-accessible search engines that search for content on the Internet, wherein the at least one text search query is generated, at least in part, by performing speech recognition on the voice input;

wherein the at least one text search query comprises at least two text search queries;

wherein the act of generating further comprises;

generating a first of the at least two text search queries, at least in part by performing speech recognition on the voice input using a first language model associated with a first of the plurality of search engines; and

generating a second of the at least two text search queries, at least in part by performing speech recognition on the voice input using a second language model, different from the first language model, associated with a second of the plurality of search engines;

wherein the first language model is one that was trained on content indexed by the first of the plurality of search engines;

wherein the first of the plurality of search engines is a site-specific search engine; and

wherein the second of the plurality of search engines is a general search engine.

35. The at least one tangible computer-readable medium of claim 34, wherein the method further comprises:

issuing the first of the at least two text search queries to the first of the plurality of search engines; and

issuing the second of the at least two text search queries to the second of the plurality of search engines.

36. The at least one tangible computer-readable medium of claim 34, wherein the first of the at least two text search queries is different from the second of the at least two text search queries.

37. The at least one tangible computer-readable medium of claim 34, wherein the act of generating further comprises:

selecting one of a plurality of available language models to be used in performing automatic speech recognition on the voice input; and

performing automatic speech recognition on the voice input using the selected one of the plurality of language models.

38. The at least one tangible computer-readable medium of claim 37, wherein the act of selecting further comprises:

selecting the one of a plurality of available language models to be used in performing automatic speech recognition on the voice input based, at least in part, on the content of the voice input.

39. The at least one tangible computer-readable medium of claim 34, wherein the method further comprises:

periodically updating the first language model and/or the second language model based on frequently searched terms.

40. The at least one tangible computer-readable medium of claim 34, wherein the method further comprises:

updating the first language model and/or the second language model based on the user'"'"'s previous selection of search results.

41. The at least one computer-readable medium of claim 34, wherein the plurality of Internet-accessible search engines each returns results selected from the group consisting of web pages hosted on one or more web sites or domains, content hosted on one or more web sites or domains, and hyperlinks to web pages and/or content hosted on one or more web sites or domains.

42. The at least one computer-readable medium of claim 34, wherein the second language model is one that was trained on content indexed by the second of the plurality of search engines.

43. The at least one tangible computer readable medium of claim 34, wherein the first language model is one that was trained on content related to the content indexed by the first of the plurality of search engines in addition to the content indexed by the first of the plurality of search engines.

44. The at least one tangible computer readable medium of claim 34, wherein the first language model is one that was trained on content in addition to the content indexed by the first of the plurality of search engines.

45. The at least one tangible computer readable medium of claim 34, wherein generating the first of the at least two search queries further comprises performing speech recognition on the voice input using a language model different from the first language model.

46. The at least one tangible computer readable medium of claim 45, wherein the language model different from the first language model is a general language model.

47. The at least one tangible computer readable medium of claim 46, wherein the language model different from the first language model is the second language model.

48. The at least one tangible computer readable medium of claim 34, wherein generating the first of the at least two search queries further comprises:

performing speech recognition on the voice input using a different language model, different from the first language model, to generate a first recognition result for the voice input; and

determining a topic of the voice input by analyzing the first recognition result;

determining that the first language model is appropriate for the topic;

wherein performing the speech recognition on the voice input using the first language model generates a second recognition result for the voice input; and

wherein generating the first of the at least two search queries further comprises using the second recognition result to form the first of the at least two search queries.

49. The at least one tangible computer readable medium of claim 48, wherein the different language model is a general language model.

50. The at least one tangible computer readable medium of claim 48, wherein the different language model is the second language model.

51. The at least one tangible computer readable medium of claim 34, wherein generating the second of the at least two search queries further comprises performing speech recognition on the voice input using a language model different from the second language model.

52. The at least one tangible computer readable medium of claim 51, wherein the language model different from the second language model is a topic-specific language model.

53. The at least one tangible computer readable medium of claim 51, wherein the language model different from the second language model is the first language model.

54. The at least one tangible computer readable medium of claim 34, wherein generating the second of the at least two search queries further comprises:

performing speech recognition on the voice input using the second language model to generate a first recognition result for the voice input;

determining a topic of the voice input by analyzing the first recognition result;

determining that a topic-specific language model is appropriate for the topic;

performing speech recognition on the voice input using the topic-specific language model to generate a second recognition result for the voice input; and

wherein generating the second of the at least two search queries further comprises using the second recognition result to form the second of the at least two search queries.

55. The at least one tangible computer readable medium of claim 54, wherein the topic-specific language model is the first language model.

56. The at least one tangible computer readable medium of claim 54, wherein the second language model is a general language model.

57. The at least one tangible computer readable medium of claim 54, wherein determining that the topic-specific language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the topic-specific language model.

58. The at least one tangible computer readable medium of claim 54, wherein determining that the first language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the first language model.

59. The at least one tangible computer readable medium of claim 34,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein generating the first of the at least two text search queries comprises using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

60. The at least one tangible computer readable medium of claim 59, wherein the selected language model is a general language model.

61. The at least one tangible computer readable medium of claim 59, wherein the selected language model is the first language model.

62. The at least one tangible computer readable medium of claim 59, wherein the selected language model is the second language model.

63. The at least one tangible computer readable medium of claim 34,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein generating the second of the at least two text search queries comprises using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

64. The at least one tangible computer readable medium of claim 63, wherein the selected language model is a topic-specific language model.

65. The at least one tangible computer readable medium of claim 63, wherein the selected language model is the first language model.

66. The at least one tangible computer readable medium of claim 63, wherein the second language model is a general language model.

67. A system comprising:

at least one tangible memory that stores processor-executable instructions for performing a search for content via the Internet; and

at least one hardware computer processor, coupled to the at least one tangible memory, that executes the processor-executable instructions to;

receive voice input provided from a user; and

generate at least one text search query for a plurality of Internet-accessible search engines that search for content on the Internet, wherein the at least one text search query is generated, at least in part, by performing speech recognition on the voice input;

wherein the at least one text search query comprises at least two text search queries;

wherein the at least one hardware computer processor generates the at least one text search query by;

generating a first of the at least two text search queries, at least in part by performing speech recognition on the voice input using a first language model associated with a first of the plurality of search engines; and

generating a second of the at least two text search queries, at least in part, by performing speech recognition on the voice input using a second language model, different from the first language model, associated with a second of the plurality of search engines;

wherein the first language model is one that was trained on content indexed by the first of the plurality of search engines;

wherein the first of the plurality of search engines is a site-specific search engine; and

wherein the second of the plurality of search engines is a general search engine.

68. The system of claim 67, wherein the at least one hardware computer processor executes the processor-executable instructions to:

issue the first of the at least two text search queries to the first of the plurality of search engines; and

issue the second of the at least two text search queries to the second of the plurality of search engines.

69. The system of claim 67, wherein the first of the at least two text search queries is different from the second of the at least two text search queries.

70. The system of claim 67, wherein the at least one hardware processor generates the at least one text search query by:

selecting one of a plurality of available language models to be used in performing automatic speech recognition on the voice input; and

performing automatic speech recognition on the voice input using the selected one of the plurality of language models.

71. The system of claim 70, wherein the at least one hardware processor selects the one of the plurality of available language models by:

selecting the one of a plurality of available language models to be used in performing automatic speech recognition on the voice input based, at least in part, on the content of the voice input.

72. The system of claim 67, wherein the at least one hardware processor executes the processor-executable instructions to:

periodically update the first language and/or the second language model based on frequently searched terms.

73. The system of claim 67, wherein the at least one hardware processor executes the processor-executable instructions to:

update the first language model and/or the second language model based on the user'"'"'s previous selection of search results.

74. The system of claim 67, wherein the plurality of Internet-accessible search engines each returns results selected from the group consisting of web pages hosted on one or more web sites or domains, content hosted on one or more web sites or domains, and hyperlinks to web pages and/or content hosted on one or more web sites or domains.

75. The system of claim 67, wherein the second language model is one that was trained on content indexed by the second of the plurality of search engines.

76. The system of claim 67, wherein the first language model is one that was trained on content related to the content indexed by the first of the plurality of search engines in addition to the content indexed by the first of the plurality of search engines.

77. The system of claim 67, wherein the first language model is one that was trained on content in addition to the content indexed by the first of the plurality of search engines.

78. The system of claim 67, wherein the at least one hardware processor generates the first of the at least two search queries by performing speech recognition on the voice input using a language model different from the first language model.

79. The system of claim 78, wherein the language model different from the first language model is a general language model.

80. The system of claim 79, wherein the language model different from the first language model is the second language model.

81. The system of claim 67, wherein the at least one hardware processor generates the first of the at least two search queries at least by:

performing speech recognition on the voice input using a different language model, different from the first language model, to generate a first recognition result for the voice input; and

determining a topic of the voice input by analyzing the first recognition result;

determining that the first language model is appropriate for the topic;

wherein performing the speech recognition on the voice input using the first language model generates a second recognition result for the voice input; and

wherein the at least one hardware processor generates the first of the at least two search queries at least by using the second recognition result to form the first of the at least two search queries.

82. The system of claim 81, wherein the different language model is a general language model.

83. The system of claim 81, wherein the different language model is the second language model.

84. The system of claim 67, wherein the at least one hardware processor generates the second of the at least two search queries at least by performing speech recognition on the voice input using a language model different from the second language model.

85. The system of claim 84, wherein the language model different from the second language model is a topic-specific language model.

86. The system of claim 84, wherein the language model different from the second language model is the first language model.

87. The system of claim 67, wherein the at least one hardware processor generates the second of the at least two search queries at least by:

performing speech recognition on the voice input using the second language model to generate a first recognition result for the voice input;

determining a topic of the voice input by analyzing the first recognition result;

determining that a topic-specific language model is appropriate for the topic;

performing speech recognition on the voice input using the topic-specific language model to generate a second recognition result for the voice input; and

wherein the at least one hardware processor generates the second of the at least two search queries at least by using the second recognition result to form the second of the at least two search queries.

88. The system of claim 87, wherein the topic-specific language model is the first language model.

89. The system of claim 87, wherein the second language model is a general language model.

90. The system of claim 87, wherein determining that the topic-specific language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the topic-specific language model.

91. The system of claim 87, wherein determining that the first language model is appropriate for the topic is based, at least in part, on a score and/or confidence value associated with processing the voice input using the first language model.

92. The system of claim 67,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein the at least one hardware processor generates the first of the at least two text search queries at least by using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

93. The system of claim 92, wherein the selected language model is a general language model.

94. The system of claim 92, wherein the selected language model is the first language model.

95. The system of claim 92, wherein the selected language model is the second language model.

96. The system of claim 67,

wherein the voice input is processed by a plurality of language models comprising at least the first and second language models;

wherein the at least one hardware processor generates the second of the at least two text search queries at least by using a recognition result produced by a selected language model of the plurality of language models; and

wherein the selected language model is selected based, at least in part, on a score and/or confidence value associated with processing the voice input using the selected language model.

97. The system of claim 96, wherein the selected language model is a topic-specific language model.

98. The system of claim 96, wherein the selected language model is the first language model.

99. The system of claim 96, wherein the second language model is a general language model.