Multi-modal search wildcards

• US 8,090,738 B2
• Filed: 08/28/2008
• Issued: 01/03/2012
• Est. Priority Date: 05/14/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Paek, Timothy Seung Yoon, Thiesson, Bo, Ju, Yun-Cheng, Lee, Bongshin, Meek, Christopher A.
• Primary Examiner(s): Breene, John E
• Assistant Examiner(s): Bullock, Joshua
• Application Number:
  US12/200,625
• Publication Number:
  US 20090287681A1
• Time in Patent Office: 1,223 Days
• Field of Search: 707765-767, 704/243
• US Class Current: 707/765
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A multi-modal search system (and corresponding methodology) that employs wildcards is provided. Wildcards can be employed in the search query either initiated by the user or inferred by the system. These wildcards can represent uncertainty conveyed by a user in a multi-modal search query input. In examples, the words “something” or “whatchamacallit” can be used to convey uncertainty and partial knowledge about portions of the query and to dynamically trigger wildcard generation.

Claims

1. A system that facilitates multi-modal search, comprising:

a wildcard identification component that recognizes one or more pre-selected trigger words in a multi-modal speech input as triggering at least one wildcard symbol to be used for performing text-based searching using the multi-modal speech input; and

a search engine component that generates one or more text search queries, each text search query including the at least one wildcard symbol and one or more fixed words from the multi-modal speech input.

2. The system of claim 1, further comprising a wildcard generation component that replaces the one or more pre-selected trigger words with the at least one wildcard symbol.

3. The system of claim 2, wherein the wildcard identification component uses speech recognition to identify the one or more pre-selected trigger words in the multi-modal speech input.

4. The system of claim 2, wherein the one or more pre-selected trigger words include least one of ‘

something’

or ‘

whatchamacallit’

.

5. The system of claim 1, wherein the at least one wildcard symbol conveys uncertainty, and non-wildcard characters in the one or more fixed words represent constraints for the search engine component.

6. The system of claim 1, wherein each text search query is a regular expression query that includes the at least one wildcard symbol.

7. The system of claim 1, wherein the one or more search queries represent approximate matches to at least a portion of the multi-modal speech input.

8. The system of claim 1, further comprising a user interface that enables a user to associate the one or more pre-selected trigger words with the at least one wildcard symbol.

9. The system of claim 1, wherein the one or more search queries include an n-best list from a speech recognizer and a list of supplementary results including at least one of an exact match via a wildcard expression or an approximate match via information retrieval algorithms.

10. The system of claim 9, wherein at least a part of the n-best list obtained from the speech recognizer is submitted to an information retrieval algorithm that is indifferent to the order of words in the query.

11. The system of claim 10, wherein the information retrieval algorithm is an algorithm that is indifferent to word order in the query or is the Term Frequency Inverse Document Frequency (TFIDF) algorithm.

12. The system of claim 1, further comprising an artificial intelligence (AI) component that employs at least one of a probabilistic and a statistical-based analysis that infers an action that a user desires to be automatically performed.

13. A computer-implemented method of multi-modal search, comprising:

receiving a multimodal speech input from a user that includes one or more wildcard trigger words and one or more fixed words;

identifying the one or more wildcard trigger words present in the multi-modal speech input and replacing each wildcard trigger word in the multi-modal speech input with a corresponding wildcard symbol; and

establishing a wildcard text query expression that represents a desired search intent using one or more wildcard symbols and the one or more fixed words from the multi-modal speech input.

14. The computer-implemented method of claim 13, further comprising generating a plurality of search queries based upon the wildcard text query expression.

15. The computer-implemented method of claim 13, further comprising defining a correspondence between a wildcard trigger word and a wildcard symbol.

16. The computer-implemented method of claim 13, wherein the multi-modal speech input further includes a text hint that constrains the wildcard text query expression.

17. The computer-implemented method of claim 13, wherein the one or more wildcard trigger words include at least one of ‘

something’

or ‘

whatchamacallit’

.

18. A system, comprising:

a wildcard identification component that recognizes one or more pre-selected trigger words in a multi-modal input as triggering at least one wildcard symbol to be used for performing searches using the multi-modal input; and

a search engine component that generates one or more search queries, each search query including the at least one wildcard symbol and one or more fixed words from the multi-modal speech input.

19. The system of claim 18, wherein the at least one wildcard symbol conveys uncertainty, and non-wildcard characters in the one or more fixed words represent constraints for the one more search queries.

20. The computer-executable system of claim 18, wherein the one or more search queries represent approximate matches to at least a portion of the multi-modal input.