Recognizing multiple semantic items from single utterance

• US 8,725,492 B2
• Filed: 03/05/2008
• Issued: 05/13/2014
• Est. Priority Date: 03/05/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chambers, Robert L., Scholz, Oliver, Odell, Julian J
• Primary Examiner(s): Harper, Vincent P
• Application Number:
  US12/042,460
• Publication Number:
  US 20090228270A1
• Time in Patent Office: 2,260 Days
• Field of Search: 704/9, 704/251, 704/257, 704/275
• US Class Current: 704/9
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

Semantically distinct items are extracted from a single utterance by repeatedly recognizing the same utterance using constraints provided by semantic items already recognized. User feedback for selection or correction of partially recognized utterance may be used in a hierarchical, multi-modal, or single step manner. An accuracy of recognition is preserved while the less structured and more natural single utterance recognition form is allowed to be used.

Claims

1. A method to be executed at least in part in a computing device for recognizing multiple semantic items from a single utterance, the method comprising:

receiving a single utterance including at least two semantically distinct items from a user;

performing a speech recognition operation on the single utterance to recognize a first item of the at least two semantically distinct items;

determining a constraint based on the recognition of the first item;

performing another speech recognition operation on the single utterance to recognize a second item of the at least two semantically distinct items based on the determined constraint, wherein performing the speech recognition operation includes obtaining a plurality of alternative values for the first item;

providing the alternative values for the first item to the user; and

receiving a user selection for one of the alternative values, wherein providing the alternative values to the user includes one of;

a single step presentation that includes a combination of an alternative value for the first item and a value for the second item based on the alternative value for the first item selected according to a statistical language model; and

a visual menu presentation that includes a listing of combinations of the alternative values for the first item and values for the second item based on the alternative values for the first item selected according to the statistical language model.

2. The method of claim 1, further comprising:

if the semantically distinct items have a hierarchical relationship, employing the hierarchical relationship to apply the determined constraint on possible recognition values of the second item.

3. The method of claim 2, wherein applying the constraint includes selecting among a plurality of language models for recognizing the second item.

4. The method of claim 1, further comprising:

determining the constraint based on the selected alternative value for the first item.

5. The method of claim 1, wherein the alternative values are provided to the user through one of an audio and a visual user interface presentation.

6. The method of claim 1, further comprising:

determining another constraint based on the recognition of the second item; and

performing a further speech recognition operation on the single utterance to recognize a third item of the at least two semantically distinct items based on the determined other constraint.

7. The method of claim 1, further comprising:

providing the recognized first item to the user;

receiving one of a user correction and a user confirmation for the provided first item; and

determining the constraint based on one of the user corrected and user confirmed first item.

8. The method of claim 7, further comprising:

performing at least one of;

additional speech recognition operations and application-specific operations associated with an application consuming the recognized first and second items while the recognized first item is being provided to the user and one of the user correction and the user confirmation is received.

9. A computing device for recognizing multiple semantic items from a single utterance, the computing device comprising:

a memory;

a processor coupled to the memory, the processor capable of executing a first application for speech recognition and a second application for consuming results of the speech recognition, wherein the first application is configured to;

receive a single utterance including at least two semantically distinct items from a user, the semantically distinct items comprising at least one from a set of;

words, phrases, and fragments;

process the single utterance to recognize a first item of the at least two semantically distinct items;

provide the recognized first item to the user for one of confirmation and correction;

receive one of the user correction and confirmation for the first item;

determine a specific language model based on the first item;

process the single utterance again to recognize a second item of the at least two semantically distinct items applying the specific language model;

determine alternative values for the first item based on recognizing the first item;

provide the alternative values to the user and the second application;

receive input from the second application for specific language models associated with each of the alternative values;

receive a user selection for one of the alternative values; and

recognize the second item based on one of the specific language models associated with the selected alternative value for the first item; and

wherein the second application is configured to;

in response to consuming the first item, provide input to the first application for the specific language model; and

in response to consuming the second item, provide feedback to the user based on a combination of the first and second items.

10. The computing device of claim 9, wherein the second application is one of:

a browsing application, a navigational assistance application, a directory assistance application, and a search engine.

11. The computing device of claim 9, wherein at least one of the first application and the second application are executed in a distributed manner over a plurality of computing devices communicating through a network.

12. The computing device of claim 11, wherein the second application is further configured to perform at least one of:

internal operations and communication operations while one of user correction and user confirmation is received.

13. The computing device of claim 9, wherein the second application is a web-based search application, the first item is a geographical location, and the second item is a business name.

14. The computing device of claim 9, wherein the first application is further configured to:

derive a constraint from the recognition of the first item; and

apply the constraint to the second item utilizing repeated recognitions.

15. A computer-readable storage device with computer-executable instructions stored thereon which, when executed by a computer, perform a method for recognizing multiple semantic items from a single utterance, the method comprising:

receiving a single utterance including a plurality of semantically distinct items from a user;

performing a plurality of speech recognition operations on the single utterance to recognize one of the plurality of semantically distinct items during each operation, wherein performing the plurality of speech recognition operations includes obtaining a plurality of alternative values for a first item in the plurality of semantically distinct items wherein the semantically distinct items are hierarchically related, and wherein a statistical language model for recognizing one of the plurality of semantically distinct items is determined based on a prior recognition during each operation;

providing each recognized item to the user for one of confirmation and correction between recognition operations;

upon receiving one of user correction and confirmation, providing the recognized plurality of semantically distinct items to a web-based search application;

receiving input from the web-based search application;

providing the received input to the user;

determining constraints based on all alternative values for the first semantically distinct item simultaneously;

providing the alternative values for the first semantically distinct item to the user, wherein providing the alternative values for the first semantically distinct item to the user includes one of;

a single step presentation that includes a combination of an alternative value for the first semantically distinct item and a value for a second semantically distinct item based on the alternative value for the first semantically distinct item selected according to a statistical language model; and

a visual menu presentation that includes a listing of combinations of the alternative values for the first semantically distinct item and values for the second semantically distinct item based on the alternative values for the first semantically distinct item selected according to the statistical language model;

receiving a user selection for one of the alternative values; and

employing a constraint corresponding to the selected alternative value for the first semantically distinct item in recognizing the second semantically distinct item.

16. The computer-readable storage device of claim 15, wherein the semantically distinct items include at least two from a set of:

a geographical location, a business type, and a business name.

17. The computer-readable storage device of claim 15, further comprising:

providing the selected alternative value to the web-based search application as the recognized item.