Recognizing multiple semantic items from single utterance

• US 20090228270A1
• Filed: 03/05/2008
• Published: 09/10/2009
• Est. Priority Date: 03/05/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chambers, Robert L., Odell, Julian J., Scholz, Oliver
• Granted Patent: US 8,725,492 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

Semantically distinct items are extracted from a single utterance by repeatedly recognizing the same utterance using constraints provided by semantic items already recognized. User feedback for selection or correction of partially recognized utterance may be used in a hierarchical, multi-modal, or single step manner. An accuracy of recognition is preserved while the less structured and more natural single utterance recognition form is allowed to be used.

Claims

1. A method to be executed at least in part in a computing device for recognizing multiple semantic items from a single utterance, the method comprising:

receiving a single utterance including at least two semantically distinct items from a user;

performing a speech recognition operation on the single utterance to recognize a first item of the at least two semantically distinct items;

determining a constraint based on the recognition of the first item; and

performing another speech recognition operation on the single utterance to recognize a second item of the at least two semantically distinct items based on the determined constraint.

2. The method of claim 1, further comprising:

if the semantically distinct items have a hierarchical relationship, employing the hierarchical relationship to apply the determined constraint on possible recognition values of the second item.

3. The method of claim 2, wherein applying the constraint includes selecting among a plurality of language models for recognizing the second item.

4. The method of claim 1, wherein the performing the speech recognition operation includes obtaining a plurality of alternative values for the first item.

5. The method of claim 4, further comprising:

providing the alternative values for the first item to the user;

receiving a user selection for one of the alternative values; and

determining the constraint based on the selected alternative value for the first item.

6. The method of claim 5, wherein providing the alternative values to the user includes one of:

a hierarchical presentation that includes the alternative value for the first item and a type for the second item;

a multi-modal presentation that includes a listing of alternative values for the first item;

a single step presentation that includes a combination of an alternative value for the first item and a value for the second item based on the alternative value for the first item selected according to a statistical language model; and

a visual menu presentation that includes a listing of combinations of the alternative values for the first item and values for the second item based on the alternative values for the first item selected according to the statistical language model.

7. The method of claim 4, further comprising:

determining constraints based on all alternative values for the first item simultaneously;

providing the alternative values for the first item to the user;

receiving a user selection for one of the alternative values; and

employing a constraint corresponding to the selected alternative value for the first item in recognizing the second item.

8. The method of claim 1, wherein the alternative values are provided to the user through one of an audio and a visual user interface presentation.

9. The method of claim 1, further comprising:

determining another constraint based on the recognition of the second item; and

performing a further speech recognition operation on the single utterance to recognize a third item of the at least two semantically distinct items based on the determined other constraint.

10. The method of claim 1, further comprising:

providing the recognized first item to the user;

receiving one of a user correction and a user confirmation for the provided first item; and

determining the constraint based on one of the user corrected and user confirmed first item.

11. The method of claim 10, further comprising:

performing at least one of;

additional speech recognition operations and application-specific operations associated with an application consuming the recognized first and second items while the recognized first item is being provided to the user and one of the user correction and the user confirmation is received.

12. A computing device for recognizing multiple semantic items from a single utterance, the computing device comprising:

a memory;

a processor coupled to the memory, the processor capable of executing a first application for speech recognition and a second application for consuming results of the speech recognition, wherein the first application is configured to;

receive a single utterance including at least two semantically distinct items from a user, the semantically distinct items comprising at least one from a set of;

words, phrases, and fragments;

process the single utterance to recognize a first item of the at least two semantically distinct items;

provide the recognized first item to the user for one of confirmation and correction;

receive one of the user correction and confirmation for the first item;

determine a specific language model based on the first item; and

process the single utterance again to recognize a second item of the at least two semantically distinct items applying the specific language model; and

wherein the second application is configured to;

in response to consuming the first item, provide input to the first application for the specific language model; and

in response to consuming the second item, provide feedback to the user based on a combination of the first and second items.

13. The computing device of claim 12, wherein the second application is one of:

a browsing application, a navigational assistance application, a directory assistance application, and a search engine.

14. The computing device of claim 12, wherein at least one of the first application and the second application are executed in a distributed manner over a plurality of computing devices communicating through a network.

15. The computing device of claim 14, wherein the second application is further configured to perform at least one of:

internal operations and communication operations while one of user correction and user confirmation is received.

16. The computing device of claim 12, wherein:

the first application is further configured to;

determine alternative values for the first item based on recognizing the first item;

provide the alternative values to the user and the second application;

receive input from the second application for specific language models associated with each of the alternative values;

receive a user selection for one of the alternative values; and

recognize the second item based on one of the specific language models associated with the selected alternative value for the first item.

17. The computing device of claim 16, wherein the second application is a web-based search application, the first item is a geographical location, and the second item is a business name.

18. A computer-readable storage medium with instructions stored thereon for recognizing multiple semantic items from a single utterance, the instructions comprising:

receiving a single utterance including a plurality of semantically distinct items from a user;

performing a plurality of speech recognition operations on the single utterance to recognize one of the plurality of semantically distinct items during each operation, wherein the semantically distinct items are hierarchically related, and wherein a statistical language model for recognizing one of the plurality of semantically distinct items is determined based on a prior recognition during each operation;

providing each recognized item to the user for one of confirmation and correction between recognition operations;

upon receiving one of user correction and confirmation, providing the recognized plurality of semantically distinct items to a web-based search application;

receiving input from the web-based search application;

providing the received input to the user.

19. The computer-readable storage medium of claim 18, wherein the semantically distinct items include at least two from a set of:

a geographical location, a business type, and a business name.

20. The computer-readable storage medium of claim 18, wherein the instructions further comprise:

providing alternative values for at least one of the semantically distinct items to the user;

receiving user selection of one of the alternative values; and

providing the selected alternative value to the web-based search application as the recognized item.