Centralized method and system for clarifying voice commands

• US 9,972,317 B2
• Filed: 12/08/2014
• Issued: 05/15/2018
• Est. Priority Date: 11/16/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mowatt, David, Chambers, Robert L., Andrew, Felix G. T. I.
• Primary Examiner(s): Ortiz Sanchez, Michael
• Application Number:
  US14/563,255
• Publication Number:
  US 20150095030A1
• Time in Patent Office: 1,254 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method and system for facilitating centralized interaction with a user includes providing a recognized voice command to a plurality of application modules. A plurality of interpretations of the voice command are generated by at least one of the plurality of application modules. A centralized interface module visually renders the plurality of interpretations of the voice command on a centralized display. An indication of selection of an interpretation is received from the user.

Claims

1. A computing device implemented method comprising:

receiving, at a module, first information corresponding to a first set of commands associated with a first application module;

receiving, at the module, second information corresponding to a second set of commands associated with a second application module;

obtaining, by the module, a voice command, recognized by processing a user input using a speech recognizer;

based at least in part on the module determining, using the first information, that the voice command includes at least one word corresponding to a first object or a first action included in at least a first command included in the first set of commands,

obtaining a first indication of the correspondence between the voice command and the first command;

based at least in part on the module determining, using the second information, that the voice command includes at least one word corresponding to a second object or a second action included in at least a second command included in the second set of commands,

obtaining a second indication of the correspondence between the voice command and the second command;

based at least in part on the first indication and the second indication, preparing a list of third information corresponding, respectively, to the first command associated with the first application module and the second command associated with the second application module;

visually rendering items in the list of third information on a display;

receiving an indication of a user selection of one of the items in the list of third information, wherein the selected item corresponds to a selected one of the first or second command; and

in response to the indication of the user selection, automatically executing the selected command associated with the first application module or the second application module.

2. The computing device implemented method of claim 1, and further comprising:

receiving a voice input from the user; and

processing, by the speech recognizer, the voice input to recognize the voice command.

3. The computing device implemented method of claim 2, wherein the module comprises an interface module that is communicatively coupled to the first application module, the second application module, and the speech recognizer.

4. The computing device implemented method of claim 1, wherein the module comprises a grammar generator module configured to generate a grammar based on the first and second information.

5. The computing device implemented method of claim 1, and further comprising determining that the voice command is ambiguous based on at least one of:

a determination that the voice command can be executed to perform more than one action;

or

a determination that more one than one different application module can execute the voice command; and

wherein the module obtains the first and second indications based on the determination that the voice command is ambiguous.

6. The computing device implemented method of claim 1, wherein each of the items in the list corresponds to an action performable by one of the first or second application modules.

7. The computing device implemented method of claim 6, wherein visually rendering the items in the list comprises:

for each item in the list, visually identifying which of the first or second application modules corresponds to the item.

8. The computing device implemented method of claim 7, and further comprising:

identifying an action, that corresponds to the item selected from the list;

identifying one of the first or second application module that corresponds to the identified action; and

providing an indication of the identified action to the identified application module.

9. A computing system comprising:

a processor; and

memory storing instructions executable by the processor, wherein the instructions, based at least on execution, configure the computing system to;

receive first information corresponding to a first set of commands associated with a first application module;

receive second information corresponding to a second set of commands associated with a second application module;

store the first and second information in a data store;

identify a voice command received from a user and recognized based at least in part on using a speech recognizer;

based at least in part on a determination, using the stored first information, that the voice command includes at least one word corresponding to a first object or a first action included in at least a first command included in the first set of commands,

obtain a first indication of the correspondence between the voice command and the first command;

based at least in part on a determination, using the stored second information, that the voice command includes at least one word corresponding to a second object or a second action included in at least a second command included in the second set of commands,

obtain a second indication of the correspondence between the voice command and the second command;

based at least in part on the first indication and the second indication, prepare a list of third information corresponding, respectively, to the first command associated with the first application module and the second command associated with the second application module;

visually render items in the list of third information on a display;

for each item in the visually rendered list, visually identify on the display which of the first or second application modules corresponds to the item;

receive an indication of a user selection of one of the items in the list of third information, wherein the selected item corresponds to a selected one of the first or second command; and

based on the indication of the user selection, automatically execute the selected command.

10. The computing system of claim 9, wherein the instructions configure the computing system to:

receive a voice input from a user; and

process the voice input using a speech recognizer to recognize the voice command.

11. The computing system of claim 9, wherein the instructions configure the computing system to:

generate a grammar based on the first and second information; and

recognize the voice command based on the grammar.

12. The computing system of claim 9, wherein the instructions configure the computing system to:

determine that the voice command is ambiguous, wherein the first and second indications are obtained based on the determination that the voice command is ambiguous.

13. The computing system of claim 12, wherein the voice command is determined to be ambiguous based on at least one of:

a determination that the voice command can be executed to perform more than one action;

or

a determination that more than one different application module can execute the voice command.

14. The computing system of claim 9, wherein each of the items in the list corresponds to an action performable by one of the first or second application modules.

15. The computing system of claim 9, wherein the instructions configure the computing system to:

identify an action that corresponds to the item selected from the list;

identify one of the first or second application module that corresponds to the identified action; and

provide an indication of the identified action to the identified application module.

16. A computer-implemented method comprising:

receiving, at an interface module, first information corresponding to a first set of commands associated with a first application module;

receiving, at the interface module, second information corresponding to a second set of commands associated with a second application module;

obtaining a voice command recognized using a speech recognizer that processes a speech input from a user;

obtaining, based on the first information, a first indication of a correspondence between the voice command and a first command associated with the first application module, wherein the first indication is based on a determination that the voice command includes at least one word corresponding to a first object or a first action included in the first command;

obtaining, based on the second information, a second indication of a correspondence between the voice command and a second command associated with the second application module, wherein the second indication is based on a determination that the voice command includes at least one word corresponding to a second object or a second action included in the second command;

based on the first and second indications, generating a list of items, each item in the list corresponding to at least one of the first or second commands;

rendering the list of items to the user;

receiving an indication of a user selection of one of the items in the list, wherein the selected item corresponds to a selected one of the first or second command; and

in response to the indication of the user selection, automatically executing the selected command associated with the first application module or the second application module.

17. The computer-implemented method of claim 16, wherein each of the items in the list is associated with an action performable by one of the first or second application modules, and wherein rendering the list of items comprises:

generating a user interface display that visually renders the list and, for each item in the list, visually identifies which of the first or second application modules corresponds to the action that is associated with the item.

18. The computer-implemented method of claim 17, and further comprising:

identifying an action that corresponds to the item selected from the list;

identifying one of the first or second application module that corresponds to the identified action; and

sending an indication of the identified action to the identified application module.

19. The computer-implemented method of claim 17, wherein the user interface display includes a user interface element that identifies, for each item in the list, which of the first or second application modules corresponds to the item.

20. The computer-implemented method of claim 16, and further comprising:

determining that the voice command is ambiguous based on at least one of;

the voice command being executable to perform more than one action;

or

the voice command being executable by more than one different application module; and

wherein the voice command is provided to the first and second application modules based on the determination.