Discovering capabilities of third-party voice-enabled resources

• US 9,837,081 B2
• Filed: 12/30/2014
• Issued: 12/05/2017
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kannan, Vishwac Sena, Uzelac, Aleksandar, Campbell, Jonathan, Hwang, Daniel J.
• Primary Examiner(s): Mishra, Richa
• Application Number:
  US14/586,449
• Publication Number:
  US 20160189717A1
• Time in Patent Office: 1,071 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

Techniques are described for discovering capabilities of voice-enabled resources. A voice-controlled digital personal assistant can respond to user requests to list available voice-enabled resources that are capable of performing a specific task using voice input. The voice-controlled digital personal assistant can also respond to user requests to list the tasks that a particular voice-enabled resource can perform using voice input. The voice-controlled digital personal assistant can also support a practice mode in which users practice voice commands for performing tasks supported by voice-enabled resources.

Claims

1. A computing device comprising:

a processing unit;

memory; and

one or more microphones;

the computing device configured with a voice-controlled digital personal assistant to perform operations for discovering capabilities of third-party voice-enabled resources, the operations comprising;

receiving, via the one or more microphones, a digital voice input generated by a user;

performing natural language processing using the digital voice input to determine a user voice request, wherein the user voice request is a request for available third-party voice-enabled resources installed on the computing device that are capable of performing a task;

identifying one or more third-party voice-enabled resources that are capable of performing the task using voice input, wherein the one or more third-party voice-enabled resources are identified using a data structure that defines tasks supported by available third-party voice-enabled resources using voice input, wherein the data structure comprises;

information identifying voice commands supported by the available third-party voice-enabled resources;

information identifying voice command variations supported by the available third-party voice-enabled resources that define variations of user voice input that will perform the voice command, wherein at least one of the voice commands supports a plurality of different voice command variations; and

information identifying voice command examples supported by the available third-party voice-enabled resources, wherein the one or more voice command examples are specific examples of user voice input that, if spoken by the user, will activate the voice command, wherein at least one of the voice commands has a plurality of different voice command examples that activate the voice command;

providing a response to the user identifying the one or more third-party voice-enabled resources that are capable of performing the task;

receiving a user-initiated command, wherein the user-initiated command identifies a specific third-party voice-enabled application from the one or more third-party voice-enabled resources, wherein the user-initiated command instructs the voice-controlled digital personal assistant to delete a voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure, wherein the deleted voice command, and corresponding voice command variations, cannot be performed by user voice input; and

performing the user-initiated command to delete the voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure.

2. The computing device of claim 1 wherein the identifying the one or more third-party voice-enabled resources that are capable of performing the task comprises:

evaluating third-party voice-enabled applications installed on the computing device.

3. The computing device of claim 1 wherein the identifying the one or more third-party voice-enabled resources that are capable of performing the task comprises:

evaluating third-party voice-enabled resources available from a remote web service.

4. The computing device of claim 1 wherein providing the response to the user comprises, for each of the one or more third-party voice-enabled resources that are capable of performing the task:

displaying an indication of at least one voice command variation supported by the third-party voice-enabled resource that will perform the task; and

displaying an indication of at least one voice command example for performing the task.

5. The computing device of claim 1 the operations further comprising:

receiving a user-initiated command, wherein the user-initiated command identifies a specific third-party voice-enabled resource from the one or more third-party voice-enabled resources and a request to practice tasks supported by the specific third-party voice-enabled resource.

6. The computing device of claim 1 the operations further comprising:

receiving a user-initiated command, wherein the user-initiated command identifies a specific third-party voice-enabled resource from the one or more third-party voice-enabled resources and a request to practice a specific voice command variation for performing a specific task supported by the specific third-party voice-enabled resource.

7. The computing device of claim 1 wherein the task is a task for playing a movie, wherein the one or more third-party voice-enabled resources are capable of playing a movie on the computing device, and wherein the response to the user lists the one or more third-party voice-enabled resources that are capable of playing a movie.

8. A method, implemented by a computing device comprising a microphone, for discovering capabilities of third-party voice-enabled resources, the method comprising:

receiving, by a voice-controlled digital personal assistant, a digital voice input generated by a user, wherein the digital voice input is received via the microphone

performing natural language processing using the digital voice input to determine a user voice request, wherein the user voice request is a request for available tasks capable of being performed by a specific third-party voice-enabled resource;

identifying, by the voice-controlled digital personal assistant, the available tasks capable of being performed by the specific third-party voice-enabled resource, wherein the available tasks are identified using a data structure that defines the available tasks supported by the specific third-party voice-enabled resource using voice input, wherein the specific third-party voice-enabled resource is a specific third-party voice-enabled application, and wherein the data structure comprises;

information identifying each of the available tasks supported by the specific third-party voice-enabled application;

information identifying voice command variations supported by the specific third-party voice-enabled application that define variations of user voice input that will perform the task, wherein at least one of the tasks supports a plurality of different voice command variations; and

information identifying voice command examples supported by the specific third-party voice-enabled application, wherein the voice command examples are specific examples of user voice input that, if spoken by the user, will activate the task, wherein at least one of the tasks has a plurality of different voice command examples that activate the task;

providing, by the voice-controlled digital personal assistant, a response to the user identifying each of the available tasks capable of being performed by the specific third-party voice-enabled resource;

receiving a user-initiated command, wherein the user-initiated command identifies the specific third-party voice-enabled application, wherein the user-initiated command instructs the voice-controlled digital personal assistant to delete a voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure, wherein the deleted voice command, and corresponding voice command variations, cannot be performed by user voice input; and

performing the user-initiated command to delete the voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure.

9. The method of claim 8 wherein the specific third-party voice-enabled resource is a remote web resource, the method further comprising:

retrieving the data structure from a remote web site.

10. The method of claim 8 wherein providing the response to the user comprises, for each of one or more tasks of the available tasks capable of being performed by the specific third-party voice-enabled resource:

displaying, by the voice-controlled digital personal assistant, an indication of at least one voice command variation supported by the specific third-party voice-enabled resource that will perform the task; and

displaying, by the voice-controlled digital personal assistant, an indication of at least one voice command example for performing the task.

11. A computer-readable storage medium storing computer-executable instructions for causing a computing device to perform operations for discovering capabilities of third-party voice-enabled resources, the operations comprising:

receiving, by a voice-controlled digital personal assistant, a user request to practice voice commands provided by a third-party voice-enabled resource;

receiving, by the voice-controlled digital personal assistant, a digital voice input generated by the user, wherein the digital voice input is received via a microphone of the computing device;

performing natural language processing using the digital voice input to determine a user voice request;

determining, by the voice-controlled digital personal assistant, whether the user voice request matches a voice command variation associated with a task of the third-party voice-enabled resource, wherein the third-party voice-enabled resource is a specific third-party voice-enabled application, wherein a data structure defines available tasks supported by the voice-controlled digital personal assistant using voice input, the data structure comprising;

information identifying each of the available tasks supported by the voice-controlled digital personal assistant;

information identifying voice command variations supported by the voice-controlled digital personal assistant that define variations of user voice input that will perform the task, wherein at least one of the tasks supports a plurality of different voice command variations; and

information identifying voice command examples supported by the voice-controlled digital personal assistant, wherein the voice command examples are specific examples of user voice input that, if spoken by the user, will activate the task, wherein at least one of the tasks has a plurality of different voice command examples that activate the task;

providing, by the voice-controlled digital personal assistant, a response to the user indicating whether the user voice request matches a voice command variation associated with the task;

receiving a user-initiated command, wherein the user-initiated command identifies the specific third-party voice-enabled application, wherein the user-initiated command instructs the voice-controlled digital personal assistant to delete a voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure, wherein the deleted voice command, and corresponding voice command variations, cannot be performed by user voice input; and

performing the user-initiated command to delete the voice command of the specific third-party voice-enabled application from the data structure, and to delete all of the voice command'"'"'s corresponding voice command variations from the data structure that are associated with the voice command in the data structure.

12. The computer-readable storage medium of claim 11 the operations further comprising:

presenting, by the voice-controlled digital personal assistant, a list of available tasks supported by the third-party voice-enabled resource, wherein the task is one of the list of available tasks.

13. The computer-readable storage medium of claim 11 wherein the user voice request matches a voice command variation associated with the task, and wherein the response provided to the user indicates that the user has successfully spoken the user voice request that would perform the task.

14. The computer-readable storage medium of claim 11 wherein the user voice request matches a voice command variation associated with the task, and wherein providing the response to the user comprises:

providing, by the voice-controlled digital personal assistant, a voice response comprising an emulated response of the third-party voice-enabled resource to the user voice request, wherein the emulated response is obtained from the data structure maintained by the voice-controlled digital personal assistant that defines supported tasks and their associated emulated voice responses, and wherein the voice-controlled digital personal assistant provides the emulated response without involvement of the third-party voice-enabled resource.

15. The computer-readable storage medium of claim 11 wherein the user voice request does not match a voice command variation associated with the task, and wherein providing the response to the user comprises:

providing, by the voice-controlled digital personal assistant, a response comprising one or more voice command examples for performing the task, wherein the one or more voice command examples are obtained from the data structure maintained by the voice-controlled digital personal assistant that defines supported tasks and their associated voice command examples.