Extending digital personal assistant action providers

• US 10,192,549 B2
• Filed: 04/01/2015
• Issued: 01/29/2019
• Est. Priority Date: 11/28/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kannan, Vishwac Sena, Uzelac, Aleksandar, Hwang, Daniel J., Chambers, Robert L., Pinch, David, Siddall, Zachary Thomas John
• Primary Examiner(s): Yang, Qian
• Application Number:
  US14/676,603
• Publication Number:
  US 20160155442A1
• Time in Patent Office: 1,399 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

An electronic device can receive user input via voice or text that includes tasks to be performed. A digital personal assistant infrastructure service can control to which registered action provider the task is assigned. Per-task action provider preferences can be stored. If a preferred action provider is not able to complete the task, the task can still be performed by a registered action provider that has appropriate capabilities. Machine learning can determine a user'"'"'s preferences. Resource conservation and effective user interaction can result.

Claims

1. A method of controlling which action provider out of a plurality of different registered action providers performs a task, the method comprising:

receiving a spoken utterance detected by a microphone;

performing speech recognition on the spoken utterance, the speech recognition recognizing a task in the spoken utterance; and

responsive to and after recognizing the task in the spoken utterance and based on stored registrations for the different registered action providers and stored task-to-provider association preferences, assigning the task in the spoken utterance to a particular action provider out of the different registered action providers, wherein a plurality of the different registered action providers are currently registered for the task in the spoken utterance and the plurality of different registered action providers currently registered for the task in the spoken utterance comprise a default action provider registered for the task in the spoken utterance and a secondary action provider registered for the task in the spoken utterance, and wherein assigning the task in the spoken utterance comprises choosing, from the plurality of different registered action providers currently registered for the task in the spoken utterance, the particular action provider to perform the task in the spoken utterance and sending instructions to perform the task in the spoken utterance to the particular action provider;

wherein choosing the particular action provider to perform the task in the spoken utterance supports explicit override of the default registered action provider for the task based on an explicit indication of a name of the secondary action provider in the spoken utterance, even though the default action provider and the secondary action provider are both currently registered for the task.

2. The method of claim 1 further comprising:

determining that a prerequisite for an action provider is not satisfied; and

responsive to determining that the prerequisite for the action provider is not satisfied, identifying the secondary action provider for the task, and using the secondary action provider as the particular action provider for performing the task.

3. The method of claim 2 wherein:

the prerequisite specifies that a network connection is present.

4. The method of claim 1 wherein:

the task-to-provider association preferences indicate a preferred action provider for the task; and

choosing the particular action provider to perform the task comprises choosing the preferred action provider for the task.

5. The method of claim 1, wherein the assigning assigns the task to the default action provider, the method further comprising:

receiving an utterance comprising an explicit indication of a non-default action provider also registered for the task and a second instance of the task, wherein the explicit indication of the non-default action provider comprises a name of the non-default action provider; and

responsive to recognizing the explicit indication of the non-default action provider, sending instructions to perform the second instance of the task to the non-default action provider, even though the default action provider is also registered for the task, wherein the non-default action provider is different from the default action provider.

6. The method of claim 1 further comprising:

based on a stored registration for the particular action provider, determining that a parameter slot for the task is missing; and

before sending instructions to the particular action provider, prompting for a parameter value to fill the parameter slot;

receiving another spoken utterance;

performing speech recognition on the other spoken utterance, the speech recognition recognizing a spoken parameter value; and

submitting the spoken parameter value to the particular action provider when assigning the task to the particular action provider.

7. The method of claim 1 wherein:

the speech recognition further recognizes a parameter value in the spoken utterance; and

assigning the task to the particular action provider further comprises sending the parameter value to the particular action provider.

8. The method of claim 1 further comprising:

registering the particular action provider with a digital personal assistant infrastructure service, wherein the registering comprises storing an indication of one or more actions supported by the particular action provider.

9. The method of claim 1 wherein:

the plurality of registered action providers comprise a non-native action provider and a native action provider.

10. The method of claim 1 further comprising:

via machine learning, determining that the particular action provider is a preferred action provider for the task; and

changing the default action provider for the task to be the particular action provider.

11. The method of claim 1 wherein:

performing speech recognition on the spoken utterance comprises recognizing, in the utterance, a contact name of a contact;

the task comprises a communication task; and

assigning the particular action provider to perform the task comprises initiating communication with the contact via the communication task with the particular action provider.

12. The method of claim 11 further comprising:

via machine learning, determining that the particular action provider is a preferred action provider for the task and contact; and

changing the default action provider for the task to be the particular action provider when communicating with the contact.

13. The method of claim 1 further comprising:

responsive to determining that no registered action provider supports the task, identifying an action provider that supports the task, installing the action provider that supports the task, and registering the action provider that supports the task.

14. The method of claim 1 wherein:

a stored registration for the particular action provider indicates that a message is to be presented when a current location falls within a boundary; and

the method further comprises, responsive to determining that the current location falls within the boundary, presenting the message.

15. The method of claim 1 further comprising:

performing the task with the particular action provider.

16. An electronic device comprising:

one or more processors;

memory storing a plurality of stored registrations for registered action providers, wherein the stored registrations comprise a stored registration for a particular registered action provider indicating a calling task supported by the particular registered action provider, wherein a plurality of the registered action providers are registered for the calling task;

memory storing a plurality of stored preferences for registered action providers, wherein the stored preferences comprise a preference indicating a preferred registered action provider as preferred for a particular calling task, and a second registered action provider is also registered for the particular calling task;

memory storing computer-executable instructions that, when executed, cause the one or more processors to;

recognize a calling task in a spoken utterance received by a microphone of the electronic device or a typed command received by text input hardware of the electronic device; and

control to which action provider the calling task is directed for completion based on the plurality of stored registrations and the stored preferences, and wherein directing the calling task comprises choosing, from the registered action providers registered for the calling task, the particular registered action provider to perform the calling task and sending instructions to perform the calling task to the particular registered action provider;

wherein;

explicit override of the preferred registered action provider for the calling task is supported based on an explicit indication of a name of the second registered action provider in a received utterance or received typed text, even though the preferred registered action provider and the second registered action provider are both currently registered for the task in the spoken utterance.

17. The electronic device of claim 16 wherein:

the stored registration for the preferred registered action provider comprises a prerequisite condition for the calling task supported by the preferred registered action provider; and

the computer-executable instructions comprise instructions that, when executed, cause the one or more processors to;

evaluate the prerequisite condition and choose a different action provider if the prerequisite condition is not met.

18. The electronic device of claim 17 wherein:

the prerequisite condition specifies that a data connection is present.

19. One or more memories, magnetic storage media, or optical storage media comprising machine-executable instructions causing an electronic device to perform a method controlling which communication action provider out of a plurality of registered communication action providers perform a communication task supported by a digital personal assistant infrastructure service, the method comprising:

performing speech recognition on an utterance, the speech recognition comprising recognizing the supported communication task and a contact name in the utterance; and

based on stored registrations for the registered communication action providers, stored action-to-provider association preferences, and availability of contact information for initiating the communication task for the contact name, choosing a particular registered communication action provider, and initiating communication with a contact identified by the contact name using the chosen particular registered communication action provider via the supported communication task;

wherein choosing a particular registered communication action provider comprises initially considering a first communication action provider, but choosing a second communication action provider responsive to determining that availability of contact information for the first communication action provider is missing.

20. The one or more memories, magnetic storage media, or optical storage media of claim 19 wherein the method further comprises:

specifying a contract registration for the second communication action provider for a contract associated with the communication task, wherein the contract registration extends a virtual digital assistant.