Task assistant

• US 10,783,139 B2
• Filed: 03/06/2013
• Issued: 09/22/2020
• Est. Priority Date: 03/06/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Mauro, David Andrew, Bouvier, Henri, Peters, Stephen Douglas, Dykstra-Erickson, Elizabeth Ann, Daniel, Susan Dawnstarr, Piercy, Aimee, Attayadmawittaya, Paweena, Watson, Andrew Jonathan
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US13/787,729
• Publication Number:
  US 20140258323A1
• Time in Patent Office: 2,757 Days
• Field of Search: 707706, 707723, 707765, 707766, 707769, 707771, 704231, 704251, 704270, 704275
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A method of providing a task assistant to provide an interface to an application is described. The method comprises receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output.

Claims

1. A method comprising:

activating, by a first computing device and for a user, a task assistant, wherein a previous activation of the task assistant has already occurred on a second computing device;

receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;

receiving, by the task assistant and from the user, speech input;

translating, based on the context data, the speech input to a formatted query;

providing the formatted query to an application;

receiving, from the application and based on the formatted query, a first result data;

causing display of the first result data and a filter;

receiving a selection of the filter;

receiving, from the application and based on the filter, a second result data;

causing display of the second result data;

initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and

deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.

2. The method of claim 1, wherein displaying the first result data comprises displaying the first result data in an overlay, displayed on top of an interface of the task assistant.

3. The method of claim 1, further comprising displaying, with the first result data, information indicative of the formatted query.

4. The method of claim 1, further comprising:

enabling the user to interact with the formatted query to modify the formatted query.

5. The method of claim 1, further comprising:

determining, based on the formatted query, a hierarchy of filters, wherein each filter of the hierarchy of filters indicates a constraint used to determine the first result data, and wherein the hierarchy of filters is ordered based on, for each filter of the hierarchy of filters, a measure of restriction associated with the constraint.

6. The method of claim 1, wherein the task assistant context data indicates one or more prior queries associated with the previous activation of the task assistant.

7. The method of claim 1, further comprising:

based on activation of the task assistant, determining the task assistant context data; and

wherein translating the speech input to the formatted query is performed based on the task assistant context data.

8. The method of claim 7, further comprising:

based on activation of the task assistant, initiating the timer that has an expiration period; and

based on determining that the user did not cause another formatted query to be translated on another computing device prior to expiration of the expiration period, deleting the context data.

9. The method of claim 1, further comprising:

sending, to the second computing device, an indication that the task assistant is activated; and

wherein the task assistant context data is a response to the sending of the indication.

10. One or more non-transitory computer-readable media, storing executable instructions that, when executed, cause an apparatus to perform operations comprising:

activating, by a first computing device and for a user, a task assistant, wherein a previous activation of the task assistant has already occurred on a second computing device;

receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;

receiving, by the task assistant and from the user, speech input;

translating, based on the context data, the speech input to a formatted query;

providing the formatted query to an application;

receiving, from the application and based on the formatted query, a first result data;

causing display of the first result data and a filter;

receiving a selection of the filter;

receiving, from the application and based on the filter, a second result data;

causing display of the second result data;

initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and

deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.

11. The one or more non-transitory computer-readable media of claim 10, wherein the operations further comprise translating the input based on a user interface of the application.

12. The one or more non-transitory computer-readable media of claim 10, wherein the operations further comprise translating the input based on the task assistant context data, wherein the task assistant context data is determined after causing the apparatus to activate the task assistant.

13. The one or more non-transitory computer-readable media of claim 10, wherein the selection of the filter is based on a verbal instruction, from the user, to remove the filter from a plurality of filters.

14. The one or more non-transitory computer-readable media of claim 10, wherein the filter indicates a number of results within the first result data to display.

15. An apparatus comprising:

one or more processors; and

memory storing executable instructions, that, when executed by the one or more processors, cause the apparatus to perform operations comprising;

activating, by a first computing device and for a user, a task assistant wherein a previous activation of the task assistant has already occurred on a second computing device;

receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;

receiving, by the task assistant and from the user, speech input;

translating, based on the context data, the speech input to a formatted query;

providing the formatted query to an application;

receiving, from the application and based on the formatted query, a first result data;

causing display of the first result data and a filter;

receiving a selection of the filter;

receiving, from the application and based on the filter, a second result data;

causing display of the second result data;

initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and

deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.

16. The apparatus of claim 15, wherein the operations further comprise based on activation of the task assistant, determining the task assistant context data;

and

translating the speech input to the formatted query based on the task assistant context data.