TASK ASSISTANT

• US 20140258323A1
• Filed: 03/06/2013
• Published: 09/11/2014
• Est. Priority Date: 03/06/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Mauro, David Andrew, Bouvier, Henri, Dykstra-Erickson, Elizabeth Ann, Daniel, Susan Dawnstarr, Piercy, Aimee, Attayadmawittaya, Paweena, Peters, Stephen Douglas, Watson, Andrew Jonathan
• Granted Patent: US 10,783,139 B2
• Time in Patent Office: Days
• US Class Current: 707/766
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A method of providing a task assistant to provide an interface to an application is described. The method comprises receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output.

Claims

1. A method of providing a task assistant to provide an interface to an application, the method comprising:

receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input;

interpreting the input, and providing a formatted query to the application;

receiving data from the application in response to the query; and

providing a response to the user through multimodal output including a plurality of;

speech output, text output, non-speech audio output, haptic output, and visual non-text output.

2. The method of claim 1, wherein the response is provided through an overlay, displayed on top of a display of the application.

3. The method of claim 2, wherein the task assistant is displayed as a semi-transparent overlay on the application.

4. The method of claim 1, wherein providing the response to the user further includes displaying the interpreted input to the user, such that information about the query is provided with the response.

5. The method of claim 4, further comprising:

enabling a user to interact with the interpreted input display, to modify the query.

6. The method of claim 1, further comprising:

providing a soft invocation for the task assistant.

7. The method of claim 6, wherein the soft invocation comprises one of:

a button in a menu bar of the application, a button in a title bar of the application, a button in a header space of the application, a transparent ghost button floating over standard display of the application, a touch-input pattern, a movement input pattern, and an audio wake-up command.

8. The method of claim 1, wherein interpreting the input comprises:

determining whether the input is unambiguous, and when the input is complete, generating the formatted query for the application.

9. The method of claim 8, further comprising:

when the input is not unambiguous, requesting clarification from the user, to enable construction of a complete query.

10. A task assistant to provide an interface to an application, the task assistant comprising:

an input user interface to receive input from a user through multimodal input including a plurality of speech input, typing input, and touch input;

a command interpreter to interpret the input, and providing a formatted query to the application;

a communication logic to receive data from the application in response to the query; and

an output user interface to provide a response to the user through multimodal output including a plurality of;

speech output, text output, non-speech audio output, haptic output, and visual non-text output.

11. The task assistant of claim 10, wherein the response is provided through an overlay, displayed on top of a display of the application.

12. The task assistant of claim 10, wherein the task assistant is displayed as a semi-transparent overlay on the application.

13. The task assistant of claim 10, wherein providing the response to the user further includes displaying the interpreted input to the user, such that information about the query is provided with the response.

14. The task assistant of claim 13, further comprising:

the input user interface further to enable a user to interact with the interpreted input display, to modify the query.

15. The task assistant of claim 10, further comprising:

a soft invoke logic to provide a soft invocation for the task assistant.

16. The task assistant of claim 15, wherein the soft invocation comprises one of:

a button in a menu bar of the application, a button in a title bar of the application, a button in a header space of the application, a transparent ghost button floating over standard display of the application, a touch-input pattern, a movement input pattern, and an audio wake-up command.

17. The task assistant of claim 10, wherein interpreting the input comprises:

the command interpreter to determine whether the input is unambiguous, and when the input is complete, generating the formatted query for the application.

18. The task assistant of claim 17, further comprising:

when the input is not unambiguous, the system requesting clarification from the user, to enable construction of a complete query.

19. A task assistant system to improve interaction with an underlying application, the task assistant comprising:

a client application comprising;

an input user interface to receive input from a user through multimodal input including a plurality of speech input, typing input, and touch input;

an output user interface to provide a response to the user through multimodal output including a plurality of;

speech output, text output, non-speech audio output, haptic output, and visual non-text output; and

a communication logic to enable the client application to communicate with other elements of the system;

a multimodal input interpreter to receive the input from the client application, the multimodal input interpreter comprising;

a command interpreter to interpret the input, and provide a formatted query to the underlying application;

a communication logic to receive data from the application in response to the query, and send the data to the client application for display in the output user interface.

20. The task assistant system of claim 19, further comprising:

a soft invoke logic to provide a soft invocation of the client application.