Context-based actions

• US 9,646,611 B2
• Filed: 04/30/2015
• Issued: 05/09/2017
• Est. Priority Date: 11/06/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Vogel, Matthew E., Krishna, Om, Yu, Jamie, Thota, Venkateshwar Rao, Seto, Julie C., Yang, Chenguang, Bai, Annie Y., Stepanich, Darron J., Kikin Gil, Erez
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US14/700,319
• Publication Number:
  US 20160133254A1
• Time in Patent Office: 740 Days
• Field of Search: 704275, 704E15022, 704E15044, 704200, 704231, 704243, 704256, 704 9, 704255, 705 2642, 707706, 707E17109, 7079991, 707999102
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A computing device receives voice command inputs from a user. The device obtains a language processing result based on the voice command input. The result includes an intent and a set of arguments. The device also obtains a variety of different types of contextual information. An action is identified based on the intent, the arguments, and the contextual information, and the device then suggests the action by displaying a user selectable input mechanism that can be actuated by the user to perform the action. The device can automatically perform the action as well.

Claims

1. A computing system, comprising:

a processor; and

memory storing instructions executable by the at least one processor, wherein the instructions, when executed, configure the computing system to provide;

a user interface mechanism configured to receive an utterance, send the utterance to a linguistic processing system, and receive a linguistic processing result indicative of an intent and a set of arguments recognized in the utterance;

a context identification system configured to identify a current context of the computing system;

an action identifier configured to identify an action based on the current context and based on the intent in the linguistic processing result; and

a cross-source search component configured to identify content on which the action is to be performed by searching multiple different information sources based on the current context and based on the set of arguments in the linguistic processing result.

2. The computing system of claim 1, wherein the instructions, when executed, configure the computing system to provide:

an action component configured to control a controlled system to take the action on the identified content.

3. The computing system of claim 2 wherein the utterance identifies the content implicitly and wherein the cross-source search component is configured to use the arguments as search terms to search the multiple different information sources to identify a plurality of different search results, each identifying a different item of content.

4. The computing system of claim 3, wherein the instructions, when executed, configure the computing system to provide:

a relevancy generator configured to a relevancy measure for each of the different search results to obtain a given search result.

5. The computing system of claim 4 wherein the relevancy generator is configured to use the arguments and current context to disambiguate otherwise ambiguous search results to obtain the given search result, based on the current context and the arguments.

6. The computing system of claim 4 wherein the linguistic processing system comprises:

a first linguistic processing system disposed on the computing system and including a speech recognition system and a natural language understanding system, the speech recognition system configured to generate a textual representation of the utterance and the natural language understanding system identifying the intent and arguments in a first linguistic processing result, and to further disambiguate otherwise ambiguous arguments based on the current context.

7. The computing system of claim 6 wherein the user interface mechanism is configured to send the utterance to a second linguistic processing system remote from the computing system, the second linguistic processing system configured to generate a second linguistic processing result, the computing system configured to combine the first and second linguistic processing results to obtain a final linguistic processing result.

8. The computing system of claim 4 wherein the context identification system comprises:

a personal identity detector configured to identify a personal identity of the user.

9. The computing system of claim 4 wherein the context identification system comprises:

a proximity detector configured to identify other people or things within a given proximity to the computing system.

10. The computing system of claim 4 wherein the context identification system comprises:

an open application detector configured to identify open applications on the computing system.

11. The computing system of claim 4 wherein the context identification system comprises:

a device detector configured to identify a particular device that the computing system is disposed on.

12. The computing system of claim 4 wherein the context identification system comprises:

a calendar detector configured to identify a particular time and items on a calendar of the user.

13. The computing system of claim 11 wherein the particular device comprises a mobile device.

14. The computing system of claim 11 wherein the context identification system comprises:

a location detector configured to identify a location of the particular device on which the computing system is disposed.

15. The computing system of claim 4 wherein the action identifier is configured to identify the action as one of send, open, schedule a meeting, create, attach, email, set a reminder, share, present, launch and display.

16. A computer-implemented method, comprising:

receiving an utterance;

transmitting the utterance to a linguistic processing system;

receiving a linguistic processing result indicative of an intent and a set of arguments recognized in the utterance;

detecting a context of the computing system;

analyzing the context and the intent in the linguistic processing result to identify an action to take;

searching multiple different information sources based on the context and based on the set of arguments in the linguistic processing result to identify content on which the action is to be performed; and

using a computer processor to automatically control a controlled system to perform the action on the identified content.

17. The computer implemented method of claim 16 and further comprising:

generating a relevancy measure corresponding to each of the different search results; and

ranking the different search results based on the corresponding relevancy measures.

18. The computer implemented method of claim 17 and further comprising:

using the arguments and context to disambiguate otherwise ambiguous search results to obtain the given search result, based on the current context and the arguments.

19. A mobile device, comprising:

a user interface mechanism configured to receive an utterance, send the utterance to a linguistic processing system and receive a linguistic processing result indicative of an intent and a set of arguments recognized in the utterance;

a context identification system configured to identify a context of the computing system;

an action identifier configured to identify an action based on the current context and based on the intent in the linguistic processing result;

a cross-source search component configured to identify content on which the action is to be performed by searching multiple different information sources based on the context and based on the set of arguments in the linguistic processing result; and

an action component configured to control a controlled system, using a processor of the mobile device, to take the action on the identified content.

20. The mobile device of claim 19 wherein the utterance identifies the content implicitly and wherein the cross-source search component is configured to use the arguments as search terms to search the multiple different information sources to identify a plurality of different search results, each identifying a different item of content, and further comprising:

a relevancy generator configured to generate a relevancy measure for each of the different search results to obtain a given search result and to use the arguments and current context to disambiguate otherwise ambiguous search results to obtain the given search result, based on the context and the arguments.