Action generation based on voice data

• US 9,148,741 B2
• Filed: 12/05/2011
• Issued: 09/29/2015
• Est. Priority Date: 12/05/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Gordon, Clif, Woolsey, Kerry D.
• Primary Examiner(s): McFadden, Susan
• Application Number:
  US13/310,962
• Publication Number:
  US 20130144610A1
• Time in Patent Office: 1,394 Days
• Field of Search: 704/275, 704/201
• US Class Current: 1/1
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

An automated technique is discussed for processing audio data and generating one or more actions in response thereto. In particular embodiments, the audio data can be obtained during a phone conversation and post-call actions can be provided to the user with contextually relevant entry points for completion by an associated application. Audio transcription services available on a remote server can be leveraged. The entry points can be generated based on keyword recognition in the transcription and passed to the application in the form of parameters.

Claims

1. A method for generating an action based on a conversation on a mobile device, comprising:

performing by the mobile device;

receiving audio data on the mobile device;

receiving text data on the mobile device, the text data generated in real time from the received audio data;

parsing the text data, wherein the parsing comprises searching for one or more predetermined keywords within the text data generated in real time on the mobile device;

based on the parsed text data and the one or more predetermined keywords found during the searching, generating a list of possible actions to be performed by one or more applications running on the mobile device; and

displaying the list of possible actions on the mobile device.

2. The method of claim 1, wherein the mobile device is a mobile phone.

3. The method of claim 1, further including transmitting the audio data over a network for translation.

4. The method of claim 1, further including monitoring the conversation, dividing the conversation into a series of data chunks of a predetermined time duration, and transmitting the data chunks over a network for translation.

5. The method of claim 1, wherein generating the list includes sending the text data to an application and receiving in response thereto, one or more actions and associated confidence ranking associated with each action, and further including ordering the actions based on the confidence ranking.

6. The method of claim 1, further including translating the conversation from audio to the text data in real time while the conversation is occurring.

7. The method of claim 6, wherein the translating occurs over a network or on the mobile device.

8. The method of claim 1, further including receiving a selection of an item on the list and, in response thereto, launching an application on the mobile device associated with the selected item.

9. The method of claim 8, further including passing the application parameters associated with the item on the list.

10. The method of claim 1, further including receiving a selection of an item on the list and, in response thereto, automatically adding a calendar item associated with the selected item.

11. A computer-readable storage device storing computer-executable instructions operable to cause a mobile computing device to perform a method comprising:

storing a stream of voice data on the mobile computing device during a phone conversation, the phone conversation using the mobile computing device;

based on the voice data, automatically generating at least one action based on parsing text data, the text data corresponding to the voice data, wherein the parsing comprises searching the text data for one or more keywords associated with an application running on the mobile computing device; and

launching the application associated with the at least one action and the one or more keywords.

12. The computer-readable storage device of claim 11, wherein launching the application further includes passing parameters to the application, the parameters having been generated based on the voice data.

13. The computer-readable storage device of claim 11, wherein the method further includes translating the stream of voice data into the text data.

14. The computer-readable storage device of claim 13, wherein translating the stream of voice data includes transmitting the voice data over a network to be translated in real time while the voice data is being received and receiving back the translation into the text data.

15. The computer-readable storage device of claim 13, further including dividing the stream of voice data into chunks, and translating the chunks into the text data.

16. The computer-readable storage device of claim 11, wherein automatically generating at least one action includes:

sending text data, which is a translated version of the stored voice data, to two or more third-party applications;

receiving, in response to the sent text data, a potential action from two or more of the third-party applications; and

generating a list of potential actions.

17. The computer-readable storage device of claim 16, wherein the mobile computing device is a phone and wherein the list of potential actions is generated during a phone conversation.

18. The computer-readable storage device of claim 11, further including using text data, which is a translated version of the stored voice data, in conjunction with locally stored parameters to generate the list of potential actions.

19. A method for generating a list of actions based on a phone conversation, comprising:

performing by a mobile device;

dividing the phone conversation into chunks of voice data of a predetermined duration;

passing the chunks of voice data to a translation service;

receiving, from the translation service, text data associated with the translated voice data;

parsing the text data, wherein the parsing comprises searching the text data for at least one keyword associated with two or more applications running on the mobile computing device;

requesting potential actions from the two or more applications based on the parsed text data;

receiving the potential actions from the two or more applications; and

displaying a list of the potential actions.

20. The method of claim 19, wherein passing the chunks of data to the translation service includes transmitting the chunks of voice data over a network to a server computer.