Customizing actions based on contextual data and voice-based inputs

• US 9,218,819 B1
• Filed: 03/01/2013
• Issued: 12/22/2015
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Stekkelpak, Zoltan, Simonyi, Gyula
• Primary Examiner(s): BRYAR, JEREMIAH A
• Application Number:
  US13/783,145
• Time in Patent Office: 1,026 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Methods and systems are provided for customizing an action. In some implementations, voice input is received from a user and a context is determined from the voice input. Potential contextual data is identified based on the context and the voice input. A level of confidence is determined for an association of the potential contextual data and the context. An action is performed based on the voice input, the potential contextual data, and the level of confidence. The potential contextual data is used to customize the action.

Claims

1. A computer-implemented method comprising:

receiving a voice input from a user;

determining a context from the voice input, wherein the context identifies a subject of the voice input, and wherein the context is associated with an action;

identifying from previously stored data specific to the user, one or more potentially relevant current or near-term events related to the user and the subject of the voice input based on the context and the voice input;

determining a level of confidence for an association of the one or more potentially relevant current or near-term events and the context; and

when the level of confidence is sufficiently high, performing the action based on the voice input, and on the one or more potentially relevant current or near-term events, wherein the one or more potentially relevant current or near-term events are used to customize the action.

2. The method of claim 1, further comprising:

presenting a refining prompt when the level of confidence is sufficiently low.

3. The method of claim 1, wherein the one or more potentially relevant current or near-term events are identified from the group consisting of calendar data associated with the user, social network data associated with the user, email data associated with the user, text message data associated with the user, telephone conversation data associated with the user, internet activity data associated with the user, and any combination thereof.

4. The method of claim 1, wherein identifying the one or more potentially relevant current or near-term events comprises:

building an event model by identifying current and near term events related to the user, wherein the event model comprises data indicative of people, places, and times associated with the events; and

identifying data from the event model that is potentially associated with the context.

5. The method of claim 1, further comprising:

processing the voice input and the context to determine whether there is sufficient information to perform the action; and

determining one or more missing parameters for the action when there is not sufficient information, wherein the potential contextual data comprises previously stored data specific to the user.

6. The method of claim 5, wherein performing the action comprises customizing the one or more missing parameters using the identified current or near-term events.

7. The method of claim 5, further comprising:

ranking the one or more missing parameters; and

presenting a query to refine the voice input based on the ranking when the level of confidence is sufficiently low.

8. The method of claim 5, further comprising:

accessing data associated with other users; and

identifying data that are potentially associated with the one or more missing parameters from the data associated with other users.

9. The method of claim 5, further comprising:

identifying trend in user data associated with the user over time; and

determining information indicative of the one or more missing parameters based on the trends.

10. The method of claim 5, further comprising presenting options for the missing parameters to the user in order to allow the user to select one or more of the options.

11. A system comprising:

one or more computers configured to perform operations comprising;

receiving a voice input from a user;

determining a context from the voice input, wherein the context identifies a subject of the voice input, and wherein the context is associated with an action;

identifying, from previously stored data specific to the user, one or more potentially relevant current or near-term events related to the user and the subject of the voice input based on the context and the voice input;

determining a level of confidence for an association of the one or more potentially relevant current or near-term events and the context; and

when the level of confidence is sufficiently high, performing the action based on the voice input, and on the one or more potentially relevant current or near-term events, wherein the one or more potentially relevant current or near-term events are used to customize the action.

12. The system of claim 11, the operations further comprising:

presenting a refining prompt when the level of confidence is sufficiently low.

13. The system of claim 11, wherein the one or more potentially relevant current or near-term events are identified from the group consisting of calendar data associated with the user, social network data associated with the user, email data associated with the user, text message data associated with the user, telephone conversation data associated with the user, internet activity data associated with the user, and any combination thereof.

14. The system of claim 11 wherein identifying the one or more potentially relevant current or near-term events comprises:

building an event model by identifying current and near term events related to the user, wherein the event model comprises data indicative of people, places, and times associated with the events; and

identifying data from the event model that is potentially associated with the context.

15. The system of claim 11, the operations further comprising:

processing the voice input and the context to determine whether there is sufficient information to perform the action; and

determining one or more missing parameters for the action when there is not sufficient information, wherein the potential contextual data comprises previously stored data specific to the user.

16. The system of claim 15, wherein performing the action comprises customizing the one or more missing parameters using the identified current or near-term events.

17. The system of claim 15, the operations further comprising:

ranking the one or more missing parameters; and

presenting a query to refine the voice input based on the ranking when the level of confidence is sufficiently low.

18. The system of claim 15, the operations further comprising:

accessing data associated with other users; and

identifying data that are potentially associated with the one or more missing parameters from the data associated with the other users.

19. The system of claim 15, the operations further comprising:

identifying trend in user data associated with the user over time; and

determining information indicative of the one or more missing parameters based on the trends.

20. The system of claim 15, further comprising presenting options for the missing parameters to the user in order to allow the user to select one or more of the options.

21. A computer program product encoded on one or more non-transitory computer storage media, the computer program product comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a voice input from a user;

determining a context from the voice input, wherein the context identifies a subject of the voice input, and wherein the context is associated with an action;

identifying, from previously stored data specific to the user, one or more potentially relevant current or near-term events related to the user and the subject of the voice input based on the context and the voice input;

determining a level of confidence for an association of the one or more potentially relevant current or near-term events and the context; and

when the level of confidence is sufficiently high, performing the action based on the voice input, and on the one or more potentially relevant current or near-term events, wherein the one or more potentially relevant current or near-term events are used to customize the action.