Customizing actions based on contextual data and voice-based inputs

• US 10,062,383 B1
• Filed: 11/20/2017
• Issued: 08/28/2018
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Stekkelpak, Zoltan, Simonyi, Gyula
• Primary Examiner(s): Pham, Thierry L
• Application Number:
  US15/818,555
• Time in Patent Office: 281 Days
• Field of Search: 704206, 704250, 704255, 704266, 704275
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Methods and systems are provided for customizing an action. In some implementations, voice input is received from a user and a context is determined from the voice input. Potential contextual data is identified based on the context and the voice input. A level of confidence is determined for an association of the potential contextual data and the context. An action is performed based on the voice input, the potential contextual data, and the level of confidence. The potential contextual data is used to customize the action.

Claims

1. A computer-implemented method comprising:

receiving a voice input from a user that includes a command to perform a specific action, the action having a plurality of parameters, and each parameter specifying information necessary to perform the action;

determining that the voice input is missing values for one or more parameters required to perform the action;

in response to determining that the voice input is missing values for one or more parameters required to perform the action, identifying one or more current or future events related to the user that are relevant to the action, wherein each current or future event is either (i) currently occurring relative to a time when the voice input was received or (ii) will occur in the future relative to the time when the voice input was received;

customizing the action by assigning values to the one or more parameters missing values from data associated with the one or more current or future events; and

performing the customized action.

2. The method of claim 1, wherein identifying one or more current or future events related to the user comprises:

searching a structured collection of data and relationships associated with the events for data associated with the user, wherein the data is indicative of people, places, and times associated with the events.

3. The method of claim 1, further comprising:

identifying an action to be performed in response to the voice input by;

determining a context for the voice input that represents a subject of the voice input; and

determining that the action is associated with the context for the voice input.

4. The method of claim 1, wherein:

identifying one or more current or future events related to the user that are relevant to the action comprises;

identifying user data that is potentially relevant to the action;

determining a level of confidence for an association of one or more potentially relevant current or future events from the identified user data and a context of the voice input; and

identifying one or more current or future events related to the user that are relevant to the action from the potentially relevant events based on the determined level of confidence.

5. The method of claim 4, further comprising, when the level of confidence is sufficiently low, presenting a refining prompt to allow the user to select one or more possible values for the one or more missing parameters, the one or more possible values being determined based on data associated with the one or more current or future events.

6. The method of claim 5, further comprising, when the level of confidence is sufficiently high, assigning values to the one or more missing parameters based on data associated with the one or more current or future events without presenting the refining prompt.

7. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:

receiving a voice input from the user that includes a command to perform a specific action, the action having a plurality of parameters, and each parameter specifying information necessary to perform the action;

determining that the voice input is missing values for one or more parameters required to perform the action;

in response to determining that the voice input is missing values for one or more parameters required to perform the action, identifying one or more current or future events related to the user that are relevant to the action, wherein each current or future event either is (i) currently occurring relative to a time when the voice input was received or (ii) will occur in the future relative to the time when the voice input was received;

customizing the action by assigning values to the one or more parameters missing values from data associated with the one or more current or future events; and

performing the customized action.

8. The system of claim 7, wherein identifying one or more current or future events related to the user comprises:

searching a structured collection of data and relationships associated with the events for data associated with the user, wherein the data is indicative of people, places, and times associated with the events.

9. The system of claim 7, the operations further comprising:

identifying an action to be performed in response to the voice input by;

determining a context for the voice input that represents a subject of the voice input; and

determining that the action is associated with the context for the voice input.

10. The system of claim 7, wherein:

identifying one or more current or future events related to the user that are relevant to the action comprises;

identifying user data that is potentially relevant to the action;

determining a level of confidence for an association of the one or more potentially relevant current or future events from the identified user data and a context of the voice input; and

identifying one or more current or future events related to the user that are relevant to the action from the potentially relevant events based on the determined level of confidence.

11. The system of claim 10, the operations further comprising, when the level of confidence is sufficiently low, presenting a refining prompt to allow the user to select one or more possible values for the one or more missing parameters, the one or more possible values being determined based on data associated with the one or more current or future events.

12. The system of claim 11, the operations further comprising, when the level of confidence is sufficiently high, assigning values to the one or more missing parameters based on data associated with the one or more current or future events without presenting the refining prompt.

13. One or more non-transitory computer storage media encoded with a computer program, the computer program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a voice input from the user that includes a command to perform a specific action, the action having a plurality of parameters, and each parameter specifying information necessary to perform the action;

determining that the voice input is missing values for one or more parameters required to perform the action;

in response to determining that the voice input is missing values for one or more parameters required to perform the action, identifying one or more current or future events related to the user that are relevant to the action, wherein each current or future event is either (i) currently occurring relative to a time when the voice input was received or (ii) will occur in the future relative to the time when the voice input was received;

customizing the action by assigning values to the one or more parameters missing values form data associated with the one or more current or future events; and

performing the customized action.

14. The non-transitory computer storage media of claim 13, the operations further comprising:

identifying an action to be performed in response to the voice input by;

determining a context for the voice input that represents a subject of the voice input; and

determining that the action is associated with the context for the voice input.

15. The non-transitory computer storage media of claim 13, wherein identifying events related to the user comprises:

identifying user data that is potentially relevant to the action;

determining a level of confidence for an association of the one or more potentially relevant current or future events from the identified user data and a context of the voice input; and

identifying one or more current or future events related to the user that are relevant to the action from the potentially relevant events based on the determined level of confidence.

16. The non-transitory computer storage media of claim 15, the operations further comprising, when the level of confidence is sufficiently low, presenting a refining prompt to allow the user to select one or more possible values for the one or more missing parameters, the one or more possible values being determined based on data associated with the one or more current or future events.

17. The non-transitory computer storage media of claim 16, the operations further comprising, when the level of confidence is sufficiently high, assigning values to the one or more missing parameters based on data associated with the one or more current or future events without presenting the refining prompt.