Systems, methods, and apparatuses for resuming dialog sessions via automated assistant

• US 10,269,351 B2
• Filed: 05/16/2017
• Issued: 04/23/2019
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Aggarwal, Vikram, Kim, Jung Eun, Binay, Deniz
• Primary Examiner(s): Zhu, Richard Z
• Application Number:
  US15/597,088
• Publication Number:
  US 20180336897A1
• Time in Patent Office: 707 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 9/461 Saving or restoring of program or task context
• CPC Class Codes: G06F 9/485 Task life-cycle, e.g. stopping, restarting, resuming execution G06F9/4881 takes precedence
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, apparatus, systems, and computer-readable media are provided for storing incomplete dialog sessions between a user and an automated assistant in order that the dialog sessions can be completed in furtherance of certain actions. While interacting with an automated assistant, a user can become distracted and not complete the interaction to the point of the automated assistant performing some action. In response, the automated assistant can store the interaction as a dialog session. Subsequently, the user may express interest, directly or indirectly, in completing the dialog session, and the automated assistant can provide the user with a selectable element that, when selected, causes the dialog session to be reopened. The user can then continue the dialog session with the automated assistant in order that the originally intended action can be performed by the automated assistant.

Claims

1. A method implemented by one or more processors, the method comprising:

processing content of a human-to-computer dialog session between a user and an automated assistant application,

wherein the user engages with the automated assistant application using a client device of one or more client devices operated by the user, and

wherein the content of the human-to-computer dialog session includes a request for a task to be performed via the automated assistant application;

determining, based on the processing, that the human-to-computer dialog session did not result in the task being completed via the automated assistant application;

generating, based on determining that the human-to-computer dialog session did not result in the task being completed via the automated assistant application, data that characterizes a state of the human-to-computer dialog session and a priority of the human-to-computer dialog session,

wherein the priority is generated at least partially based on the content of the human-to-computer dialog session;

determining, subsequent to generating the data, that the user is engaged in an activity that is associated with the content; and

causing, in response to determining that the user is engaged in the activity, at least one client device of the one or more client devices to provide a selectable element that, when selected by the user, causes the automated assistant application to operate in furtherance of completing the task and resuming the human-to-computer dialog session,

wherein causing the at least one client device to provide the selectable element includes determining that the human-to-computer dialog session has priority over at least one other human-to-computer dialog session that is assigned another priority.

2. The method of claim 1, further comprising:

generating other data that characterizes a similarity between the activity in which the user is engaged in and the content of the human-to-computer dialog session,

wherein determining that the human-to-computer dialog session has priority over the at least one other human-to-computer dialog session is based on the other data that characterizes the similarity between the activity and the content.

3. The method of claim 1, further comprising:

determining, subsequent to causing the at least one client device of the one or more client devices to provide the selectable element, that the user selected the selectable element; and

causing, in response to determining that the user selected the selectable element, the at least one client device to provide an interface that identifies other content identified by the user during the human-to-computer dialog session.

4. The method of claim 3, wherein causing the at least one client device to provide the interface that identifies the other content includes causing another selectable element to be presented by the at least one client device, wherein the other selectable element includes natural language content that characterizes a value for a parameter of the task to be completed via the automated assistant application.

5. The method of claim 4, wherein the interface includes other natural language content that characterizes a different value for the parameter of the task to be completed via the automated assistant application.

6. The method of claim 1, wherein the activity includes the user interacting with a computing device that is providing a graphical user interface that is presenting natural language content associated with the content of the human-to-computer dialog session.

7. The method of claim 1, wherein the other priority of the at least one other human-to-computer dialog session is based on another selectable element that was previously presented to the user and, when previously presented, the other selectable element was unselected by the user.

8. The method of claim 1, wherein resuming the human-to-computer dialog session causes the automated assistant application to:

provide, as output at the one or more client devices, at least one previous response of the automated assistant application.

9. A system, comprising:

one or more processors; and

memory configured to store instructions that when executed by the one or more processors, cause the one or more processors to perform operations that include;

processing content of a human-to-computer dialog session between user and an automated assistant application,

wherein the user engages with the automated assistant application using a client device of one or more client devices operated by the user, and

wherein the content of the human-to-computer dialog session includes a request for a task to be performed via the automated assistant application;

determining, based on the processing, that the human-to-computer dialog session did not result in the task being completed via the automated assistant application;

generating, based on determining that the human-to-computer dialog session did not result in the task being completed via the automated assistant application, data that characterizes a state of the human-to-computer dialog session and a priority of the human-to-computer dialog session,

wherein the priority is generated at least partially based on the content of the human-to-computer dialog session;

determining, subsequent to generating the data, that the user is engaged in an activity that is associated with the content; and

causing, in response to determining that the user is engaged in the activity, at least one client device of the one or more client devices to provide a selectable element that, when selected by the user, causes the automated assistant application to operate in furtherance of completing the task and resuming the human-to-computer dialog session,

wherein causing the at least one client device to provide the selectable element includes determining that the human-to-computer dialog session has priority over another priority of at least one other human-to-computer dialog session.

10. The system of claim 9, further comprising:

generating other data that characterizes a similarity between the activity in which the user is engaged in and the content of the human-to-computer dialog session,

wherein determining that the human-to-computer dialog session has priority over the at least one other human-to-computer dialog session is based on the other data that characterizes the similarity between the activity and the content.

11. The system of claim 9, further comprising:

determining, subsequent to causing the at least one client device of the one or more client devices to provide the selectable element, that the user selected the selectable element; and

causing, in response to determining that the user selected the selectable element, the at least one client device to provide an interface that identifies other content identified by the user during the human-to-computer dialog session.

12. The system of claim 11, wherein causing the at least one client device to provide the interface that identifies the other content includes causing another selectable element to be presented by the at least one client device, wherein the other selectable element includes natural language that characterizes a value for a parameter of the task to be completed via the automated assistant application.

13. The system of claim 12, wherein the interface includes other natural language content that characterizes a different value for the parameter of the task to be completed via the automated assistant application.

14. The system of claim 9, wherein the other priority of the other human-to-computer dialog session is based on another selectable element that was previously presented to the user and, when previously presented, the other selectable element was unselected by the user.

15. The system of claim 9, wherein the activity includes the user interacting with a computing device that is providing a graphical user interface that is presenting natural language content associated with the content of the human-to-computer dialog session.

16. The system of claim 9, wherein resuming the human-to-computer dialog session causes the automated assistant application to provide, as output at the one or more client devices, at least one previous response of the automated assistant application.

17. A non-transitory computer readable medium configured to store instructions that, when executed by one or more processors, cause the one or more processors to perform operations that include:

processing content of a human-to-computer dialog session between user and an automated assistant application,

wherein the user engages with the automated assistant application using a client device of one or more client devices operated by the user, and

wherein the content of the human-to-computer dialog session includes a request for a task to be performed via the automated assistant application;

determining, based on the processing, that the human-to-computer dialog session did not result in the task being completed via the automated assistant application;

generating, based on determining that the human-to-computer dialog session did not result in the task being completed via the automated assistant application, data that characterizes a state of the human-to-computer dialog session and a priority of the human-to-computer dialog session,

wherein the priority is generated at least partially based on the content of the human-to-computer dialog session;

determining, subsequent to generating the data, that the user is engaged in an activity that is associated with the content; and

causing, in response to determining that the user is engaged in the activity, at least one client device of the one or more client devices to provide a selectable element that, when selected by the user, causes the automated assistant application to operate in furtherance of completing the task and resuming the human-to-computer dialog session,

wherein causing the at least one client device to provide the selectable element includes determining that the human-to-computer dialog session has priority over another priority of at least one other human-to-computer dialog session.

18. The non-transitory computer readable medium of claim 17, further comprising:

generating other data that characterizes a similarity between the activity in which the user is engaged in and the content of the human-to-computer dialog session,

wherein determining that the human-to-computer dialog session has priority over the at least one other human-to-computer dialog session is based on the other data that characterizes the similarity between the activity and the content.

19. The non-transitory computer readable medium of claim 17, further comprising:

determining, subsequent to causing the at least one client device of the one or more client devices to provide the selectable element, that the user selected the selectable element; and

causing, in response to determining that the user selected the selectable element, the at least one client device to provide an interface that identifies other content identified by the user during the human-to-computer dialog session.

20. The non-transitory computer readable medium of claim 19, wherein causing the at least one client device to provide the interface that identifies the other content includes causing another selectable element to be presented by the at least one client device, wherein the other selectable element includes natural language that characterizes a value for a parameter of the task to be completed via the automated assistant application.