Providing prompt in an automated dialog session based on selected content of prior automated dialog session

• US 10,446,144 B2
• Filed: 11/21/2016
• Issued: 10/15/2019
• Est. Priority Date: 11/21/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Aggarwal, Vikram, Lim, Yew Jin, Goldstein, Adam
• Primary Examiner(s): Kim, Jonathan C
• Application Number:
  US15/357,733
• Publication Number:
  US 20180144743A1
• Time in Patent Office: 1,058 Days
• Field of Search: None
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Methods, apparatus, and computer readable media related to soliciting feedback from a user regarding one or more content parameters of a suggestion or other content provided by the automated assistant. The user'"'"'s feedback may be used to influence future suggestions and/or other content subsequently provided, by the automated assistant in future dialog sessions, to the user and/or to other users. In some implementations, content is provided to a user by an automated assistant in a dialog session between the user and the automated assistant—and the automated assistant provides a prompt that solicits user feedback related to the provided content in a future dialog session between the user and the automated assistant. In some of those implementations, the prompt is provided following input from the user and/or output from the automated assistant, in the future dialog session, that is unrelated to the content provided in the previous dialog session.

Claims

1. A method implemented by one or more processors, comprising:

as part of a dialog session between a user and an automated assistant implemented by one or more of the processors;

receiving natural language input that is based on user interface input provided by the user via a user interface input device, and

providing, as a reply by the automated assistant to the natural language input, content that is responsive to the natural language input,

wherein the content is provided for presentation to the user via a user interface output device,

wherein the content includes at least one content parameter selected by the automated assistant from a plurality of candidate content parameters, and

wherein the content is a suggestion to be acted upon by the user in the future;

determining that the user actually acted upon the suggestion after providing the content that includes the content parameter, wherein determining that the user actually acted upon the suggestion is based on both;

location data from a mobile client device of the user, and transaction data associated with the user;

as part of an additional dialog session between the user and the automated assistant that is temporally separated from the dialog session and that is initiated by the user with additional natural language input that is unrelated to the content of the dialog session;

providing a prompt that solicits feedback on the selected content parameter,

wherein the prompt is provided for presentation to the user via the user interface output device or an additional user interface output device,

wherein the prompt is generated to solicit feedback on the content parameter based on the content parameter being previously selected by the automated assistant and based on the content, that includes the at least one content parameter and that is the suggestion to be acted upon by the user in the future, being previously provided for presentation to the user as part of the dialog session, and

wherein providing the prompt is based on determining that the user acted upon the suggestion, and

receiving additional input that is responsive to the prompt, the additional input being based on additional user interface input provided by the user via the user interface input device or an additional user interface input device;

using the additional input to alter a value stored in association with the content parameter, wherein the value influences future providing, by the automated assistant, of further content that includes the content parameter; and

using the value, as altered by the additional input, in selecting a further suggestion to provide to the user in response to further natural language input of the user in a further dialog session between the user and the automated assistant.

2. The method of claim 1, further comprising, as part of the additional dialog session:

providing additional dialog session output that is responsive to the additional natural language input and that is also unrelated to the content of the dialog session;

wherein providing the prompt occurs following providing the additional dialog session output.

3. The method of claim 2, further comprising:

determining that one or more criteria are satisfied by at least one of;

the additional natural language input, and the additional dialog session output;

wherein providing the prompt is further based on determining that the criteria are satisfied.

4. The method of claim 3, wherein the criteria comprises at least one of:

the additional dialog session output being of a certain semantic type; and

an occurrence in the additional dialog session output of at least one n-gram of a set of criteria n-grams.

5. The method of claim 1, wherein the user interface input device or the additional user interface input device used to generate the additional input that is responsive to the prompt includes a microphone, and further comprising:

based on providing the prompt, preemptively activating at least one component configured to process the user interface input provided via the microphone.

6. The method of claim 1, further comprising, as part of the additional dialog session:

prior to providing the prompt, providing additional dialog session output;

wherein providing the prompt occurs following providing the additional dialog session output.

7. The method of claim 1, further comprising:

identifying an additional content parameter provided to the user as part of an additional prior dialog session between the user and the automated assistant; and

determining, based on one or more criteria, to provide the prompt based on the content parameter instead of an alternative prompt that is based on the additional content parameter.

8. The method of claim 7, wherein the one or more criteria comprise corresponding temporal proximities of the providing of the content parameter and of the providing of the additional content parameter.

9. The method of claim 7, wherein the one or more criteria comprise semantic types assigned to the content parameter and the additional content parameter.

10. The method of claim 1, wherein the suggestion is for a particular physical location and for a particular item consumable at the particular physical location, and wherein the content parameter identifies the particular item.

11. The method of claim 1, wherein the suggestion is for a particular physical location, and wherein the content parameter identifies a category to which the particular physical location belongs.

12. The method of claim 1, wherein the input and the additional input are both generated via the user interface input device and wherein the content and the prompt are both provided for presentation via the user interface output device.

13. The method of claim 12, wherein the user interface input device includes a microphone of a single device and the user interface output device includes a speaker of the single device.

14. The method of claim 1, wherein:

the user interface input is generated via the user interface input device of a first computing device,

the content is provided for presentation via the user interface output device of the first computing device,

the additional natural language input is voice-based natural language input generated via a microphone of an additional computing device; and

the prompt is provided for presentation via the additional user interface output device of the additional computing device based on determining that the voice-based natural language input conforms to a voice profile of the user.

15. A system, comprising:

memory storing instructions;

one or more processors executing the instructions stored in the memory to;

as part of a dialog session between a user and an automated assistant implemented by one or more of the processors;

receive natural language input that is based on user interface input provided by the user via at least one microphone, and

provide, as a reply by the automated assistant to the natural language input, content that is responsive to the natural language input,

wherein the content is provided for presentation to the user via at least one speaker,

wherein the content includes at least one content parameter selected by the automated assistant from a plurality of candidate content parameters, and

wherein the content is a suggestion to be acted upon by the user in the future;

determine that the user actually acted upon the suggestion after providing the content that includes the content parameter, wherein determining that the user actually acted upon the suggestion is based on both;

location data from a mobile client device of the user, and transaction data associated with the user;

as part of an additional dialog session between the user and the automated assistant that is temporally separated from the dialog session and that is initiated by the user with additional natural language input that is unrelated to the content of the dialog session;

provide a prompt that solicits feedback on the selected content parameter,

wherein the prompt is provided for presentation to the user via the at least one speaker or an additional user interface output device of an additional computing device,

wherein the prompt is generated to solicit feedback on the content parameter based on the content parameter being previously selected by the automated assistant and based on the content, that includes the at least one content parameter and that is the suggestion to be acted upon by the user in the future, being previously provided for presentation to the user as part of the dialog session, and

wherein providing the prompt is based on determining that the user acted upon the suggestion, and

receive additional input that is responsive to the prompt, the additional input being based on additional user interface input provided by the user via the at least one microphone or an additional user interface input device of the additional computing device;

use the additional input to alter a value stored in association with the content parameter, wherein the value influences future providing, by the automated assistant, of further content that includes the content parameter; and

use the value, as altered by the additional input, in selecting a further suggestion to provide to the user in response to further natural language input of the user in a further dialog session between the user and the automated assistant.

16. The system of claim 15, wherein in executing the instructions one or more of the processors are further to, as part of the additional dialog session:

provide additional dialog session output that is responsive to the additional natural language input and that is also unrelated to the content of the dialog session;

wherein providing the prompt occurs following providing the additional dialog session output.

17. The system of claim 15, wherein in executing the instructions one or more of the processors are further to:

based on providing the prompt, preemptively activate at least one component configured to process the additional input provided by the user via the at least one microphone.

18. The system of claim 15, wherein in executing the instructions one or more of the processors are further to, as part of the additional dialog session:

prior to providing the prompt, provide additional dialog session output;

wherein providing the prompt occurs following providing the additional dialog session output.

19. The system of claim 15, wherein the suggestion is for a particular physical location, and wherein the content parameter identifies at least one of:

a particular item consumable at the particular physical location, and a category to which the particular physical location belongs.