Providing access to user-controlled resources by automated assistants

• US 10,685,187 B2
• Filed: 10/10/2018
• Issued: 06/16/2020
• Est. Priority Date: 05/15/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Badr, Ibrahim, Adan, Yariv, Santos, Hugo
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US16/157,017
• Publication Number:
  US 20190042564A1
• Time in Patent Office: 615 Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 9/468 Specific access rights for resources, e.g. using capability register
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

Techniques are described herein for allowing automated assistants serving users to access data controlled by others, including resources available to automated assistants serving others. In various implementations, a voice input may be received from a first user. A task request may be recognized from the voice input and analyzed. In response to the analysis, a determination may be made that the task request relates to a second user. An access control list relating to an automated assistant engaged by the second user may be checked to determine whether the first user has appropriate access rights as regards the second user. In response to determining that the first user has appropriate access rights as regards the second user, action may be taken on the task request.

Claims

1. A cloud-based system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to execute an automated assistant to which a respective automated assistant client on each of a plurality of client computing devices is communicatively coupled via one or more networks, wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services, wherein the automated assistant is configured to:

receive a voice input provided by a first user of the plurality of users at an input device of a first client computing device of the plurality of client computing devices that is operated by the first user, wherein the voice input from the first user is directed to the automated assistant client executing on the first client computing device;

perform automated voice recognition on the voice input;

recognize a task request from an output of the automated voice recognition;

analyse the task request;

based on the analysis, determine that the task request relates to polling multiple other users of the plurality of users for information pertaining to an upcoming event and checking an access control list relating to the multiple other users to determine whether the first user has appropriate access rights as regards the multiple other users for action to be taken on at least part of the task request;

in response to the determination that the first user has appropriate access rights as regards the multiple other users, take action on at least part of the task request, wherein taking action on at least part of the task request causes the automated assistant clients operating on multiple other client computing devices operated by the multiple other users to engage the multiple other users in respective natural language human-to-computer dialogs, and wherein the respective natural language human-to-computer dialogs solicit free-form natural language responses to the polling;

collect free-form natural language responses to the polling provided by the multiple other users to their respective automated assistant clients;

generate a natural language statement that summarizes the collected free-form natural language polling responses; and

output, by the automated assistant client installed on the first client computing device or another client computing device operated by the first user, the natural language statement.

2. The system of claim 1, wherein the access control list indicates access rights regarding multiple services with which the plurality of other users have respective accounts.

3. The system of claim 1, wherein the access control list indicates access rights regarding services selected from the group of:

a schedule service, an automated assistant liaison service, a location service, a shopping list service, and a reminder service.

4. The system of claim 1, wherein checking the access control list to determine whether the first user has appropriate access rights as regards the plurality of other users comprises determining that the first user is a member of a first group and determining that the first group has appropriate access rights.

5. The system of claim 1, wherein the natural language statement is output to the first user in response to a determination that all of the multiple other users have provided responses to the polling.

6. The system of claim 1, wherein the natural language statement is output to the first user in response to a determination that the event is to occur a predetermined time interval in the future.

7. The system of claim 1, wherein at least one of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided audibly.

8. The system of claim 7, wherein another of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided visually.

9. A method implemented using one or more processors and comprising:

executing an automated assistant to which a respective automated assistant client on each of a plurality of client computing devices is communicatively coupled via one or more networks, wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services, wherein the automated assistant is configured to;

receiving a natural language input provided by a first user of the plurality of users at an input device of a first client computing device of the plurality of client computing devices that is operated by the first user, wherein the natural language input from the first user is directed to the automated assistant client executing on the first client computing device;

performing semantic processing on the natural language input to recognize a task request;

analyzing the task request;

based on the analyzing, determining that the task request relates to polling multiple other users of the plurality of users for information pertaining to an upcoming event and checking an access control list relating to the multiple other users to determine whether the first user has appropriate access rights as regards the multiple other users for action to be taken on at least part of the task request;

in response to the determination that the first user has appropriate access rights as regards the multiple other users, taking action on at least part of the task request, wherein taking action on at least part of the task request causes the automated assistant clients operating on multiple other client computing devices operated by the multiple other users to engage the multiple other users in respective natural language human-to-computer dialogs, and wherein the respective natural language human-to-computer dialogs solicit free-form natural language responses to the polling;

collecting free-form natural language responses to the polling provided by the multiple other users to their respective automated assistant clients;

generating a natural language statement that summarizes the collected free-form natural language polling responses; and

outputting, by the automated assistant client installed on the first client computing device or another client computing device operated by the first user, the natural language statement.

10. The method of claim 9, wherein the access control list indicates access rights regarding multiple services with which the plurality of other users have respective accounts.

11. The method of claim 9, wherein checking the access control list to determine whether the first user has appropriate access rights as regards the plurality of other users comprises determining that the first user is a member of a first group and determining that the first group has appropriate access rights.

12. The method of claim 9, wherein the natural language statement is output to the first user in response to a determination that all of the multiple other users have provided responses to the polling.

13. The method of claim 9, wherein the natural language statement is output to the first user in response to a determination that the event is to occur a predetermined time interval in the future.

14. The method of claim 9, wherein at least one of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided audibly.

15. The system of claim 14, wherein another of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided visually.

16. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:

executing an automated assistant to which a respective automated assistant client on each of a plurality of client computing devices is communicatively coupled via one or more networks, wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services, wherein the automated assistant is configured to;

receiving a natural language input provided by a first user of the plurality of users at an input device of a first client computing device of the plurality of client computing devices that is operated by the first user, wherein the natural language input from the first user is directed to the automated assistant client executing on the first client computing device;

performing semantic processing on the natural language input to recognize a task request;

analyzing the task request;

based on the analyzing, determining that the task request relates to polling multiple other users of the plurality of users for information pertaining to an upcoming event and checking an access control list relating to the multiple other users to determine whether the first user has appropriate access rights as regards the multiple other users for action to be taken on at least part of the task request;

in response to the determination that the first user has appropriate access rights as regards the multiple other users, taking action on at least part of the task request, wherein taking action on at least part of the task request causes the automated assistant clients operating on multiple other client computing devices operated by the multiple other users to engage the multiple other users in respective natural language human-to-computer dialogs, and wherein the respective natural language human-to-computer dialogs solicit free-form natural language responses to the polling;

collecting free-form natural language responses to the polling provided by the multiple other users to their respective automated assistant clients;

generating a natural language statement that summarizes the collected free-form natural language polling responses; and

outputting, by the automated assistant client installed on the first client computing device or another client computing device operated by the first user, the natural language statement.

17. The at least one non-transitory computer-readable medium of claim 16, wherein the natural language statement is output to the first user in response to a determination that all of the multiple other users have provided responses to the polling.

18. The at least one non-transitory computer-readable medium of claim 16, wherein the natural language statement is output to the first user in response to a determination that the event is to occur a predetermined time interval in the future.

19. The at least one non-transitory computer-readable medium of claim 16, wherein at least one of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided audibly.

20. The at least one non-transitory computer-readable medium of claim 19, wherein another of the respective natural language human-to-computer dialogs that solicits a free-form natural language response to the polling is provided visually.