Providing access to user-controlled resources by automated assistants

• US 10,127,227 B1
• Filed: 05/15/2017
• Issued: 11/13/2018
• Est. Priority Date: 05/15/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Badr, Ibrahim, Adan, Yariv, Santos, Hugo
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/595,004
• Publication Number:
  US 20180329889A1
• Time in Patent Office: 547 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 9/468 Specific access rights for resources, e.g. using capability register
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

Techniques are described herein for allowing automated assistants serving users to access data controlled by others, including resources available to automated assistants serving others. In various implementations, a voice input may be received from a first user. A task request may be recognized from the voice input and analyzed. In response to the analysis, a determination may be made that the task request relates to a second user. An access control list relating to an automated assistant engaged by the second user may be checked to determine whether the first user has appropriate access rights as regards the second user. In response to determining that the first user has appropriate access rights as regards the second user, action may be taken on the task request.

Claims

1. A method comprising:

executing a respective automated assistant client on each of at least two client computing devices communicatively coupled via one or more networks with an automated assistant, wherein the automated assistant is cloud-based and wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services;

receiving a voice input from one of the plurality of users at an input device of a first client computing device operated by the first user;

performing automatic voice recognition on the voice input;

recognizing a task request from an output of the automatic voice recognition;

analyzing the task request to identify that the task request seeks access to user-controlled resources of a second user of the plurality of users, the second user being associated with a mobile second client computing device;

analyzing the task request to identify a geographic constraint and a time constraint imposed by the task request;

checking an access control list relating to the plurality of services of the user-controlled resources engine to determine whether the first user has appropriate access rights as regards the second user for action to be taken on at least a portion of the task request;

determining that the first user has appropriate access rights as regards the second user for action to be taken on the portion of the task request;

reading information from one or more accounts of the second user to which the first user has appropriate access rights including an account of the second user with a location service that makes available, upon request, a position of the second user provided by the mobile second client computing device;

obtaining a current position of the second user from the account of the second user with the location service;

verifying that the current position of the second user satisfies the geographic constraint;

verifying that a current time satisfies the time constraint;

causing the automated assistant to engage in a natural language human-to-computer dialog with the second user via the automated assistant client operating on the mobile second client computing device at the current position and the current time; and

conveying the portion of the task request to the second user via an output component of the mobile second client computing device.

2. The method of claim 1, wherein the access control list further indicates access rights regarding services selected from the group of:

a schedule service, an automated assistant liaison service a shopping list service, and a reminder service.

3. The method of claim 1, wherein checking the access control list comprises determining that the first user is a member of a first group and determining that the first group has appropriate access rights.

4. A cloud-based system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to execute an automated assistant to which a respective automated assistant client on each of at least two client computing devices is communicatively coupled via one or more networks, wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services wherein the automated assistant is configured to perform the following operations:

receiving a voice input from a first user of the plurality of users at an input device of a first client computing device operated by the first user;

performing automatic voice recognition on the voice input;

recognizing a task request from an output of the automatic voice recognition;

analyzing the task request to identify that the task request seeks access to user-controlled resources of a second user of the plurality of users, the second user being associated with a mobile second client computing device;

analyzing the task request to identify a geographic constraint and a time constraint imposed by the task request;

checking an access control list relating to the plurality of services of the user-controlled resources engine to determine whether the first user has appropriate access rights as regards the second user for action to be taken on at least a portion of the task request;

determining that the first user has appropriate access rights as regards the second user for action to be taken on the portion of the task request;

reading information from one or more accounts of the second user to which the first user has appropriate access rights including an account of the second user with a location service that makes available, upon request, a position of the second user provided by the mobile second client computing device;

obtaining a current position of the second user from the account of the second user with the location service;

verifying that the current position of the second user satisfies the geographic constraint;

verifying that a current time satisfies the time constraint;

engaging in a natural language human-to-computer dialog with the second user via the automated assistant client operating on the mobile second client computing device at the current position and the current time; and

conveying the portion of the task request to the second user via an output component of the mobile second client computing device.

5. The system of claim 4, wherein the access control list further indicates access rights regarding services selected from the group of:

a schedule service, an automated assistant liaison service, a shopping list service, and a reminder service.

6. The system of claim 4, wherein checking the access control list comprises determining that the first user is a member of a first group and determining that the first group has appropriate access rights.

7. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to execute an automated assistant that serves a first user by performing the following operations in an environment in which a respective automated assistant client executes on each of at least two client computing devices communicatively coupled via one or more networks with an automated assistant, wherein the automated assistant is cloud-based and wherein the automated assistant is further communicatively coupled to a user-controlled resources engine that includes a plurality of services accessible to a plurality of users of the automated assistant, each of the plurality of users having one or more accounts with one or more of the plurality of services:

receiving a voice input from a first user of the plurality of users at an input device of a first client computing device operated by the first user;

performing automatic voice recognition on the voice input;

recognizing a task request from an output of the automatic voice recognition;

analyzing the task request to identify that the task request seeks access to user-controlled resources of a second user of the plurality of users, the second user being associated with a mobile second client computing device;

analyzing the task request to identify a geographic constraint and a time constraint imposed by the task request;

checking an access control list relating to the plurality of services of the user-controlled resources engine to determine whether the first user has appropriate access rights as regards the second user for action to be taken on at least a portion of the task request;

determining that the first user has appropriate access rights as regards the second user for action to be taken on the portion of the task request;

reading information from one or more accounts of the second user to which the first user has appropriate access rights including an account of the second user with a location service that makes available, upon request, a position of the second user provided by the mobile second client computing device;

obtaining a current position of the second user from the account of the second user with the location service;

verifying that the current position of the second user satisfies the geographic constraint;

verifying that a current time satisfies the time constraint;

causing the automated assistant to engage in a natural language human-to-computer dialog with the second user via the automated assistant client operating on the mobile second client computing device at the current position and the current time; and

conveying the portion of the task request to the second user via an output component of the mobile second client computing device.

8. The at least one non-transitory computer-readable medium of claim 7, wherein the access control list further indicates access rights regarding services selected from the group of:

a schedule service, an automated assistant liaison service, a shopping list service, and a reminder service.

9. The at least one non-transitory computer-readable medium of claim 7, wherein checking the access control list comprises determining that the first user is a member of a first group and determining that the first group has appropriate access rights.