Selection of computational agent for task performance

• US 10,853,747 B2
• Filed: 11/16/2017
• Issued: 12/01/2020
• Est. Priority Date: 10/03/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wang, Bo, Zhong, Lei, James, Barnaby John, Krishnakumaran, Saisuresh, Stets, Robert, Caprita, Bogdan, Nygaard, Valerie
• Primary Examiner(s): Pullias, Jesse S
• Application Number:
  US15/815,353
• Publication Number:
  US 20180096283A1
• Time in Patent Office: 1,111 Days
• Field of Search: 704231-257, 704270-275
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

An example method includes receiving, by a computational assistant executing at one or more processors, a representation of an utterance spoken at a computing device; identifying, based on the utterance, a task to be performed; determining a capability level of a first party (1P) agent to perform the task; determining capability levels of respective third party (3P) agents of a plurality of 3P agents to perform the task; responsive to determining that the capability level of the 1P agent does not satisfy a threshold capability level, that a capability level of a particular 3P agent of the plurality of 3P agents is a greatest of the determined capability levels, and that the capability level of the particular 3P agent satisfies the threshold capability level, selecting the particular 3P agent to perform the task; and performing one or more actions determined by the selected agent to perform the task.

Claims

1. A method comprising:

receiving, by a computational assistant executing at one or more processors, a representation of an utterance spoken at a computing device;

identifying, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determining a capability level of a first party agent to perform the task;

determining whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level;

determining capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein determining that the capability level of the first party agent does not satisfy the threshold capability level comprises determining that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determining whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level;

selecting the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein determining that the particular capability level of the particular third party agent satisfies the threshold capability level comprises determining that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

causing the selected agent to perform the task.

2. The method of claim 1, wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the method further comprising:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level;

determining additional capability levels of additional respective third party agents, of a plurality of non-configured third party agents to perform the task;

determining whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level;

selecting the particular non-configured third party agent, from among the plurality of non-configured third party agents to perform the task.

3. The method of claim 1, wherein the one or more processors are included in the computing device.

4. The method of claim 1, wherein the one or more processors are included in a computing system.

5. A computing device comprising:

at least one processor; and

at least one memory comprising instructions that when executed, cause the at least one processor to execute an assistant configured to;

receive a representation of an utterance spoken at one or more microphones operably connected to the computing device;

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level;

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level;

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.

6. The device of claim 5, wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the assistant further configured to:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level;

determine additional capability levels of additional respective third party agents of a plurality of non-configured third party agents to perform the task;

determine whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level;

select the particular non-configured third party agent, from among of the plurality of non-configured third party agents to perform the task.

7. The device of claim 5, wherein the assistant is further configured to:

select a first party agent of the one or more first party agents responsive to determining that satisfaction of the utterance needs special assistance.

8. A computing system comprising:

one or more communication units;

at least one processor; and

at least one memory comprising instructions that when executed, cause the at least one processor to execute an assistant configured to;

receive, from a computing device and via the one or more communication units, a representation of an utterance spoken at the computing device;

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level;

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level;

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.

9. The system of claim 8, wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the assistant further configured to:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level;

determine additional capability levels of additional respective third party agents of a plurality of non-configured third party agents to perform the task;

determine whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level;

select the particular non-configured third party agent, from among of the plurality of non-configured third party agents to perform the task.

10. The system of claim 8, wherein the assistant is further configured to:

select a first party agent of the one or more first party agents responsive to determining that satisfaction of the utterance needs special assistance.

11. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to execute an assistant configured to:

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level;

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level;

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.